# DataLad: git and git-annex turned into a distributed dataset system > DataLad is a Python command line tool that stands on git and git-annex to manage code, data and the relationship between them as datasets that can be installed from remote sources. It suits research groups sharing data under real permission constraints, and it is the wrong shape for a team that just needs a backup of files. **datalad/datalad** — Keep code, data, containers under control with git and git-annex - Repository: https://github.com/datalad/datalad - Website: http://datalad.org - Stars: 669 · Forks: 130 - Language: Python - License: NOASSERTION - Published: 2026-10-21 · Updated: 2026-10-21 · Language: en - Canonical page: https://hysenlabs.com/projects/datalad-datalad ## A distribution system for data you are not allowed to store DataLad's stated purpose is to make data management and data distribution more accessible. The way it does that is by standing on the shoulders of Git and Git-annex to deliver a decentralized system for data exchange, and this includes automated ingestion of data from online portals and exposing it in readily usable form as Git and git-annex repositories, or datasets. The last clause of that sentence is the one that matters most. The README states that the actual data storage and permission management remains with the original data providers. So DataLad is not a place to put your data. It is a mechanism for describing where your data lives and for making it usable at a new location, including at a site that is not allowed to see the whole thing. A neuroimaging study, for example, might hold consent-restricted data at one institution, publish a derivative dataset openly, and let collaborators install the derivative and, if their credentials allow, the restricted data as well. The model is decentralized in the git sense. There is no central server that must be operated, and no account to create. The audience is research groups, and specifically the ones that already live in a git and git-annex workflow and have a real problem of distributing data under uneven permission constraints. The README also carries a JOSS DOI badge, a Zenodo DOI badge and an RRID identifier, and the topics list includes usable. The project homepage is datalad.org, with the full documentation at docs.datalad.org and a hands-on crash course in the handbook at handbook.datalad.org. ## Installation assumes git-annex is already there The install instructions in the README are unusually careful about prerequisites, and the prerequisite that matters is git-annex. For pip installs the README says to make sure you install a recent version of git-annex first, and only afterwards install the latest version of datalad from PyPI. It recommends a dedicated virtual environment: ```bash # Create and enter a new virtual environment (optional) virtualenv --python=python3 ~/env/datalad . ~/env/datalad/bin/activate # Install from PyPI pip install datalad ``` Distribution packages exist for the major platforms. On Debian-based systems the README recommends enabling the NeuroDebian repository, and then a single command installs it: ```bash apt-get install datalad ``` On Gentoo-based systems, the ::science overlay is the recommended route, using emerge datalad. For other Linux systems and for macOS, conda is offered, with the most recently released version from conda-forge and release candidates from a separate label: ```bash conda install -c conda-forge datalad ``` The pip install gives you the core functionality for managing datasets, and the README notes that additional installation schemes exist for an enhanced install. Two details in the dependency list are worth noticing for an evaluator. The core dependencies include annexremote, which is the Python client for remote git-annex repositories, along with keyring and keyrings.alt for credential handling, boto3 and requests for downloaders, patool for archive handling, and python-gitlab for the publish functionality. That is a real dependency surface, and it is the reason the README insists on git-annex being present: without the annex protocol underneath, the tool is only half a system. The Python requirement is 3.10 or newer, with classifiers listing 3.10 through 3.15, and the project is marked as production stable. ## Extensions are separate packages, and they pull DataLad in The extension mechanism is one of the more interesting design decisions in the project, and it is worth understanding because it changes how you install and how you upgrade. Extensions are separate packages that provide additional functionality and are installed in addition to DataLad. The README explains the practical consequence: to install DataLad customized for a particular domain, you can install an extension directly, and DataLad itself will be automatically installed with it. That is a small sentence with real consequences. It means there is no need to know which version of the core an extension wants, and it means the install command for a domain tool is the install command for the extension. For someone setting up a workflow for a specific kind of data, that removes a class of dependency problem. The downside is that the extension ecosystem is a separate compatibility surface. An extension can lag behind a core release, and the core can change behaviour that an extension depends on. The repository reflects this with a separate test_extensions workflow badge alongside the main test workflow, which suggests extensions are exercised in CI, but the badge also confirms extensions are not part of the same test surface as the core. The annotated list of extensions lives in the DataLad handbook at handbook.datalad.org/extension_pkgs.html, and that page is the right place to look before choosing a domain tool, because the list tells you what each extension actually does and how mature it is, which the core README does not. The default branch is named maint rather than master or main, while several of the README's badge links still point at master branches, which is the usual sign of a project with a maintained release line and an active development line. ## The command line, the issue tracker and NeuroStars DataLad is a command line tool, and the repository shows a serious development setup: a tox configuration, a pre-commit configuration, a coveragerc, benchmarks, a sandbox directory, a changelog.d directory for news fragments, a CITATION.cff, versioneer for version derivation, and a Makefile whose comments say the ideas for its build environment handling were borrowed from the scikit-learn and PyMVPA Makefiles. The Makefile is worth a look for one specific reason. The release-pypi target has a comment explaining that it avoids uploading stale builds with a test that no dist directory exists, and a commented-out wheel build with a note that the wheels produced would be broken on Windows because they install an incompatible entrypoint script, with an issue reference. That is a real operational constraint from the project's own history: a pip install of a released DataLad on Windows may behave differently from a source build. Support runs through two channels. Bugs, concerns and enhancement requests go to the GitHub issue tracker. Questions about how to use DataLad go to NeuroStars, which the README describes as a platform similar to StackOverflow but dedicated to neuroinformatics, and it asks that questions be tagged with a datalad tag. The pre-filled template asks for the version, which is run with datalad --version, and the operating system, with a pointer to datalad plugin wtf for checking on that. All previous DataLad questions are collected under a tag on the same site, which for a project of this age is a genuine asset. Old questions remain available, and a lot of the hard-won answers about annex special repositories and permissions are in that archive rather than in the documentation. ## Where DataLad is the wrong tool The first wrong-tool case is a team that only needs to move files around. DataLad is a distribution system built on a decentralized version control protocol and an annex extension. If the requirement is a shared folder, an rsync, or a synchronised object store, then the entire git-annex substrate is overhead you do not need. The second is a project that wants the platform to hold the data. The README is explicit that storage and permission management stay with the original data providers, and that is a feature for the research use case but a problem for anyone who assumed DataLad was a data platform. Nothing in the tool holds a canonical copy of your dataset unless you point an annex special repository at one yourself. The third is a Windows-only workflow. The Makefile comment about broken Windows wheels, the presence of colorama as a Windows-only dependency, and the general shape of the tooling all point to a Unix-first project. The README's install section covers Debian, Gentoo, other Linuxes via conda, and macOS via pip. Windows appears in the packaging, not in the installation guidance. The fourth is a workflow with no data-sharing dimension. DataLad earns its complexity when the same data must be usable by people with different access rights, and when the set of usable files differs per site. A single-team analysis on a single machine does not need it. The fifth is a team unwilling to learn git-annex. That is the real cost. The tool's power comes from annex special repositories and their per-directory permissions, and that model is subtle. A team that tries to shortcut the learning ends up with a dataset nobody can install, and the troubleshooting path runs through the issue tracker and NeuroStars rather than a manual. ## Maintenance, licensing and the shape of the releases The release history shows a project on a fast cadence right now. Version 1.6.6 was published on 2026-10-05, 1.7.0 the same day later on, and 1.7.1 on 2026-10-06. The last push to the repository was on 2026-10-21, and the repository is not archived. Three releases in two days is worth reading carefully rather than celebrating. A 1.6.6 patch, then a 1.7.0 minor, then a 1.7.1 patch, all within about a day, is the pattern of a release train being cut and then immediately corrected. The README documents a workflow for this with a changelog.d directory for news fragments and a Makefile target that converts CHANGELOG.md into reStructuredText for the documentation, so the release process is automated end to end. The licence situation needs the same care as the releases. The README badge shows MIT, and the repository root carries a COPYING file, and the setup.py header says the copyright and licence terms are in the COPYING file distributed with the package. GitHub's own classification of the repository licence is not one it can identify, which usually means the file has a non-standard header or a custom grant. Read COPYING directly if the licence is load-bearing for you, and treat the badge as a pointer rather than the answer. The scientific community is a real part of the picture here. The JOSS paper DOI, the Zenodo archival DOI, the CITATION.cff, the RRID identifier and the testimonials wiki all indicate a project that is treated as research infrastructure and archived accordingly. The code of conduct follows the Contributor Covenant 2.1. For an academic group, that apparatus is part of why the tool is trusted with restricted data, and it is also a signal that the project is a long-term commitment rather than a side project. ## Conclusion DataLad fits a research group that needs to distribute datasets to collaborators who each hold different permissions on the underlying data, and that already lives in a git and git-annex world. It does not fit a team that only needs file synchronisation, a project that cannot accept the git-annex dependency, or anyone expecting the tool to store the data for them, since the README is explicit that storage and permission management stay with the original data provider. Before adopting it, install git-annex first as the README requires, confirm the extension you need exists in the annotated extension list, and plan for the annex special repository to remain the operational centre of your setup. ## FAQ ### What is DataLad used for? It is a distributed system for joint management of code, data and their relationship, built on git and git-annex, that automates ingestion from online portals and exposes data as installable datasets. ### Does DataLad store the data itself? No, the README states that the actual data storage and permission management remains with the original data providers, and DataLad is the mechanism for describing and distributing it. ### What do I need before installing DataLad with pip? The README asks you to install a recent version of git-annex first, then install datalad from PyPI, preferably inside a dedicated virtual environment. ### How do DataLad extensions work? Extensions are separate packages installed in addition to DataLad, and installing an extension directly will pull in DataLad itself automatically, with an annotated list maintained in the DataLad handbook. ### Where do I ask a question about using DataLad? Questions go to NeuroStars with a datalad tag, and bugs and enhancement requests go to the GitHub issue tracker, with previous questions collected under the same tag on NeuroStars. ## Sources - [datalad/datalad on GitHub](https://github.com/datalad/datalad) - [Issues](https://github.com/datalad/datalad/issues) - [Project website](http://datalad.org) - [README](https://github.com/datalad/datalad/blob/maint/README.md) - [Releases](https://github.com/datalad/datalad/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/datalad-datalad