# Intake describes data, catalogs it, and lets a driver do the loading

> Intake is a Python package for describing data declaratively, gathering data sets into catalogs, searching catalogs and services, and loading, transforming and outputting data in many formats, with the concrete formats implemented as plugins registered through entry points. The repository is mid refactor, with a Take 2 notebook and a README_refactor.md, and the licence is stated inconsistently between the repository metadata and the packaging file.

**intake/intake** — Intake is a lightweight package for finding, investigating, loading and disseminating data.

- Repository: https://github.com/intake/intake
- Website: https://intake.readthedocs.io/
- Stars: 1,086 · Forks: 153
- Language: Python
- License: BSD-2-Clause
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/intake-intake

## One description of the data, used by a catalog, a search and a loader

Intake is described as taking the pain out of data access and distribution, and the README frames it as a general python package for describing, loading and processing data. Five capabilities are listed. You can describe your data declaratively, gather data sets into catalogs, search catalogs and services to find the right data you need, load, transform and output data in many formats, and work with third party remote storage and compute platforms. The design assumption is that the interesting variation between data sets is in where they live and how they are encoded, not in the code that consumes them, so that variation belongs in a description rather than in a script. The audience is data scientists and engineers who have grown a pile of near-identical loading code, and the alternative the project is aimed at is not a missing library but the absence of a shared vocabulary for what a data set is.

## Drivers are entry points, and six ship in the box

The plugin mechanism is visible in the packaging, and it is the part of the design that generalises. The project declares entry points under the group intake.drivers, and each one names a source class:

```toml
[project.entry-points."intake.drivers"]
csv = "intake.source.csv:CSVSource"
jsonfiles = "intake.source.jsonfiles:JSONFileSource"
numpy = "intake.source.npy:NPySource"
tiled_cat = "intake.source.tiled:TiledCatalog"
textfiles = "intake.source.textfiles:TextFilesSource"
ndzarr = "intake.source.zarr:ZarrArraySource"
```

The consequence is that a format is added by installing a package, not by patching the framework, and that the core stays small enough to read. The six built-in drivers cover delimited text, JSON files, NumPy arrays, tiled catalogs, text files and Zarr arrays, and the note in the installation section says you will probably need specific drivers and other plugins, which usually have additional dependencies of their own. That is the honest shape of a plugin system, and it is also the main operational cost, since the dependency graph of a data environment is the sum of the drivers in it. The core dependencies are modest, with fsspec, pyyaml, platformdirs and networkx.

## Catalogs are the unit of sharing

The catalog is the concept that turns a loader into a data access system. Rather than shipping a script that opens a file, you describe a collection of data sets, and the catalog is what you search, browse and load from. That is why the driver list includes tiled_cat, a TiledCatalog, and why the feature list talks about searching catalogs and services rather than files. A service is the remote counterpart, which is how the work with third party remote storage and compute platforms line fits in, since a remote service is a catalog you reach over a network rather than one you have on disk. The practical benefit is a single interface for a local notebook, a scheduled job and a dashboard, and the cost is one more layer between you and the bytes, which is the trade every catalogue system makes. The examples directory currently holds a single notebook, examples/Take2.ipynb, which is worth looking at because it is the clearest statement of how the new API is meant to be used.

## Take 2, and a refactor you are told about

The README is headed Intake: Take 2, and the repository has a README_refactor.md next to it, which tells you a redesign is in progress rather than a settled API. There is also a notebook in the tree, examples/Take2.ipynb, and the notebook name matches the heading. That is a mild warning for an adopter, because a package whose own documentation is being rewritten is a package whose API you should check before you build on it, and the project is honest enough to leave the refactor note in the repository rather than quietly renaming things. The repository has no GitHub releases, so the version you install comes from the packaging metadata, which uses setuptools_scm with the version written to intake/_version.py. That means the version tracks the commit, which is convenient for reproducibility if you pin a commit and awkward if you expect tagged releases.

## Install with conda, or pip and only the drivers you need

The recommended installation is conda:

```bash
conda install -c conda-forge intake
```

The pip route is also supported, and the README explains the choice clearly. With pip you choose how many of the optional dependencies you install, with the simplest having the least requirements:

```bash
pip install intake
```

and the caveat follows immediately, that you may well need specific drivers and other plugins, which usually have additional dependencies of their own. The project requires Python 3.11 or above, per the requires-python field in the packaging metadata, and the classifiers call the development status Beta. Support comes in part from Anaconda, with the README noting that work on the repository is supported in part by Anaconda advancing AI through open source, which is the usual arrangement for a project that a company maintains for its own data stack and releases under a permissive licence. The documentation is on Read the Docs and issues go to the GitHub tracker.

## The development setup, and the linting it expects

The contributor instructions are short and specific. You create a development Python environment with the required dependencies, ideally with conda, and the requirements live in the yml files in the scripts/ci/ directory, for example conda env create -f scripts/ci/environment-py311.yml followed by conda activate test_env. You then install the package in editable mode with pip install -e ., run the tests with pytest, and create a fork to submit pull requests. The house style is stated as a preference rather than a rule, that the project respects but does not enforce pep8 standards, and that all new code should be covered by tests. The tooling configuration is modern and visible in the packaging file, with black at a line length of 100, ruff at 130, isort with the black profile, and per-file ignores for tests and for unused imports in __init__ files, plus a flake8 configuration kept alongside for older setups. There is a pre-commit configuration at the root, which is how those settings are applied consistently.

## Where intake is the wrong choice

Three honest limits. The first is that for a single script reading a single CSV, intake is pure overhead, because the description, the catalog and the driver machinery all cost more than the code they replace. The second is the dependency surface, since the driver model means the environment you deploy is the environment of every plugin you enabled, and a plugin that pulls a heavy or version-conflicting dependency can destabilise a working data stack for a benefit that is easy to lose track of. The third is the state of the project, with a Take 2 heading, a refactor document in the tree, no GitHub releases, and a Beta development status. None of that makes the design wrong, but it does mean the cost of starting a new project on it today is higher than the cost of reading it. The practical move for a team that likes the model is to keep the core dependency small, install only the drivers you use, and pin your versions.

## Conclusion

Intake is worth a look if your team keeps writing the same data loading code for a dozen formats and wants one description of the data that a catalog, a search index and a loader can all read, because the entry point mechanism is the part that generalises and the built-in drivers cover the common cases. It is the wrong tool for a single one-off script, where the abstraction is pure overhead, and it is a poor fit for a project that needs a stable API, since the README is titled Take 2 and the refactor is still in the tree. Before adopting it, read README_refactor.md to see which parts of the API are changing, check the licence in the LICENSE file yourself, because the packaging metadata and the repository metadata disagree, and install only the drivers you need rather than the full optional dependency set.

## FAQ

### What is Intake used for?

Intake is a general Python package for describing, loading and processing data. It lets you describe data declaratively, gather data sets into catalogs, search catalogs and services, load, transform and output data in many formats, and work with third party remote storage and compute platforms.

### How does Intake add support for a new data format?

As a plugin, registered through entry points in the group intake.drivers. The built-in drivers map csv, jsonfiles, numpy, tiled_cat, textfiles and ndzarr to source classes such as intake.source.csv:CSVSource, and third-party drivers are installed the same way as separate packages with their own dependencies.

### How do I install Intake?

The recommended method is conda install -c conda-forge intake, or pip install intake, where the pip route lets you choose how many optional dependencies to install. The package requires Python 3.11 or above, and you will usually need specific drivers for the formats you actually use.

### Is the Intake API stable?

The README is headed Intake: Take 2 and the repository contains a README_refactor.md, so a redesign is in progress. The package is classified as Development Status 4, Beta, and the repository has no GitHub releases, with the version derived from the commit through setuptools_scm.

## Sources

- [intake/intake on GitHub](https://github.com/intake/intake)
- [Issues](https://github.com/intake/intake/issues)
- [License: BSD-2-Clause](https://github.com/intake/intake/blob/master/LICENSE)
- [Project website](https://intake.readthedocs.io/)
- [README](https://github.com/intake/intake/blob/master/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/intake-intake