# TorchData: checkpointable DataLoaders and a composable iterator library

> TorchData is Meta's project for making PyTorch data loading scalable, and it is two things in one repository: a stateful DataLoader that can be checkpointed mid-epoch, and torchdata.nodes, a library of composable iterators for streaming data pipelines. The version compatibility table is the part to read first, because the package is released in lockstep with torch and the pairing is not obvious.

**meta-pytorch/data** — A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.

- Repository: https://github.com/meta-pytorch/data
- Stars: 1,267 · Forks: 183
- Language: Python
- License: BSD-3-Clause
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/meta-pytorch-data

## An enhancement to the DataLoader, not a replacement for it

TorchData is described as an iterative enhancement to the PyTorch `torch.utils.data.DataLoader` and `torch.utils.data.Dataset` and `IterableDataset`, with the goal of making them scalable and performant dataloading solutions, and the work is being iterated in the torchdata repository. The framing matters, because it tells you what the project is competing with. PyTorch's DataLoader is not slow in the abstract; it is limited in specific ways, chiefly in what it can tell you about its own progress, and in how you resume a job that was interrupted. A DataLoader that runs for hours across many workers, with a shuffle, a sampler and a transform pipeline, has no built-in way to be checkpointed, because its state is spread across the workers and the sampler, and a restart means starting the epoch again. The project's approach is to add that capability while keeping the existing interface, so that the migration path is a class swap rather than a rewrite. The repository description frames the wider purpose as a place for data loading and utilities to be shared by the PyTorch domain libraries, which is why the layout looks like a library with several subpackages rather than a single feature.

## The Stateful DataLoader, and what a checkpoint has to capture

The first change described in the README is adding checkpointing to the standard DataLoader, delivered as `torchdata.stateful_dataloader.StatefulDataLoader`, a drop-in replacement for `torch.utils.data.DataLoader` that provides `state_dict` and `load_state_dict` functionality. The two methods are the whole interface, and what they carry is the interesting part. The README says they enable mid-epoch checkpointing, so a job can be checkpointed and resumed in the middle of an epoch rather than only at epoch boundaries. They also provide an API for users to track custom iteration progress, and other custom states from the dataloader workers such as token buffers and/or RNG states. That last clause is the one that shows the real difficulty of the problem. Saving the index is easy; saving the state of a pipeline that has already consumed, shuffled and tokenized data is not, and a loader that claims mid-epoch resume has to capture the state of the workers, not just the position of the sampler. A loader that checkpoints the sampler but not the worker state will resume into a different data order than the one it left, which for a training run is a silent correctness problem. The project has a Colab notebook with examples, and the main page for the Stateful DataLoader is linked from the README for the details.

## torchdata.nodes, and why it is built from iterators

The second half of the repository is `torchdata.nodes`, a library of composable iterators that let you chain together common dataloading and pre-processing operations. The README makes a distinction that is easy to miss and that defines the design, saying that these are iterators and not iterables. An iterable is something you can restart, and an iterator is something that has already consumed part of its input, and building a pipeline out of iterators means the data flows through the stages rather than being materialised and re-read. It follows a streaming programming model, although the README notes that a sampler plus Map-style configuration can still be set up if you want it. The practical difference shows up in memory behaviour and in where you can put a transform. In a classic Dataset and DataLoader arrangement, the map style transform runs on the worker side and the batching and shuffling are handled by the loader's own machinery, which makes it awkward to insert a stage that needs to see the stream as a whole. With composable iterators, that stage is just another link in the chain. The honest caveat is that this half of the project is younger than the Stateful DataLoader. The README says to stay tuned for a tutorial on torchdata.nodes coming soon, which tells you where the documentation stands.

## The version table is the operational crux

Installation instructions come with a version compatibility table, and it is the single most important thing in the README for anyone deploying this. The pairs are explicit. The master and nightly branches of torch go with the main and nightly branches of torchdata, on Python from 3.9 up to and including 3.13. Torch 2.6.0 goes with torchdata 0.11.0 on the same Python range. Torch 2.5.0 goes with torchdata 0.10.0 and with 0.9.0, on Python 3.9 to 3.12. Torch 2.4.0 goes with 0.8.0, torch 2.0.0 with 0.6.0, torch 1.13.1 with 0.5.1, torch 1.12.1 with 0.4.1, torch 1.12.0 with 0.4.0 and torch 1.11.0 with 0.3.0. The pattern is a strict lockstep, and the Python ceiling moves with it, which is the kind of detail that catches people out when their environment is older than the newest release. The install itself is unremarkable.

The commands themselves are short:

```bash
pip install torchdata
```

```bash
conda create --name torchdata
conda activate torchdata
```

 You create an environment, with conda or with `python -m venv`, then install with `pip install torchdata` or `conda install -c pytorch torchdata`, and there is a from-source route with `pip install .`, where a failure means installing a nightly PyTorch build first, following the guide on the contributing page. For nightlies, `pip install --pre torchdata --index-url https://download.pytorch.org/whl/nightly/cpu`, or the conda nightly channel.

## A build that compiles extensions and stamps its own version

The packaging tells you this is not a pure Python package. The build system in `pyproject.toml` requires setuptools and wheel, and also ninja, cmake and torch, which is the signature of a package that compiles something. The `setup.py` confirms it, since it pulls extension modules from a helper in `tools/setup_helpers`, and it has a deliberate dance around the clean, egg_info and sdist commands so that the build dependency step is skipped for those. The version handling is worth a mention, because it is more careful than most projects manage. The version comes from `version.txt`, and the setup script then looks up the current git commit with `git rev-parse HEAD` and appends the short hash to the version, so a source build identifies itself by commit, and it honours a `BUILD_VERSION` environment variable when one is set, overriding the file. It then writes the result into a generated `torchdata/version.py` with both the version and the git version. The runtime requirements are light, since `requirements.txt` lists urllib3 1.25 or newer, requests, smart_open and fsspec, which are the pieces you need for reading objects from remote stores. The code style is black with a line length of 120 and a target version of py39, there is a `mypy.ini`, a `.flake8` configuration, a `.pre-commit-config.yaml` and a prettier configuration for the non-Python files.

## Benchmarks and examples, and a project still asking for feedback

The repository has a `benchmarks/` directory and an `examples/` directory whose names map onto the kinds of pipelines people actually build, with `criteo_torcharrow` for a columnar dataset, `text` for language data, `vision` for images and a `nodes` example for the iterator library. That split is a good indication of where the difficulty lies, since a Criteo style tabular dataset and a text corpus exercise very different parts of a loading pipeline. There is also a `docs/` directory, `packaging/`, `scripts/`, `test/` and a `benchmarks/` area for performance work. The maintenance picture is where a reader should slow down. The newest release tag is v0.11.0 from February 2025, with 0.10.1 in December 2024 and 0.9.0 in October 2024, while the last push to the branch was on 2026-09-16, so the source is moving but the packaged releases are not keeping the same pace. The README has a section titled Beta Usage and Feedback, and the project asks early adopters to raise an issue if they want to use the tooling, which is the clearest signal of the maturity level. The licence is BSD, as the LICENSE file states.

## Conclusion

Adopt TorchData if you need to resume a large data loading job mid-epoch, where the standard DataLoader cannot tell you where it was, or if you want a streaming pipeline built from composable iterators instead of a sampler plus a map style dataset. Do not adopt it if your loader is small and restartable, since the standard `torch.utils.data.DataLoader` is already there, and note that torchdata.nodes is explicitly still being documented, with the tutorial promised for later. Before installing, check the version compatibility table and match the torchdata version to your torch version and your Python version, because the pairs are pinned and installing the latest torchdata against an older torch is a supported way to break. The licence is BSD-3-Clause, the newest tag is v0.11.0 from 2025-02-21, and the last push was on 2026-09-16.

## FAQ

### What does torchdata.stateful_dataloader.StatefulDataLoader do?

It is a drop-in replacement for `torch.utils.data.DataLoader` that adds `state_dict` and `load_state_dict`, which enable mid-epoch checkpointing, let you track custom iteration progress, and capture other custom state from the dataloader workers such as token buffers and RNG states.

### Which torchdata version should I install for my torch version?

The README's compatibility table pairs them strictly: torch 2.6.0 with torchdata 0.11.0, torch 2.5.0 with 0.10.0 and 0.9.0, torch 2.4.0 with 0.8.0, torch 2.0.0 with 0.6.0, torch 1.13.1 with 0.5.1, torch 1.12.1 with 0.4.1, torch 1.12.0 with 0.4.0 and torch 1.11.0 with 0.3.0, with a Python range that also changes per pair.

### How do I install TorchData?

Create an environment with conda or venv, then run `pip install torchdata` or `conda install -c pytorch torchdata`. From source it is `pip install .`, and for nightly builds `pip install --pre torchdata --index-url https://download.pytorch.org/whl/nightly/cpu` or the conda nightly channel.

### What is torchdata.nodes?

It is a library of composable iterators, not iterables, for chaining dataloading and pre-processing operations. It follows a streaming programming model, although a sampler plus Map-style setup can still be configured. The README notes a tutorial for it is still to come.

## Sources

- [Issues](https://github.com/meta-pytorch/data/issues)
- [License: BSD-3-Clause](https://github.com/meta-pytorch/data/blob/main/LICENSE)
- [meta-pytorch/data on GitHub](https://github.com/meta-pytorch/data)
- [README](https://github.com/meta-pytorch/data/blob/main/README.md)
- [Releases](https://github.com/meta-pytorch/data/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/meta-pytorch-data