# cache-dit puts caching and parallelism in front of a diffusion transformer

> cache-dit is a PyTorch-native inference engine for diffusion transformers, built on the Hugging Face Diffusers library and claiming support for nearly all of the DiT pipelines in it, across 40 plus pipeline families and 120 plus variants. It provides hybrid cache acceleration with methods such as DBCache, TaylorSeer and SCM, a full set of parallelism options including context parallelism, tensor parallelism and hybrid two and three dimensional parallelism, plus quantization, CPU offload and compilation.

**vipshop/cache-dit** — A PyTorch-native inference engine with cache, parallelism, quantization and cpu offload for DiTs.

- Repository: https://github.com/vipshop/cache-dit
- Website: https://cache-dit.readthedocs.io
- Stars: 1,293 · Forks: 93
- Language: Python
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/vipshop-cache-dit

## It sits between Diffusers and the hardware

The description is a PyTorch-native inference engine with cache, parallelism, quantization and CPU offload for DiTs, and the README says the engine is built on top of the Diffusers library and now supports nearly all the DiTs from Diffusers. That placement is the design. Diffusers gives you the pipeline and the model classes, and cache-dit gives you a way to make the transformer inside it faster without rewriting the pipeline, which is what makes it usable on an existing codebase. The claim in the README is a 9x speedup from cache-dit with cache, context parallelism and compilation, which is a headline number for a specific combination, so it is worth reading the linked speedup page rather than treating it as a general multiplier. The project is from vipshop, the maintainers are listed as DefTruth and the team at vipshop, and the documentation is published on Read the Docs.

## Hybrid cache acceleration, and why it is hybrid

The caching side is the first feature listed, hybrid cache acceleration, with DBCache, TaylorSeer and SCM among the methods, and there is a dedicated page in the user guide for the cache API. The idea behind cache methods in a diffusion transformer is that consecutive denoising steps are similar enough that the output of a block can be reused instead of recomputed, and a hybrid strategy decides per block and per step whether to recompute or to reuse. That is why the word hybrid appears, because the reuse policy is not uniform across the network, and a fixed policy either wastes compute or breaks the output. The three method names are the implementations on offer, and the number of them suggests the project treats caching as a tuning problem where different models and different step counts want different policies.

## Parallelism, including the parts nobody else writes

The parallelism side is described as comprehensive, and the list explains why it is longer than context parallelism. There is Context Parallelism, Tensor Parallelism, and hybrid 2D or 3D parallelism, plus dedicated extra parallelism support for the Text Encoder, the VAE and ControlNet. That last part is the valuable one for a practitioner, because most attention goes to the denoising transformer and the other three stages are where a pipeline quietly spends its time. The optional dependency list backs it up, with an einops extra for parallelism and a ray extra for the distributed execution, with safetensors along for the ride. The dependency surface is small, which means the parallelism is implemented in the project rather than delegated, and the ray extra is what tells you which of the two paths a given mode takes.

## The support matrix is per transformer class, not per pipeline

The supported models table is the most useful documentation in the repository, and its structure is the point. It covers 40 plus DiT pipeline families and 120 plus variants, and every row names a modality, a pipeline series, the transformer class, the number of variants, and four columns marked C, P, Q and OF, which stand for caching, parallelism, quantization and offload. The rows make the granularity obvious, since FLUX is listed as 10 plus variants with all four columns ticked, while SD3, Sana, AuraFlow, HiDream, Bria, PRX, OmniGen, Lumina and Lumina 2 each carry a cross in the parallelism column, which is a real limitation rather than a formatting quirk. The transformer class is the key into the code, since every row points at the same file, src/cache_dit/caching/block_adapters/adapters.py, which is where the per-class adapter lives. FLUX.2 and FLUX.2 Klein are separate rows, and so are HunyuanDiT and HunyuanDiT-PAG, which tells you how quickly the model landscape moves.

## Quantization and offload, and the hardware floor underneath

Quantization is an optional extra built on torchao and bitsandbytes, and the project also has its own quantized kernel path, SVDQuant, which the repository topics name. The setup.py at the root is where the hardware constraint becomes visible, with a CACHE_DIT_BUILD_SVDQUANT build flag and a CACHE_DIT_WHEEL_PLATFORM_TAG variable, and a long comment explaining that the sm100 target for Blackwell B100 and B200 is not in the default list because the NVFP4 block-scaled MMA instruction used by the SVDQuant FP4 kernel requires sm_120a or higher, while the INT4 kernels work correctly on sm100. The comment cites the PTX ISA notes on the mxf4nvf4 and mxf4 kinds and the supported architectures. The practical reading is that a full FP4 path needs a very new accelerator, an INT4 path runs on sm100, and a user who only needs INT4 can build for sm100 explicitly. That is exactly the kind of detail a project usually buries, and here it is in the build script.

## Two wheels, one name

The packaging story is worth a paragraph because it explains the release tags. The comment in setup.py says the distribution name is intentionally not managed through a PACKAGE_NAME environment variable, because under PEP 621 setuptools reads the project name from the active workspace's pyproject.toml, so default builds use the repository root pyproject.toml and so do the release builds, and the pure-Python wheel and the CUDA wheel share the distribution name, with only the CUDA wheel's local version, +cu13, telling them apart. That is why the release list has both v1.5.3+cu13 and v1.5.3 on the same day, and why the setup script defines a CACHE_DIT_RELEASE_FLAVOR variable and a CACHE_DIT_VERSION_WRITE_TO variable, and why tools/release_workspace.py exists to prepare the workspace the release build runs in. The version itself comes from setuptools_scm, which reads the tags, and the setup script refuses to run without setuptools-scm installed, raising an ImportError that says so.

## Where it plugs in, and where it stops

The integration surface is wide, and the README lists it as compatibility with compilation, CPU offloading and quantization, full integration with SGLang Diffusion, vLLM-Omni, TensorRT-LLM and ComfyUI, and native runs on NVIDIA GPUs, Ascend NPUs and AMD GPUs. That is the argument for adopting it rather than writing the same layer yourself, because the parallelism and the caching are the kind of code that has to be right for a dozen model classes. The limit is that the caching itself is scoped to the diffusion transformer. A pipeline whose cost is dominated by the VAE decode, or by the text encoder, or by the scheduler, will see much less of the advertised speedup, and the honest way to evaluate it is to profile where the time goes first. The other limit is the matrix itself, since a new model family needs a new adapter, and the project is a fast moving one, with a v1.5.3 release on 2026-10-15 and a CUDA flavoured build published the same day. The Apache 2.0 licence and the pure Python core with optional extras for parallelism, ray, quantization and metrics make the adoption decision mostly about the hardware and the model you serve.

## Conclusion

cache-dit is worth evaluating if you are serving a diffusion transformer and the bottleneck is the transformer blocks rather than the text encoder or the VAE, because the cache methods and the parallelism are the two levers that matter for that part of the pipeline and the model matrix tells you which combinations are supported. It is the wrong tool if your model family is not in the matrix, because the support is per transformer class rather than generic, and the wrong tool if you need a single-feature install, since the optional extras pull in whole toolchains. Before you adopt it, check the row for your own model in the support matrix, since the C, P, Q and OF columns mark which of caching, parallelism, quantization and offload are actually available for that transformer, and read the wheel story, because the CUDA build is a separate artifact from the pure Python one.

## FAQ

### What does cache-dit do to a diffusion transformer pipeline?

It is a PyTorch-native inference engine built on the Diffusers library that adds hybrid cache acceleration such as DBCache, TaylorSeer and SCM, plus context parallelism, tensor parallelism, hybrid 2D and 3D parallelism, and dedicated parallelism for the text encoder, the VAE and ControlNet, alongside quantization, CPU offload and compilation.

### How many models does cache-dit support?

It claims support for 40 plus DiT pipeline families and 120 plus variants from Diffusers. The support matrix lists each pipeline series with its transformer class, the number of variants, and four columns for caching, parallelism, quantization and offload, and a cross in the parallelism column marks the rows where that feature is not available for that transformer.

### Why are there two releases of cache-dit with the same version number?

The pure-Python wheel and the CUDA wheel share the distribution name, and only the CUDA wheel carries a local version suffix such as +cu13 to tell them apart, which is why the release list shows both v1.5.3+cu13 and v1.5.3. The setup script refuses to run without setuptools-scm installed, and the version comes from the git tags.

### What are the optional extras of cache-dit?

parallelism with einops, ray with safetensors for distributed execution, quantization with torchao and bitsandbytes, and metrics with scipy, scikit-image, image-reward and lpips. The core dependencies are torch 2.7.1 or newer, diffusers 0.36.0 or newer, transformers 4.55.2 or newer, packaging and pyyaml, and Python 3.10 or newer is required.

## Sources

- [License: Apache-2.0](https://github.com/vipshop/cache-dit/blob/main/LICENSE)
- [Project website](https://cache-dit.readthedocs.io)
- [README](https://github.com/vipshop/cache-dit/blob/main/README.md)
- [Releases](https://github.com/vipshop/cache-dit/releases)
- [vipshop/cache-dit on GitHub](https://github.com/vipshop/cache-dit)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/vipshop-cache-dit