xDiT: hybrid sequence and pipeline parallelism for diffusion transformers

The problem is quadratic attention on inputs that keep growing

The README's opening statement is the whole motivation in two sentences: diffusion transformers drive high-quality image and video generation, and as the input context length in DiTs grows, the computational demand of the attention mechanism grows quadratically. The consequence it draws is that multi-GPU and multi-machine deployments become necessary to meet real-time requirements in online services.

That framing matters because it explains why this is an inference engine rather than a training framework. Nobody is asking xDiT to fit a model; they are asking it to serve one fast enough to be behind an API. Video models make the pressure worse because the sequence includes both spatial and temporal dimensions, so a resolution bump multiplies the token count and the attention matrix together.

The repository metadata is Apache-2.0 with a `LICENSE.txt` at the root, 2,729 stars and 345 forks, 54 open issues, no archive flag, and a last push of 2026-09-24. One detail stands out as unusual: the repository has no GitHub topics set at all. For a project this visible that is a small loss, since topic tags are how people discover projects adjacent to diffusion inference, and it suggests the metadata simply never got filled in rather than that the authors declined.

There is also no homepage field configured on the repository, even though the README links a Discord server, GitHub Discussions, a Medium blog at `@xditproject`, per-model performance reports under `docs/performance/`, and arXiv links for the methods it implements.

Four parallel methods designed to be composed

xDiT's central design claim is that the four parallel methods it implements can be configured in a hybrid manner to suit the network hardware. That composition, rather than any individual method, is the contribution.

The first is USP, unified sequence parallelism, proposed by this project's own authors and described as combining DeepSpeed-Ulysses with Ring-Attention, with an arXiv link given. The second is PipeFusion, a sequence-level pipeline parallelism described as similar to TeraPipe but taking advantage of the input temporal redundancy characteristics of diffusion models, which is the property that makes a video pipeline different from a language model one. The third is plain data parallelism, processing multiple prompts or generating multiple images from a single prompt in parallel across images. The fourth is CFG parallelism, also called split batch, which activates when classifier-free guidance is used and then runs at a constant parallelism of 2.

That last detail is the one that will surprise you if you skim. Because CFG evaluates a conditional and an unconditional batch together, splitting that batch in two is a free win whenever CFG is on, which is why the parallelism is fixed rather than tunable. The cost is that it is unavailable when CFG is off, and the support table reflects this by marking CFG support per model.

Two more strategies are listed as implemented for reference rather than as part of the tuned path: tensor parallelism, and DistriFusion, which has its own arXiv link. Whether you would reach for the reference implementations instead of the four primary methods depends on your model and your interconnect, and the support table is where that is decided.

Single-GPU acceleration is treated as a separate axis

Caching and compilation are presented as orthogonal to parallelism, and that separation is the practical part of the design. Parallelism decides where work goes; caching decides how much work exists.

The cache methods named are TeaCache, First-Block-Cache, which comes from the ParaAttention project, and DiTFastAttn. All three exploit computational redundancy between different steps of the diffusion model, which is the correct target: a denoising trajectory revisits similar states, so recomputing identical attention on consecutive steps is waste. The tree carries a matching entrypoint at `examples/run_fastditattn.sh`.

On the compilation side, the README says xDiT uses well-known attention optimization libraries plus compilation acceleration through `torch.compile` and `onediff`. The kernel level work shows up in the release notes rather than the README, and that is where the more telling detail lives: version 0.6.0 adds fused QK-RMSNorm with RoPE for FLUX.1, enables QKV all-to-all packing, integrates a new AITER MHA v4 FMHA entrypoint, and adds AITER RMSNorm. AITER is AMD's Triton-based attention stack, and the contributor names on those changes carry an `-amd` suffix, which tells you AMD hardware is a first-class target rather than an afterthought.

Release 0.5.0, published the day before 0.6.0, adds FP8 GEMM quantization for Flux and Wan models and combined QKV all-to-all for Flux2 parallel self-attention. Between the two releases you have kernel fusion, FP8 quantization, custom attention backend support, memory-efficient model loading, tile-parallel VAE execution, and a unified step caching entrypoint with dbcache presets. That is a fast rate of change, and it argues for pinning a version rather than tracking main.

The support table is the part to read before planning

There is a matrix of supported models with columns for CFG, SP, PipeFusion, TP, and a column headed MR with an asterisk, followed by a link to a performance report. A short excerpt:

| [🎬 StepVideo](https://huggingface.co/stepfun-ai/stepvideo-t2v) | NA | ✔️ | ❎ | ✔️ | ❎ |
| [🎬 HunyuanVideo](https://github.com/Tencent/HunyuanVideo) | NA | ✔️ | ❎ | ❎ | ✔️ |
| [🎬 HunyuanVideo-1.5](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5) | ❎ | ✔️ | ❎ | ❎ | ✔️ |
| [🎬 ConsisID-Preview](https://github.com/PKU-YuanGroup/ConsisID) | ✔️ | ✔️ | ❎ | ❎ | ❎ |

Several things are worth noticing. CFG support is marked NA for StepVideo and HunyuanVideo, which aligns with those models not using classifier-free guidance, so CFG parallelism is unavailable and the split-batch win does not apply to them. StepVideo is one of the few with tensor parallelism marked, while HunyuanVideo and its 1.5 successor are the ones with the MR column checked, so the choice of method really is per-model rather than global.

The asterisk on MR is not expanded in the table, so treat that column as named but not self-describing and check the model entries or the developer guide before relying on it. The performance report column is the other thing to check honestly: many of the newer rows, including HunyuanVideo-1.5, several Wan2.2 variants, CausalWan2.2, LTX-2 and LTX-2.5, are marked NA, meaning support is claimed without a published report. Where a report exists the path is a file under `docs/performance/`, so the claim is at least inspectable.

The README states that models in the community list ship with xDiT on day one, naming HunyuanVideo from Tencent, StepVideo from stepfun-ai, SkyReels-V1 from SkyworkAI, and Wan2.1. That is the timeline to plan around: if you are deploying a model released in the last month, being on the supported list does not yet mean being benchmarked.

The Python package is xfuser, the command is xdit

A detail that will save you a failed install: the distribution name on PyPI is not the repository name. `pyproject.toml` declares:

name = "xfuser"
description = "A Scalable Inference Engine for Diffusion Transformers (DiTs) on Multiple Computing Devices"
requires-python = ">=3.10"

The command is separate from the import name, declared as a project script:

[project.scripts]
xdit = "xfuser.cli:main"

So you import from `xfuser` and invoke `xdit` from the shell. The root directory in the tree is `xfuser/`, which matches the import rather than the project name, and the version is written into `xfuser/__version__.py` by setuptools-scm with a fallback of `0.0.0.dev`. Because versions come from SCM rather than being hardcoded, installing straight from git gives you a development version string, which is worth knowing when you are reading logs.

The dependency list is short and readable, with the heavy hitters being `torch>=2.4.1` and `diffusers>=0.33.0` alongside `accelerate`, `transformers`, `peft`, `av`, `einops`, `distvae>=0.1.0` and `yunchang>=0.6.0`. Optional extras each carry a comment explaining exactly when they are required, which is unusually helpful documentation to find in a manifest:

flash-attn = [
    "flash-attn>=2.6.0",  # NOTE: flash-attn is necessary if ring_degree > 1
]
optimum-quanto = [
    "optimum-quanto",  # NOTE: optimum-quanto is necessary if use_fp8_t5_encoder is enabled
]

The rest map the same way: `torchao` when `use_fp8_gemms` is enabled, `flask` to run xDiT as an HTTP service, `ray` when `RayDiffusionPipeline` is used, `opencv-python-headless` when `ConsisIDPipeline` is used, and `cache-dit>=1.5.0,<1.6.0` for `--cache_method dbcache`. That last upper bound is the kind of pin that breaks installs without warning, so read it before assuming the extras will resolve.

What the repository layout says about supported deployment modes

The top-level tree is small enough to read as a map of intended use: `xfuser/` for the package, `entrypoints/`, `examples/`, `benchmark/` alongside a second `benchmarks/` directory, `docker/`, `docs/`, `tests/`, plus `pyproject.toml`, `pytest.ini`, `.pre-commit-config.yaml` and a `.gitmodules`.

The presence of `.gitmodules` is worth a moment's thought, because submodule dependencies in an inference engine usually mean either model weights or a component too large to vendor. Combined with `docker/`, it points to a project that expects containerised deployment rather than a `pip install` into an existing environment.

The `examples/` directory is the most useful part for evaluation, and its shell scripts name the deployment modes directly:

xfuser/
entrypoints/
examples/
benchmark/
benchmarks/
docker/
docs/
tests/

Multi-node, service mode, single-model USP, and a Ray directory are four distinct things you might need and only one of them is the happy path. Python examples are named per model in a consistent pattern, with a plain and a `_usp` variant side by side for CogVideoX and ConsisID, which is a convenient way to see what sequence parallelism changes in a config. There is also an `examples/gemm_config.yaml` for the FP8 GEMM path.

Two operational modes deserve a note. Serving is built on Flask, which is an optional dependency rather than a core one, so the HTTP surface is deliberately separable from the engine. Distributed scheduling goes through Ray via `RayDiffusionPipeline`, again optional. Both choices mean the core library has no mandatory web or cluster framework in its dependency list, which is the right call for something people embed in pipelines as well as serve.

Editorial conclusion

xDiT exists because attention cost in a diffusion transformer grows quadratically with context length, and single-GPU workarounds only buy so much once you are generating video. The engine's real contribution is not any single parallel method but letting you compose sequence, pipeline, data and CFG parallelism against the specific interconnect you have, which is the decision a serving deployment actually has to make. The support table is the honest way to scope that: read the CFG, SP and MR columns for your model before planning, because several entries report no performance report at all. Start from the matching script in `examples/`, install only the extras your chosen path needs, and budget time for the fact that `yunchang` is still a hard dependency even though USP replaced it as the recommended attention path.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. xdit-project/xDiT on GitHub