Sparse-VideoGen and the sparsity hiding inside video diffusion attention

Three generations of the same idea: skip the redundant attention

Sparse VideoGen 1, 2 and EAR are described in the README as training-free frameworks that use the inherent sparsity in the 3D full attention operations to accelerate video generation.

That phrase, inherent sparsity, is the whole thesis. Video diffusion transformers attend over a very large token sequence, and the README claims that much of that attention is redundant. If you can identify the redundant part at run time and skip it, you get a speedup without touching the model weights. Training-free is what makes the approach attractive: there is no fine-tune, no distillation and no change to the checkpoint, which means you can apply it to a model you already have.

The three generations attack the problem at different points. Sparse VideoGen 1 identifies the spatial and temporal sparsity patterns in video diffusion models, proposes an Online Profiling Strategy to dynamically identify these patterns, and implements an end-to-end generation framework through efficient algorithm-system co-design, with hardware-efficient layout transformation and customized kernels.

Sparse VideoGen 2 tackles inaccurate token identification and computation waste, introduces semantic-aware sparse attention with efficient token permutation, and provides an end-to-end system design with a dynamic attention kernel and a flash k-means kernel.

SVG-EAR, the third and most recent, recovers skipped attention blocks with parameter-free centroid compensation and introduces error-aware routing to compute high-error blocks exactly while compensating the rest.

The install is where most of the difficulty lives

The installation section starts with a warning attached to the clone command, that you should not clone the demo because it is too large, and it skips Git LFS smudge to avoid pulling it:

GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/svg-project/Sparse-VideoGen.git # Do not clone the demo, otherwise is too large
cd Sparse-VideoGen

The environment recommendation is CUDA 12.4 or 12.8 with PyTorch 2.5.1 or 2.6.0, and the setup uses conda and uv:

# 1. Create and activate conda environment
conda create -n SVG python==3.12.9 # or 3.11.9 if have error when installing kernels
conda activate SVG

# 2. Install uv, then install other packages
pip install uv
uv pip install -e .

Then the parts that take real time. flash-attn is installed with no build isolation, the customised kernels are built from a git submodule with an updated cmake, and FlashInfer plus cuVS are installed as editable packages:

# 4. Install customized kernels. (You might need to upgrade your cmake and CUDA version.)
pip install -U setuptools # Require at least version 77.0.0
git submodule update --init --recursive
cd svg/kernels
pip install -U cmake
bash setup.sh

# 5. Install FlashInfer (standard) and cuVS
cd 3rdparty/flashinfer
pip install --no-build-isolation --verbose --editable .
pip install cuvs-cu12 --extra-index-url=https://pypi.nvidia.com

There is even a documented fallback for when the FlashInfer monkey patch fails, a manually patched FlashInfer with block sparse attention and varied block sizes, applied with git apply. The README also reassures you that you do not need to install flash-kmeans separately, because a copy is included and used by default.

Running the models is a matter of picking a script

Once installed, the inference path is a set of shell scripts, one per model and per variant, which is a clean way to see what the framework actually supports.

For Wan 2.1, both Text-to-Video and Image-to-Video inference are supported, and each has a script per variant:

# Text-to-Video
# bash scripts/wan/wan_t2v_720p_svg.sh # SVG
bash scripts/wan/wan_t2v_720p_sap.sh # SVG2
bash scripts/wan/wan_t2v_720p_ear.sh # SVG-EAR

The naming encodes the mechanism. The svg script is Sparse VideoGen 1, the sap script is Sparse VideoGen 2, and the ear script is SVG-EAR. The image-to-video set follows the same pattern with the i2v scripts.

HunyuanVideo has the same structure:

# bash scripts/hyvideo/hyvideo_t2v_720p_svg.sh # SVG
bash scripts/hyvideo/hyvideo_t2v_720p_sap.sh # SVG2
bash scripts/hyvideo/hyvideo_t2v_720p_ear.sh # SVG-EAR

The repository root also has standalone Python entry points for several model and task combinations, including wan_t2v_inference.py, wan_i2v_inference.py, hyvideo_t2v_inference.py, hyvideo_i2v_inference.py, cosmos_t2v_inference.py, cog_inference.py and orig_hyvideo_inference.py, the last named to make the unmodified baseline easy to compare against.

That baseline file is worth noticing. The end-to-end speedup table in the README is commented out, so the comparison you would want to run is one you have to set up yourself.

What the speedup claim actually is, and what the README does not show

The claim is stated in the news section rather than in a benchmark table, and it is consistently about a factor of two.

For Sparse VideoGen, the news entry from March 2025 says that HunyuanVideo and CogVideoX v1.5 can be accelerated by 2 times. For Sparse VideoGen 2, the September 2025 entry says HunyuanVideo, Wan 2.1 and Cosmos can be accelerated by 2 times. The project description itself says the frameworks accelerate video generation with high pixel-level fidelity.

The Efficiency Benchmark section exists but its end-to-end speedup table is commented out in the README, so the numbers are not visible in the document. A Customized Kernels Performance subsection begins, stating that performance is evaluated, and is then cut off.

That gap is the honest limitation of this repository as a document. The papers are where the evaluations live, with SVG 1, SVG 2 and SVG-EAR each having an arXiv entry linked from the header, and the claim of 2 times is repeated in the news items. What the README does not give you is the end-to-end table, the resolution and hardware behind each figure, or the quality metric behind the phrase high pixel-level fidelity. There is no inference code in the repository that shows a quality comparison, and the examples directory is numbered from 1 to 7 with no description of what each contains.

So the honest reading is that the acceleration factor is a headline from the papers, and the details are one click away rather than in the repository.

Publication record and the road ahead

The publication history is a series of dated milestones, and it explains the three-part structure of the codebase.

Sparse VideoGen was open-sourced in March 2025 and accepted by ICML 2025 in May of that year. Wan 2.1 support arrived in April 2025, with both T2V and I2V accelerated. Sparse VideoGen 2 was open-sourced in September 2025 and accepted by NeurIPS 2025 as a spotlight. In the same month, the project released Flash k-Means, a batched K-Means clustering algorithm implemented with Triton that it says offers more than a 10 times speedup. SVG-EAR was accepted by ECCV 2026 in June 2026.

The open-source plan at the end of the README is a checklist, and it reads as the current state of the project. Support for FP8 attention is unchecked. Support for SVG-EAR, Wan 2.1 and Cosmos are all checked. Cosmos in particular is named in the news as accelerated by Sparse VideoGen 2, and the repository has a cosmos_t2v_inference.py entry point.

The repository is not archived and the last push was on 2026-07-04. There are no GitHub releases, and the package version in pyproject.toml is 0.1.0, which is a placeholder rather than a released version. So the practical version pinning story is: pin a commit, not a release.

The dependency list tells you what this really is

The pyproject.toml is where the real requirements show, and they explain both the capability and the fragility of the project.

The base dependencies include torch at 2.0.0 or later, torchvision, numpy, scipy, scikit-learn, pillow, tqdm, matplotlib, einops and loguru, plus diffusers pinned to exactly 0.34.0 and transformers at 4.49.2 or later. The exact pin on diffusers is the kind of detail that causes friction: it means a project that needs a newer diffusers for something else cannot simply take both.

accelerate, ftfy, imageio, imageio-ffmpeg, pytest, IPython, termcolor, tabulate, lpips, scikit-image and pandas are also listed. Two of those are worth singling out. lpips is a perceptual similarity metric, which is how the project would measure the quality side of a speedup for video frames, and accelerate is the library that handles the device and dtype plumbing across the underlying diffusion pipeline.

The package is built with hatchling, requires Python 3.8 or later, is licensed Apache-2.0, and is authored under the name FirstIntelligence. The lint configuration is ruff with black, targeting Python 3.10, and the ruff lint rules extend the defaults with isort, pycodestyle errors and pyflakes, with a set of ignores that include line length.

The API key is in the LICENSE file, and the licence identifier is Apache-2.0.

Editorial conclusion

Sparse-VideoGen is a good fit for a research or production team running open video diffusion models on NVIDIA hardware and willing to maintain a stack of custom CUDA kernels, because the frameworks are training-free, so they drop into an existing checkpoint rather than requiring a fine-tune. It is a poor fit for a team without the hardware and the build patience, since the install requires compiling kernels, FlashInfer and cuVS, and it is a poor fit if you need a stable API, because the repository publishes no releases and the README's own end-to-end speedup table is commented out. Before adopting, confirm your CUDA and PyTorch versions against the recommended 12.4 or 12.8 with PyTorch 2.5.1 or 2.6.0, and check whether your model and task combination is one of the supported ones, since the run scripts are provided per model and per variant.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. svg-project/Sparse-VideoGen on GitHub