StreamDiffusionV2: a streaming video diffusion pipeline

A diffusion model that outputs frames while it works

The paper is the reason to pay attention, and the README leads with it. StreamDiffusionV2 is described as a streaming system for dynamic and interactive video generation, marked as the MLSys 2026 best paper, with the arXiv entry 2511.07399 and a long author list spanning UT Austin, UC Berkeley, Nunchaku AI, Stanford, MIT and others, with Chenfeng Xu named as project lead and corresponding author. The overview describes it as an open source interactive diffusion pipeline for real-time streaming applications, one that scales across diverse GPU setups, supports flexible denoising steps, and delivers high frame rates for creators and platforms. The project page is a GitHub Pages site, there is a Hugging Face space, and the model checkpoint was released on Hugging Face in October 2025. The timeline in the news section tells the story of how the pieces appeared, with the public release in October 2025, the paper in November, acceptance to MLSys in January 2026, the PyPI package and the optional TAEHV decoder in March, a ring-buffer KV cache update for sliding window attention in March, and initial Blackwell GPU support in May, followed by the best paper award on the eighteenth of May 2026.

The chunked encode, denoise and decode loop

The usage example shows the API, and the shape of it is the whole design. A pipeline is created against a checkpoint folder with a mode, a prompt is prepared, a source video is loaded at a given height and width, and then the video is iterated in chunks.

stream = StreamDiffusionV2Pipeline(
    checkpoint_folder="ckpts/wan_causal_dmd_v2v",
    mode="single",
)
stream.prepare("A dog walks on the grass, realistic")

for video_chunk in stream.chunk_video(video):
    encoded_chunk = stream.encode_chunk(
        video,
        video_chunk,
        previous_noise_scale=noise_scale,
        initial_noise_scale=stream.noise_scale,
    )
    noise_scale = encoded_chunk.noise_scale
    denoised_chunk = stream.denoise_chunk(encoded_chunk)
    if denoised_chunk is None:
        continue
    decoded_chunks.append(stream.decode_chunk(denoised_chunk))

The noise scale is the state that flows through the loop. Each chunk is encoded with the previous chunk's noise scale and the pipeline's initial noise scale, and the encoded chunk carries a new noise scale forward, so the sequence of chunks is temporally consistent rather than independent samples. The denoise step can return nothing, and the example skips those chunks, which is how the pipeline handles a chunk it decides not to emit. Decoded chunks are concatenated along the frame axis and written out with a frame rate, in the example 16 fps. The alternative mode shown in the README, single-wo, is the same loop without the stream batch feature, and the package registers three console scripts for the single, single without batch, and pipe variants.

Checkpoints: a Wan base model plus a project checkpoint

The checkpoint section is where the real download volume is, and the README is explicit about what goes where. There are two base models, the 1.3B and the 14B version of Wan-AI's Wan2.1 text to video model, and on top of them a project checkpoint published under the jerryfeng account on Hugging Face, with separate subdirectories for the 1.3B and 14B variants.

# 1.3B Model
huggingface-cli download --resume-download Wan-AI/Wan2.1-T2V-1.3B --local-dir wan_models/Wan2.1-T2V-1.3B
huggingface-cli download --resume-download jerryfeng/StreamDiffusionV2 --local-dir ./ckpts --include "wan_causal_dmd_v2v/*"

# 14B Model
huggingface-cli download --resume-download Wan-AI/Wan2.1-T2V-14B --local-dir wan_models/Wan2.1-T2V-14B
huggingface-cli download --resume-download jerryfeng/StreamDiffusionV2 --local-dir ./ckpts --include "wan_causal_dmd_v2v_14b/*"

The include filter is what keeps the two routes separate, since the project repository holds both model sizes and the pattern picks the one matching the base model you downloaded. The README also notes that the 14B model it uses comes from CausVid-Plus and is the one used for the offline inference demo, which is a detail worth knowing if you are reproducing the published results. There is a third, optional artefact, the lightweight TAEHV decoder checkpoint, which the code can fetch on first use but which you can place yourself to avoid that startup step.

curl -L https://github.com/madebyollin/taehv/raw/main/taew2_1.pth -o ckpts/taew2_1.pth

Installation on Linux with CUDA, and the Blackwell exception

The prerequisites are two lines, and they are not negotiable. The operating system is Linux with an NVIDIA GPU, and you need a CUDA compatible GPU and drivers. The installation section then creates a conda environment on Python 3.10 and installs the package from PyPI.

conda create -n streamdiffusionv2 python=3.10 -y
conda activate streamdiffusionv2

# PyPI
pip install streamdiffusionv2

# Needed for Blackwell GPUs
pip install torch==2.11.0 torchvision==0.26.0

# Optional but recommended for better throughput
pip install "streamdiffusionv2[flash-attn]"

Two details deserve attention. The Blackwell line, which pins torch 2.11.0 and torchvision 0.26.0, was added in May 2026 and overrides the versions the package itself pins, which are torch 2.6.0 and torchvision 0.21.0. So on a new generation card you are expected to install over the package's pins, and on older hardware you keep them. The flash attention extra is optional and is described as recommended for better throughput, and the package pins flash_attn at 2.7.4.post1 when you take it. A local checkout is installed the same way with pip install . and the same extra syntax, and the README says the Python dependencies for both offline inference and the demo backend are included, while the demo frontend separately needs Node.js 18.

The demo backend, FastAPI and websockets

The repository carries two distinct surfaces. There is the library, the streamdiffusionv2 package, and the streamv2v directory with the inference entry points that the console scripts point at, which is where the single, single without batch and pipe variants live. And there is the demo, a separate directory with its own readme, a main.py, a config.py, a connection_manager.py, a vid2vid.py and a vid2vid_pipe.py for the video to video paths, a start.sh, a test for online inference, and a frontend directory. The dependency list explains the shape of that demo, since it includes fastapi and uvicorn for the server side, websockets for the connection to the browser, and the usual inference stack of torch, torchvision, diffusers and transformers, all pinned to specific versions along with numpy, Pillow, imageio, av for video handling, scikit-image, omegaconf for configuration and pydantic for schemas. There is also a run_v2v.sh at the root for launching the pipeline, a configs directory, an examples directory holding sample media, and a models and tools directory. The keywords in the packaging metadata name fastapi alongside wan and streaming inference, which is the whole architecture in three words: a video diffusion model behind a network service that streams frames to a browser.

Versions, pins, and what the packaging promises

The packaging metadata describes a project that is still moving. The distribution is named streamdiffusionv2 at version 0.1.1, the licence is Apache-2.0, and the development status classifier says Beta, with the intended audience listed as science and research users and the operating system as Linux only. The Python requirement is a range, from 3.10 up to but not including 3.13, which is wider than the Python 3.10 that the conda command in the readme creates. The runtime dependencies are pinned hard rather than ranged, and the pins are what you would expect for a reproducible diffusion stack, with torch 2.6.0, torchvision 0.21.0, torchaudio 2.6.0, diffusers 0.35.1, transformers 4.54.0, accelerate 1.10.1, einops 0.8.1, numpy 1.24.4, Pillow 10.1.0, pydantic 2.10.6, av 13.1.0, imageio 2.37.0, scikit-image 0.25.2, omegaconf 2.3.0, uvicorn 0.37.0 and websockets 16.0. Pinning everything is the right call for a research reproduction, and it is also the source of friction when a new GPU generation needs a newer torch, as the Blackwell instructions show. The repository publishes no GitHub releases, so the version on PyPI is the thing to track, and the last push was on 2026-09-14. The arXiv paper and the checkpoint on Hugging Face are the two artefacts that will outlive any particular commit.

Editorial conclusion

Adopt StreamDiffusionV2 if you need a video model that produces frames incrementally, for an interactive demo, a live preview tool or a video to video front end, and you have an NVIDIA GPU and the patience to download multi gigabyte checkpoints. It does not fit a workstation without a CUDA capable GPU, and it does not fit a Mac or a CPU only server, because the prerequisites are explicit about that. Before you start, read the checkpoint section, because the 1.3B and 14B routes need different base models and different project checkpoints, and note the version tension in the packaging, where torch 2.6.0 is pinned while the Blackwell instructions ask for torch 2.11.0.

Frequently asked questions

Official sources

1. chenfengxu714/StreamDiffusionV2 on GitHub
2. Issues
3. License: Apache-2.0
4. README