rCM distills video diffusion into two to four steps, and Causal-rCM adds the autoregressive recipe

A distillation recipe rather than a turnkey model

rCM is the code release behind a research effort, and the README frames it as an algorithm and infrastructure recipe for diffusion distillation and causal training. The project description calls it leading and unified algorithms and infrastructures for bidirectional and autoregressive video diffusion distillation at scale, and the header describes rCM as a score-regularized continuous-time consistency model with Causal-rCM as a unified teacher-forcing and self-forcing open recipe for streaming video generation and interactive world models.

Four claims are listed as the first work the project delivers.

It scales up continuous-time consistency distillation, in the form of sCM and MeanFlow, to video diffusion models with 10B or more parameters. It open sources a FlashAttention-2 Jacobian-vector product kernel with support for parallelisms such as FSDP and CP. It identifies the quality bottleneck of sCM and addresses it with a forward-reverse divergence joint distillation framework, showing how a consistency model as a forward-divergence offline method can complement DMD as a reverse-KL on-policy method in enhancing diversity. And it delivers models that generate videos with high quality and strong diversity in only two to four steps.

That last number is the headline for anyone who has tried to run a video diffusion model. The comparison table in the README is on Wan2.1 T2V 1.3B at four steps against DMD2 and SiD, and there is a separate table showing performance under one to two steps, plus five random videos from a distilled Wan2.1 T2V 14B at four steps.

The paper is on arXiv at 2510.08431, a second paper covers the causal work at 2606.25473, and the project is marked as ICLR 2026. There is a project website under research.nvidia.com/labs/dir/rcm. The repository publishes no GitHub releases at all.

The JVP kernel is the load bearing piece

A Jacobian-vector product is the directional derivative of a function, and in consistency distillation it is what lets a student model be trained to match the teacher's output direction along a noise path rather than matching a sample point. The computational cost of that is a second forward pass through the network, and on a 10B parameter video model the memory pressure of two passes is the thing that decides whether the recipe fits on the hardware you have.

That is why the README lists an open-sourced FlashAttention-2 Jacobian-vector product kernel with support for parallelisms like FSDP and CP as a headline item rather than an implementation detail. The JVP is fused into the attention computation instead of being built out separately, which is what makes scaling to 10B+ parameters possible at all.

The comparison figure in the README is captioned as an infrastructure comparison with other codebases, and it is the asset to look at before planning a training run. It sits alongside the note that training and inference are based on native PyTorch, completely free from accelerate and diffusers.

That last detail is a design choice with consequences. Building the training loop directly on PyTorch rather than on diffusers means the code does not inherit the diffusers model conventions, the pipeline abstractions or the accelerate launcher, and it means anything you already know about those libraries does not transfer. In exchange, the training path is a single stack with no adapter layer in the middle, which is what makes the kernel level optimisations possible.

The repository layout reflects the split: rcm holds the main implementation, imaginaire is a separate module directory, evaluation holds the evaluation suite, scripts holds the entry points, and assets holds the figures and checkpoints the README references.

Building the environment means building FlashAttention from source

The environment setup section is a long list of shell commands, and the list is not optional. The training and inference stack is native PyTorch, so the FlashAttention builds are compiled rather than pulled from a wheel.

The sequence starts with a conda environment on Python 3.12.12, cmake and ninja, the NVIDIA CUDA compiler and toolkit, and then a pinned PyTorch and torchvision pair installed from the cu126 wheel index.

conda create -n rcm python==3.12.12
conda activate rcm
conda install cmake ninja
conda install -c nvidia cuda-nvcc cuda-toolkit
# depending on your cuda version
pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cu126

The rest of the dependency install is a single long pip line covering megatron-core, hydra-core, loguru, attrs, fvcore, nvidia-ml-py, imageio with ffmpeg, pandas, wandb, psutil, ftfy, regex, transformers, webdataset and safetensors. Transformer Engine is installed with the no-build-isolation flag.

Then FlashAttention itself, cloned and checked out at a specific tag and installed with MAX_JOBS set to 4.

git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention
git checkout v2.7.4.post1
MAX_JOBS=4 python setup.py install

The optional backends are where the hardware generation matters. There is a FlashAttention-3 dense attention backend on Hopper, a FlashAttention-4 CuTeDSL backend for PyTorch FlexAttention on Hopper and Blackwell, and a MagiAttention backend for masked attention, which is the relevant one for causal training.

export RCM_FLEX_BACKEND=flash  # use "auto" or "triton" to fall back to PyTorch FlexAttention defaults

export RCM_ATTENTION_BACKEND=magi

Both exports are plain environment variables, so switching backends is a matter of the shell environment rather than a code change. That is convenient, and it also means the repository supports several hardware paths without you committing to one.

Causal-rCM and the teacher-forcing versus self-forcing split

The newer part of the release is the causal and autoregressive stack, and the conceptual distinction in the README notice is worth spelling out.

The notice says rCM now includes a causal and autoregressive training stack, showing how teacher-forcing, described as forward-divergence and offline, consistency modeling complements self-forcing, described as reverse-divergence and on-policy, DMD in autoregressive video diffusion distillation.

So the same repository now covers two regimes. The bidirectional regime distills a video model into a few-step generator that produces a whole clip at once. The causal regime distills a model that generates a stream, which is what an interactive world model needs, since it has to emit frames conditioned on what came before without re-running the entire context.

The Causal-rCM section adds that the project provides a causal distillation recipe, with details in Causal_rCM.md. The included extras are listed as optimized causal inference with time benchmarking, a quantized KV cache, and bounded-memory length extrapolation, plus a simplified and reproducible VBench evaluation suite.

The quantized KV cache and the bounded-memory length extrapolation are the two items that matter operationally. A streaming generator accumulates context, and without those two the context grows until inference becomes impractical. The README also mentions a state of the art causal distillation recipe in the same breath as the algorithm description, which is the research claim rather than a specification.

There is a separate Wan.md file at the repository root, which in this repository's naming convention is where the Wan model specifics live, and evaluation holds the evaluation suite with the VBBench images referenced in the overview.

Inference needs a distilled checkpoint you supply

The inference section gives one example script for running rCM on text to video, and the argument list tells you most of what you need to know about the shape of the tool.

# Basic usage:
#   PYTHONPATH=. python rcm/inference/wan2pt1_t2v_rcm_infer.py [arguments]

# Arguments:
# --model_size         Model size: "1.3B" or "14B" (default: 1.3B)
# --num_samples        Number of videos to generate (default: 1)
# --num_steps          Sampling steps, 1–4 (default: 4)
# --sigma_max          Initial sigma for rCM (default: 80); larger choices (e.g., 1600) reduce diversity but may enhance quality
# --dit_path           Path to the distilled DiT model checkpoint (REQUIRED for inference)
# --vae_path           Path to Wan2.1 VAE (default: assets/checkpoints/Wan2.1_VAE.pth)

Two of those arguments carry the interesting trade-offs. The num_steps argument accepts one to four with a default of four, and the sigma_max argument defaults to 80 with a note that larger choices such as 1600 reduce diversity but may enhance quality. That is the quality against diversity dial for a consistency model, expressed as a single parameter, and it is the parameter you would reach for first when a generation looks flat.

The dit_path argument is marked as required for inference. The script will not run without a distilled DiT checkpoint, and the VAE path defaults to a file under assets/checkpoints. In other words, this repository gives you the distillation and the sampling code, and the model weights are a separate thing you have to obtain or produce.

The repository also ships example assets including seven image files under examples, named i2v_input_1 through i2v_input_7, which are the kind of inputs an image to video run would take.

What the documentation does not settle

The README is dense with claims and thin on a few operational points, and it is worth knowing which gaps you will have to fill yourself.

First, there are no released artefacts. The repository has no GitHub releases, and the last push was on 2026-06-25, roughly four months before the date of this writing. That is a recent enough push to show the code is alive, but there is no version to pin, no changelog to diff against and no stability statement for the Python API. The README describes the project as ICLR 2026 work with a research framing, and a research code release with a paper is a different kind of commitment from a supported library.

Second, the evaluation numbers are presented as videos and as a VBench figure in the assets, with no table of metric values in the README text. The claims of high quality and strong diversity are stated rather than tabulated, and the reproduction instructions are not spelled out in the visible portion of the README. The overview mentions a simplified and reproducible VBench evaluation suite is included, which is the place to look for what the numbers actually are.

Third, the hardware requirements are implicit rather than stated. The optional FlashAttention-3 path is described as a dense attention backend on Hopper, the FlashAttention-4 path as a CuTeDSL backend for Hopper and Blackwell, and MagiAttention is recommended for CUDA 13 while the default install path targets the cu126 wheels of PyTorch. You will have to work out yourself which combination your GPUs support, and the environment variables give you the fallback positions once you do.

Fourth, the training configuration surface is not described in the README at all. The dependency list includes hydra-core and the code lives in rcm and imaginaire, so the configuration system is presumably Hydra based, but the exact entry points and hyperparameters are in the source and in the Wan.md and Causal_rCM.md documents rather than in the front page.

How it compares to running the base model

The alternative to distilling is not distilling, and for video diffusion that comparison is stark.

The base models here are Wan2.1 text to video at 1.3B and 14B parameters, and the teacher video in the comparison table is a full step diffusion sampler. A diffusion sampler needs many denoising steps, and each step is a full forward pass of a very large network over a long sequence of video tokens. The distilled model exists to replace that with two to four passes, or as the README puts it, one to two steps in the reduced setting.

The cost side of the trade is that distillation is a training project. You need the teacher, the student, the JVP kernel, the parallelisms and the GPU memory to hold a 10B parameter model and run a second forward pass through it. You cannot get a distilled checkpoint from pip or from a model hub listing in this repository, the dit_path argument is required, and the checkpoint has to come from somewhere else.

The other alternative is to use an existing distilled checkpoint for Wan2.1 if one exists in the ecosystem, rather than reproducing this recipe. That trades the ability to train your own distillation, for instance on your own data or with your own teacher, for the convenience of a ready model. The README's contribution claim is precisely that rCM is the open recipe, which is the argument for running it yourself.

There is also the third path of not distilling at all and accepting the sampling cost of the teacher model, which is entirely reasonable for offline batch generation where a few extra minutes per clip does not matter. The distillation argument is strongest when you are generating a lot of video, or when latency matters.

Editorial conclusion

rCM fits a research team with NVIDIA GPUs, CUDA toolchain access and video diffusion expertise that wants to distill a Wan2.1 text to video model into a two to four step generator, or to train an autoregressive streaming model, and that is willing to build FlashAttention from source. It does not fit an application team that wants a stable video generation API, since the repository publishes no GitHub releases, its last push was on 2026-06-25, and it carries an ICLR 2026 research framing rather than a support commitment. Before adopting, read Causal_rCM.md and Wan.md in full, confirm your CUDA and GPU generation against the optional FlashAttention-3 and FlashAttention-4 paths, and check the distilled DiT checkpoint requirement, because the inference script will not run without one.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. NVlabs/rcm on GitHub
4. Project website
5. README