# Diffusion Forcing has two branches and a successor you should read first

> The diffusion-forcing repository is the code for a NeurIPS paper on combining next-token prediction with full-sequence diffusion, and its README opens by pointing at a v2 release that supersedes it for video. What remains here is worth reading for three things: the branch split between a reimplementation with temporal attention and the original RNN code, a Hydra command line that exposes every experiment setting, and a documented limit on how far a model rolls out past its training length.

**buoyancy99/diffusion-forcing** — code for "Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion"

- Repository: https://github.com/buoyancy99/diffusion-forcing
- Stars: 1,307 · Forks: 72
- Language: Python
- License: NOASSERTION
- Published: 2026-10-17 · Updated: 2026-10-17 · Language: en
- Canonical page: https://hysenlabs.com/projects/buoyancy99-diffusion-forcing

## The README tells you to look somewhere else first

The first paragraph after the author list is a redirect, and it is worth reading carefully before anything else.

It states that Diffusion Forcing v2 has been released, describes it as a stronger technique for rolling out extremely long video generation using modern architectures like DiT and latent diffusion, and gives a separate repository for it. The author adds that they will reimplement diffusion forcing 2 on the robot and planning domains in this repository when they have time.

So the project's own author considers v2 the better implementation for the headline application, and this repository is where the planning and robotics work lives.

The Project Instructions section repeats the same triage in more operational terms. If you are interested only in higher quality video generation, use the v2 repository. If you want a cleaner codebase and the other domains, keep reading here. If you want the latest implementation for video and planning with temporal attention, stay on this branch. If you want to reproduce the claims from the original paper, check out the branch used by the original paper with `git checkout paper`.

That is four cases and three destinations, and the distinction between main and paper is the one that catches people.

Main contains the latest reimplementation using temporal attention, which the README calls recommended. The paper branch contains the RNN code used by the original paper, kept for reproduction.

That distinction matters because the code you get determines which results you can reproduce. The published numbers came from the RNN code, so if you are checking a claim against the paper, main is the wrong branch.

## Setup is a pinned environment and a wandb account

The setup section is short and has one non-obvious requirement.

The environment is Python 3.10 through conda, named `diffusion-forcing`, and dependencies come from a single requirements file:

```
conda create python=3.10 -n diffusion-forcing
conda activate diffusion-forcing
pip install -r requirements.txt
```

Then there is wandb. You are asked to sign up for a wandb account for cloud logging and checkpointing, run `wandb login`, and then modify the wandb entity in `configurations/config.yaml` to your own account.

That last step identifies the account to use for the project's cloud logging and checkpointing.

The maze planning setup has an extra caveat that is honest about a real dependency problem. If you want maze planning, you are told to install additional complicated dependencies because of the outdated dependencies of d4rl, which involves first installing mujoco 210 and then running a separate `extra_requirements.txt`.

The requirement file itself is a research codebase's dependency list and shows the shape of the work. `torch~=2.4.0` with `torchvision~=0.19.1`, `lightning~=2.1.2` for training, `hydra-core~=1.3.2` with `omegaconf~=2.3.0` for configuration, and `wandb~=0.17.0` for tracking.

Three packages in that list are pinned exactly rather than with a compatible-release operator: `torchmetrics[image]==0.11.4`, `wandb-osh==1.2.1` and `gluonts[torch]==0.13.1`. An exact pin usually means the code depends on a specific behaviour, or that a later version broke something nobody had time to chase.

The domain-specific set includes `pytorchvideo`, `pyrealsense2`, `diffusers`, `rotary_embedding_torch`, `h5py`, `pyzmq`, `moviepy`, `opencv-python` and `internetarchive`.

## Hydra makes the experiment a command line

The quick start commands are the clearest thing in the README about how the project is meant to be used.

Autoregressively generating Minecraft video with a pretrained checkpoint:

```
python -m main +name=sample_minecraft_pretrained load=outputs/minecraft.ckpt experiment.tasks=[validation]
```

Read it as Hydra syntax. `+name=...` is an addition to the configuration rather than an override, `load=` names the checkpoint, and `experiment.tasks=[validation]` is a list of tasks.

Every part of the experiment is addressable from the command line, which is what makes a research codebase usable at all. In the maze planning examples the commands carry the observation normalisation statistics inline:

```
python -m main experiment=exp_planning algorithm=df_planning dataset=maze2d_medium dataset.action_mean=[] dataset.action_std=[] dataset.observation_mean=[3.5092521,3.4765592] dataset.observation_std=[1.3371079,1.52102] load=outputs/maze2d_medium_x.ckpt experiment.tasks=[validation] algorithm.guidance_scale=3 +name=maze2d_medium_x_sampling
```

Those mean and standard deviation vectors are per-dataset normalisation constants, and they differ between the medium and large mazes. Embedding them in the command rather than a config file means the run is fully described by the line you can copy.

`algorithm=df_planning` selects the algorithm, `dataset=maze2d_medium` the dataset group, `algorithm.guidance_scale=3` the guidance scale, and `+name=` the run name that appears in wandb.

The repository root confirms the structure: `main.py` at the top, then `algorithms/`, `configurations/`, `datasets/`, `experiments/`, `scripts/`, `utils/` and a `debug/` directory.

## Rolling out past the training length is one flag, and the caveat matters

The claim at the centre of the paper is that a model trained on a fixed sequence length can roll out longer than it was trained, and the README explains how to test that with a single appended argument.

To let the model roll out longer than it was trained on, you append `dataset.validation_multiplier=8`, and it will roll out eight times longer than the maximum sequence length it was trained on.

That is the demonstration of the paper's central claim, and it is a one-flag change to an existing command, which is exactly how such a claim should be checkable.

The caveats around it are as informative as the claim.

First, the visualisation is side by side with prediction on the left and ground truth on the right, and the README says the ground truth is expected not to align with the prediction because the sequence is highly stochastic. It is there to give an idea about quality only. So do not read a visual comparison as an accuracy measurement.

Second, the checkpoint is trained for 100K steps with a small number of frames, and the README says the authors have verified diffusion forcing works in the latent diffusion setting and can be extended to many more tokens without sacrificing compositionality, with some addition techniques outside this repository.

Third, and most usefully, the README documents experiments that did not work. There is a commented-out position and velocity setting, described as not recommended because diffusing a quantity and its derivative together creates a bad optimisation landscape.

That is a real finding from a negative result, and it is the kind of thing that saves a student a week.

There is also a checkpoint where the actions are diffused, described as a challenging setting not done in prior papers, which the authors say they have not got working as well as the original RNN version, though it has okay numbers.

## Where this repository is the wrong thing to reach for

Three cases.

If you want long video generation, this is not the repository. The README says so in its first paragraph and points to Diffusion Forcing v2 with a separate repository built on DiT and latent diffusion.

If you want to reproduce the paper's numbers, main is the wrong branch. The RNN implementation lives on the paper branch, kept for exactly that purpose.

If you need a maintained library, this is a paper repository. It has no releases, the requirements are pinned to a PyTorch generation, and the maze planning path depends on d4rl, whose own outdated dependencies are why an extra installation step exists at all.

There is also a practical constraint on the data. The README explains that the dataset is huge, which is why a mini subset and pretrained checkpoints are provided as a tarball to extract into the project root. The checkpoints were released after the fact, and the README tells you to pull the upstream code first if you forked earlier, which is a specific warning about checkpoint and code compatibility.

What it is genuinely good for is understanding how a diffusion model over sequences is set up in practice. The Hydra configuration, the inline normalisation constants, the wandb integration and the documented negative results are the parts that generalise, whatever you do with the technique.

The last push was on 2026-07-06, and the paper was published in Advances in Neural Information Processing Systems volume 37, pages 24081 to 24125, in 2025.

## Conclusion

Adopt this repository if you want the original Diffusion Forcing formulation applied to video prediction or maze planning and can work inside a pinned PyTorch 2.4 environment. Skip it for long video generation, since the README directs you to Diffusion Forcing v2 and its separate repository. Verify first which branch you need, because main carries the temporal attention reimplementation and the paper branch carries the RNN code the published results came from.

## FAQ

### Which branch of diffusion-forcing should I use?

Main contains the latest reimplementation with temporal attention and is the recommended branch, while the paper branch contains the RNN code used by the original paper for reproduction. For video generation specifically, the README points you to Diffusion Forcing v2 and its separate repository instead.

### How do I test whether a diffusion forcing model rolls out beyond its training length?

Append `dataset.validation_multiplier=8` to the sampling command, which rolls the model out eight times longer than the maximum sequence length it was trained on. The README warns that the side-by-side ground truth will not align with the prediction because the sequence is highly stochastic, and is provided for quality indication only.

### What does running diffusion forcing require besides the requirements file?

A wandb account for cloud logging and checkpointing, a `wandb login`, and editing the wandb entity in `configurations/config.yaml` to your own account. Maze planning additionally needs mujoco 210 and `extra_requirements.txt`, because of the outdated dependencies of d4rl.

## Sources

- [buoyancy99/diffusion-forcing on GitHub](https://github.com/buoyancy99/diffusion-forcing)
- [Issues](https://github.com/buoyancy99/diffusion-forcing/issues)
- [README](https://github.com/buoyancy99/diffusion-forcing/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/buoyancy99-diffusion-forcing