# FastGen: NVIDIA's framework for distilling diffusion models into fast generators

> FastGen is an Apache-2.0 PyTorch framework for building fast generative models with distillation and acceleration methods, with training and inference entry points, Hydra-style config overrides, DDP and FSDP2 support and network backbones including EDM, SDXL, Flux, Qwen-Image, WAN and Cosmos.

**NVlabs/FastGen** — NVIDIA FastGen: Fast Generation from Diffusion Models

- Repository: https://github.com/NVlabs/FastGen
- Website: https://developer.nvidia.com/blog/accelerating-diffusion-models-with-an-open-plug-and-play-offering/
- Stars: 1,036 · Forks: 84
- Language: Python
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/nvlabs-fastgen

## A training framework, not a set of checkpoints

FastGen is a PyTorch-based framework for building fast generative models using various distillation and acceleration techniques. The phrasing matters: this is a framework for training models.

The three capabilities the README claims are the scope. It supports large-scale training with 10B parameters or more. It handles different tasks and modalities, specifically T2I for text to image, I2V for image to video, and V2V for video to video. And it supports various distillation methods, including consistency models, distribution matching distillation and self-forcing.

That combination is the point. A research group that wants to try consistency distillation on a video model and distribution matching distillation on a text-to-image model would otherwise be assembling two codebases. FastGen puts the methods, the networks and the trainer in one repository, with the trainer as a single `train.py` entry point.

The authors are listed at the top of the README: Weili Nie, Julius Berner, Chao Liu and Arash Vahdat, and the homepage points to an NVIDIA developer blog post about accelerating diffusion models with an open plug-and-play offering. The licence is Apache-2.0, there is a single release tag at v0.1.0 from 2026-07-24, and the last push was on 2026-08-21.

The version number is worth noticing on its own. A framework at 0.1.0 with a single release tag is early, and the PyTorch badge in the README says 2.0 or later while the requirements ask for torch 2.6.0 or later.

## The directory layout is the architecture

The repository structure section in the README is a tree, and it reads as a design document.

The `fastgen/` package holds `trainer.py` as the main training loop, `methods/` for the training methods, listed as CM, DMD2, SFT and KD among others, `networks/` for the neural network architectures, `datasets/` for the dataset loaders, `configs/` for the configuration system with separate `experiments/` and `methods/` subdirectories, `callbacks/` for training callbacks such as EMA and profiling, `utils/` for distributed and checkpointing utilities, and `third_party/` for third-party dependencies.

Outside the package, `scripts/` holds inference and evaluation scripts, `tests/` holds unit tests, the `Makefile` holds the development commands for lint, format and test, and `train.py` is the main training entry point.

The `methods/` and `networks/` split is the important one, and it is what makes the framework extensible rather than a single script with switches. A method is a distillation or acceleration technique, a network is a backbone, and a config combines them. Adding a new method means adding a directory under `methods/` and a config under `configs/methods/`, not editing the training loop.

The callbacks directory points at the same design. EMA and gradient clipping and logging are callbacks, not code inside the trainer, so a run can be assembled from a config rather than forked.

## The method and network tables

The documentation table points at a per-component README for each area, and the two method tables are the most concrete part of the repository.

Under consistency models there are CM, sCM, TCM and MeanFlow. Under distribution matching there are DMD2, f-Distill, LADD, CausVid and Self-Forcing. Fine-tuning covers SFT and CausalSFT, and knowledge distillation covers KD and CausalKD. Each entry links to the arXiv paper it comes from, which is how you check that the implementation matches the method you have in mind.

The networks table is broader still: EDM, SD, SDXL, Flux, Qwen-Image, WAN, CogVideoX and Cosmos, along with pretrained models. EDM is the one used in the quick start example, and it is the backbone the diffusion literature usually uses when a method paper reports numbers, which makes it the sensible default for a first run.

The rest of the documentation table covers the configuration system with its environment variables and instructions for creating custom configs, dataset preparation and WebDataset loaders, training callbacks for EMA and logging and gradient clipping, inference modes covering T2I, T2V, I2V and V2V along with FID evaluation, and the third-party dependencies such as Depth Anything V2.

WebDataset loaders are worth a mention for anyone with a large image corpus, because the alternative of shuffling a directory of individual files is a known bottleneck and the repository reaches for the sharded format instead.

## Installing and the first training run

The recommended environment is the Docker container, and CONTRIBUTING.md holds the setup instructions. The alternative is a conda environment:

```bash
conda create -y -n fastgen python=3.12.3 pip; conda activate fastgen
```

Then install the package from a clone:

```bash
git clone https://github.com/NVlabs/FastGen.git
cd FastGen
pip install -e .
```

The install pulls in the requirements file, which is a real dependency set: diffusers 0.35.1, transformers 4.49.0, accelerate, hydra-core, webdataset, timm, boto3, av, einops, loguru, safetensors, wandb with media, and torch 2.6.0 or later, with numpy pinned below 2.0.0.

Before training you need data and a starting checkpoint:

```bash
python scripts/download_data.py --dataset cifar10
```

Then the training run itself:

```bash
python train.py --config=fastgen/configs/experiments/EDM/config_dmd2_test.py
```

Two details in that command matter. The config path names the experiment and the method together, so the DMD2 test config on the EDM backbone is a complete description of the run. And the config is a Python file, since Hydra loads it, which means the config is code and can be typed.

If you run out of memory, the README's advice is to lower the batch size, for example `dataloader_train.batch_size=32`, and the framework automatically uses gradient accumulation to keep the global batch size the same.

## Overriding configs, multi-GPU runs and where the output goes

Any config parameter can be overridden on the command line with Hydra-style syntax, and the README notes the separator is a dash:

```bash
python train.py --config=path/to/config.py - key=value nested.key=value
```

That single mechanism covers the whole space of experiments, which is why a framework built around one trainer can still support a large method matrix.

For multi-GPU training the entry point is torchrun, and the distributed flag is set with an override:

```bash
torchrun --nproc_per_node=8 train.py \
    --config=fastgen/configs/experiments/EDM/config_dmd2_test.py \
    - trainer.ddp=True log_config.name=test_ddp
```

For large models, FSDP2 does the model sharding, and the change is the same shape, replacing `trainer.ddp=True` with `trainer.fsdp=True`.

The output layout is designed for reproducibility. Training outputs go to `$FASTGEN_OUTPUT_ROOT/{project}/{group}/{name}/`, and each run directory holds a `checkpoints/` folder with files named by iteration as `{iteration:07d}.pth`, a `config.yaml` holding the resolved configuration, and a `wandb_id.txt` holding the W&B run ID for resuming. Writing the resolved config next to the checkpoints is the detail that makes a run reproducible later, because it records what the overrides actually resolved to rather than what the source config said.

For logging, W&B needs credentials. You can save the API key to `credentials/wandb_api.txt` or set the `WANDB_API_KEY` environment variable, and without either the tool prompts for the key interactively. The configs README also documents S3 storage and other environment variables.

Inference follows the same config-driven pattern, taking the same config plus a checkpoint path, the class count and a prompt file, and the scripts README covers the other inference modes and FID evaluation.

## The container, and what to expect from version 0.1.0

The Dockerfile is built for GPU nodes and shows the scale of the intended environment.

It starts from the NVIDIA PyTorch container `nvcr.io/nvidia/pytorch:25.06-py3`, takes the target architecture as a build argument so amd64 and arm64 are handled automatically, and installs the system dependencies for a native extension build: build-essential, ninja-build, cmake, git, curl and unzip, among others. It installs s5cmd at a pinned version with an architecture-aware download, which is an S3 client, and that is consistent with the boto3 dependency and the S3 storage support in the configuration.

Two environment variables are about performance rather than correctness. `MAX_JOBS=4` limits parallel compile jobs, and `TORCH_CUDA_ARCH_LIST` is set to a list of compute capabilities, so the compiled kernels target a fixed list of compute capabilities. The Python environment variables disable bytecode writing and unbuffer output, which is standard for containers whose logs you read in real time.

The framework also sets a timezone explicitly, which is a small detail that matters for timestamped training logs.

At version 0.1.0 with a single release tag and a last push on 2026-08-21, this is a young framework. The README names the supported methods and networks precisely, which suggests the authors want it used, and the Docker image and CONTRIBUTING.md are written for people who are not its authors. But the version number tells you to expect the configuration format and the method APIs to move.

## Conclusion

FastGen fits a research team training accelerated text-to-image, image-to-video or video-to-video models, that wants several distillation methods, several backbones and one training loop rather than three separate codebases, and that has NVIDIA GPUs to spare. It does not fit a project that wants a pretrained checkpoint to use today, since the repository ships a training framework and an example config rather than a ready model, and it is not a consumer inference tool. Before running anything, read CONTRIBUTING.md for the Docker setup, download the CIFAR-10 data and the pretrained EDM models with `scripts/download_data.py`, and set `WANDB_API_KEY` or a credentials file so runs are logged from the first step.

## FAQ

### What is NVIDIA FastGen?

It is a PyTorch-based framework for building fast generative models using distillation and acceleration techniques. It supports large-scale training with 10B or more parameters, tasks including T2I, I2V and V2V, and methods including consistency models, distribution matching distillation and self-forcing.

### How do I install FastGen?

The recommended route is the provided Docker container, with the instructions in CONTRIBUTING.md. Otherwise create a conda environment with `conda create -y -n fastgen python=3.12.3 pip`, then clone the repository and run `pip install -e .`.

### Which distillation methods does FastGen implement?

Consistency models CM, sCM, TCM and MeanFlow; distribution matching DMD2, f-Distill, LADD, CausVid and Self-Forcing; fine-tuning SFT and CausalSFT; and knowledge distillation KD and CausalKD. Each is linked to its paper in the README.

### How do I train a model with FastGen?

Download the data and pretrained models with `python scripts/download_data.py --dataset cifar10`, then run `python train.py --config=fastgen/configs/experiments/EDM/config_dmd2_test.py`. Any config value can be overridden with Hydra-style syntax, for example `python train.py --config=path/to/config.py - key=value nested.key=value`.

### How do I run FastGen on multiple GPUs?

Launch with torchrun and set the distributed option, for example `torchrun --nproc_per_node=8 train.py --config=... - trainer.ddp=True`. For model sharding with large models, replace `trainer.ddp=True` with `trainer.fsdp=True` to use FSDP2.

## Sources

- [License: Apache-2.0](https://github.com/NVlabs/FastGen/blob/main/LICENSE)
- [NVlabs/FastGen on GitHub](https://github.com/NVlabs/FastGen)
- [Project website](https://developer.nvidia.com/blog/accelerating-diffusion-models-with-an-open-plug-and-play-offering/)
- [README](https://github.com/NVlabs/FastGen/blob/main/README.md)
- [Releases](https://github.com/NVlabs/FastGen/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/nvlabs-fastgen