Fast-dLLM: Four Acceleration Tracks for Diffusion Language Models, Split Across Four Directories
Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"
At a glance
- What is it?
- Fast-dLLM is NVlabs' umbrella repository for speeding up diffusion-based language, vision-language and driving models. v1 is training-free; v2, Fast-dVLM and Fast-dDrive each require their own weights, data and training scripts, so the choice of track matters more than the repository name suggests.
- Who is it for?
- Adopt v1 if you already run Dream or LLaDA checkpoints and want latency reduction without touching weights; the README's own example is a single accelerate launch command against Dream-v0-Base-7B. Do not adopt v2, Fast-dVLM or Fast-dDrive unless you can host the published Hugging Face checkpoints (Fast_dLLM_v2_7B, Fast_dVLM_3B, Fast-dDrive) and, for the latter two, the DeepSpeed and LMFlow training path under third_party/.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 109 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What Fast-dLLM Is Actually Four Repositories Wearing One Name
The top-level README presents a four-column table, and that table is the honest description of the project. Fast-dLLM v1 is training-free inference acceleration for text diffusion models, built on Dream and LLaDA. Fast-dLLM v2 is block diffusion with fine-tuning, built on Qwen2.5. Fast-dVLM converts an autoregressive Qwen2.5-VL model into a block-diffusion vision-language model. Fast-dDrive applies section-aware block diffusion plus scaffold speculative decoding to driving data on Waymo. Each lives in its own directory (v1/, v2/, fast_dvlm/, fast_ddrive/) with its own requirements.txt or installation step and its own README. The shared claim is that diffusion decoding can be made faster without retraining, or with targeted retraining, depending on the track. If you arrive expecting one installable package, you will not find it. The repository is closer to a research monorepo: four papers, four codebases, one licence file.
The v1 Mechanism: KV Cache Plus Confidence-Threshold Parallel Decoding
v1 is the only track that requires no weight changes, and its mechanism is visible directly in the evaluation command. The Dream evaluation passes alg=confidence_threshold alongside threshold=0.9 and use_cache=true. That combination is the whole idea: instead of decoding tokens one position at a time in a fixed order, the sampler evaluates positions in parallel and commits the ones whose confidence clears the threshold, while a key-value cache avoids recomputing attention state for the positions already settled. The README's chat example exposes the same knobs in a simpler form: python llada/chat.py --gen_length 128 --steps 128 --block_size 32. So block size, step count and generation length are all user-facing parameters rather than fixed architecture. A July 2025 changelog entry notes a factor-based parallel strategy added to v1/llada/eval_gsm8k.sh, which suggests the parallel decoding policy itself is still being tuned rather than frozen. The trade-off is inherent to threshold-based commitment: a token committed early cannot be revised, so the threshold is a quality-versus-speed dial, and the README does not publish a table mapping threshold values to accuracy loss.
v2, Fast-dVLM and Fast-dDrive Change the Weights, Not Just the Sampler
The later tracks abandon the training-free premise. v2 uses hierarchical caching on top of block diffusion and ships fine-tuning scripts, DeepSpeed configs under configs/, and a training framework described as an LMFlow fork in v2/src/. Fast-dVLM uses block-size annealing and speculative decoding, and its fine-tuning example is explicitly labelled optional but requires DeepSpeed, the LMFlow fork under third_party/, and the ALLaVA-4V dataset fetched by fast_dvlm/data/download_example_dataset.sh before running fast_dvlm/train_scripts/finetune_multimodal_example.sh from the repository root. Fast-dDrive combines what the README calls SASD training with scaffold speculative decoding and test-time inference scaling. The practical consequence is that v2, Fast-dVLM and Fast-dDrive are not drop-in accelerators for an arbitrary diffusion model. They are new checkpoints with their own training recipes, published on Hugging Face as Fast_dLLM_v2_7B, Fast_dVLM_3B and Fast-dDrive. If your model is not one of those, none of these three tracks applies to you.
Getting Each Track Running: Commands From the README
For v1 the sequence is cd v1, pip install -r requirements.txt, then either the LLaDA chat command or an accelerate launch against dream/eval.py with the model_args string shown above. Note that the evaluation path uses accelerate rather than plain python, and that the model arguments are passed as a single comma-separated string, which is a common source of silent misconfiguration if you mistype a key. For v2 the sequence is cd v2, pip install -e ., then python app.py for the Gradio demo or bash eval_script.sh for evaluation. For Fast-dVLM it is cd fast_dvlm, pip install -r requirements.txt, then python run_chatbot.py with --model-name Efficient-Large-Model/Fast_dVLM_3B, --image and --prompt for one-shot inference, or no arguments for interactive mode. The top-level README's Fast-dDrive section is truncated in the supplied material, so the exact installation and run commands for that track cannot be confirmed here; read fast_ddrive/README.md directly. None of the tracks publish a memory or VRAM figure in the top-level file.
Where the Repository Is Thin
Several gaps are worth naming. The TODO list still shows vLLM support as pending, which means there is no documented path to serving these models through a standard high-throughput inference server, despite the Fast-dDrive announcement comparing against an SGLang autoregressive baseline. There are no tagged releases, so version pinning means pinning a commit hash rather than a release number. The requirements.txt files are per-directory and their contents are not shown, so dependency conflicts between the four tracks are unverified. The top-level README also does not state minimum GPU memory, supported CUDA versions, or expected runtime for any command. The speedup figures that do appear (up to 6.18x for Fast-dVLM, up to 12x for Fast-dDrive over an autoregressive baseline) come from the project's own announcements and papers, and the baselines differ between tracks, so they are not directly comparable to each other.
Fast-dLLM v1 Versus a Standard Diffusion Sampler
The closest comparison for v1 is not a different project but the default decoding loop in Dream or LLaDA itself. A conventional diffusion sampler runs a fixed number of denoising steps over the full sequence and produces tokens in a uniform schedule. Fast-dLLM v1 keeps that schedule but adds two things: a cache so that already-decided positions are not recomputed, and a confidence threshold so that positions the model is sure about are committed before the schedule ends. The difference in approach is that v1 treats decoding order as a policy question rather than a fixed property of the model. That is why it can be applied post hoc to an existing checkpoint. A conventional sampler cannot be sped up this way without changing the model, which is exactly what v2 does. So the real fork in the road is whether you can accept a slightly different decoding policy on an unchanged checkpoint (v1) or whether you need a checkpoint trained for block diffusion (v2 and beyond).
Licence and Maintenance Cost
The repository is Apache-2.0, which permits commercial use and modification subject to the usual conditions around notices and patent grant. That licence covers the code in this repository. It does not automatically cover the model weights hosted on Hugging Face, which are separate artifacts under Efficient-Large-Model, and the README does not state their licence terms. If you plan to ship a product, check the model card for each checkpoint independently of the repository licence. This is a description of what the files say, not legal advice. On maintenance: the last push recorded is 2026-05-30, and the news entries show a steady cadence of releases across four tracks. That cadence is also the cost. Each track has its own README, requirements and scripts, so upgrading means tracking four directories rather than one, and with no tagged releases you are following main. Budget for reading per-directory documentation on every update.
Editorial conclusion
Adopt v1 if you already run Dream or LLaDA checkpoints and want latency reduction without touching weights; the README's own example is a single accelerate launch command against Dream-v0-Base-7B. Do not adopt v2, Fast-dVLM or Fast-dDrive unless you can host the published Hugging Face checkpoints (Fast_dLLM_v2_7B, Fast_dVLM_3B, Fast-dDrive) and, for the latter two, the DeepSpeed and LMFlow training path under third_party/. Verify first that your GPU matches what the per-directory READMEs state, since the top-level file does not list memory requirements, and confirm whether vLLM support has landed, because the TODO list still marks it as pending.
Community notes