Streaming-dLLM: Suffix Pruning and Dynamic Decoding for Diffusion Language Models

The problem: diffusion decoding wastes work on tokens that are already settled

Autoregressive models emit one token per forward pass. Diffusion language models denoise a whole sequence over many steps, which buys parallelism but pays for it in repeated computation. Prior accelerators such as dKV-Cache, Prefix-Cache and Fast-dLLM attack that repetition from different angles, mostly by caching key-value state or by reusing prefix computation. Streaming-dLLM takes a different route: it assumes that the suffix of a partially denoised sequence carries little information for the block currently being decoded, so it prunes that suffix and keeps only a small neighbourhood next to the active block. The target user is someone who has a diffusion LM checkpoint on disk and a benchmark script they want to run faster, not someone building a new model. The README frames the whole thing as a training-free acceleration framework, which is the operative constraint: nothing here requires fine-tuning or a new checkpoint.

Approximated suffix pruning: a sliding window plus the trailing position

The mechanism is described in the figure caption rather than in prose. For each block, the nearest neighbouring region following the current block is retained using a sliding window, and that window is concatenated with the trailing position to form an approximate suffix region. Everything outside that region is dropped from the computation for the current block. The word approximate matters. This is not an exact attention mask with a proof of equivalence; it is a reconstruction of the suffix from a local window plus the final position, and the accuracy tables are the evidence that the approximation holds on the benchmarks tested. The repository does not document the window size as a named config key, and it does not state how the window is chosen per model or per generation length. That is a real gap: a reader who wants to reproduce the 13.3x GSM8K number cannot see from the README which window produced it.

Dynamic decoding and the Open Pangu special case

The second half of the method is a temporal decoding module, and the README is explicit that its role changes depending on the backbone. For block-causal diffusion models the spatial redundancy reduction degenerates into what the README calls a block-topology-aware special case, so the authors apply the temporal decoding module to Open Pangu instead. That is an honest admission that the headline technique does not transfer unchanged to every diffusion architecture. The Open Pangu adapter lives under OpenPangu, runs on Ascend 910B, and reports smaller gains than the Dream and LLaDA tables: 1.4x to 1.6x across GSM8K, MATH, HumanEval, MMLU-Pro, BBH and CMMLU, with TPS figures rather than the speedup multiples used elsewhere. Note also that CMMLU accuracy drops from 75.46 to 74.72 in the Open Pangu table, so the gain is not free on every benchmark.

Getting it running: conda, requirements.txt and per-model shell scripts

Installation is a clone plus a conda environment pinned to Python 3.10.19, then pip install -r requirements.txt. The README's project structure block expects sibling directories named Dream, LLaDA-1.5, OpenPangu and Other, which means the eval scripts assume a particular working directory layout rather than taking paths as arguments. For Dream you cd into Dream and run bash eval_dream.sh. For LLaDA-1.5 you cd into LLaDA-1.5 and run bash eval_llada.sh. For plain LLaDA the README says to replace the llada-1.5 path with llada first and then run the same script, which tells you the path is hardcoded inside eval_llada.sh rather than exposed as a flag. Open Pangu is the only entry point with a documented command-line interface: python generate_streaming.py --model-path /path/to/openPangu-7B-Diffusion-Base --prompt "introduce the china". Weights come from four separate Hugging Face or GitCode links, and no checksum or expected file size is given for any of them.

What the repository does not give you

There is no licence file in the material supplied, and the licence is listed as unknown. That is the first thing to resolve, because a training-free inference wrapper still links against model code and redistributes derived scripts. There are no releases, so there is no version to pin and no changelog to read when an eval script breaks. There is no test suite, no CI configuration described, and no API surface beyond the shell scripts and the one Python entry point. The README's Usage section documents no config keys at all, which means tuning the pruning window or the decoding schedule requires reading the scripts. Anyone expecting a pip-installable package with a documented Config object will be disappointed; this is a research repository shaped around reproducing a paper's tables.

The alternative: Fast-dLLM, and where the approaches diverge

The README benchmarks against Fast-dLLM directly, and the comparison is the clearest way to see the design difference. On Dream at generation length 512 for GSM8K-CoT, Fast-dLLM reports 41.7 tokens per second against 94.1 for Streaming-dLLM, and on LLaDA-1.5 MBPP at 512 the gap widens to 16.5 versus 61.4. Both are training-free accelerators, so the difference is not about retraining. Fast-dLLM's numbers are consistently lower but its accuracy profile is more even in places: on Dream HumanEval at 256 it reaches 54.3 accuracy where Streaming-dLLM also reports 54.3, and on LLaDA-1.5 HumanEval at 256 Streaming-dLLM drops to 39.0 while the baseline LLaDA-1.5 reports 43.9. So the trade is throughput for a small accuracy cost on some instruction-style code benchmarks, and a reader who cares more about HumanEval accuracy than tokens per second should read those two rows carefully before switching.

Maintenance cost and what to verify before adopting

The last push to the repository is dated 2026-06-24 and it is not archived, so it is active rather than frozen. There are no releases, which means upgrades happen by pulling main and re-reading the scripts; nothing in the material suggests a stable tag to fall back to. Licence terms are undeclared, so redistribution inside a commercial product is unresolved until you check. The practical verification steps are concrete: confirm that the four weight downloads complete and that the directory names match what eval_dream.sh and eval_llada.sh expect, run one benchmark at generation length 256 before trusting the 512 numbers, and check whether the pruning window is exposed anywhere in the scripts. If it is not, the speedup you observe is whatever the authors hardcoded, and you cannot trade accuracy against throughput without editing the code.

Editorial conclusion

Adopt Streaming-dLLM if you already serve Dream, LLaDA, LLaDA-1.5 or Open Pangu and want to try a training-free decoding change before committing to a retrained or distilled model. Do not adopt it if you need a supported library with a declared licence, a packaged API, or a diffusion backbone outside the four named here. Before running anything, resolve the licence question, because the repository carries no licence file, and check that eval_dream.sh and eval_llada.sh point at the weight directories you actually downloaded.

Official sources

1. Issues
2. README
3. xiaoshideta/Streaming-dLLM on GitHub