# Streaming-dLLM prunes a suffix approximation and reuses it, per model directory

> A training-free acceleration framework for diffusion language models, with one directory per backbone and a shell script per evaluation. Speedups in the tables run from 3.7x to 68.2x over the vanilla backbone, and the Open Pangu extension applies only the temporal decoding half because the spatial half degenerates for block-causal models.

**xiaoshideta/Streaming-dLLM** — Diffusion Language Model

- Repository: https://github.com/xiaoshideta/Streaming-dLLM
- Stars: 446 · Forks: 2
- Language: Python
- License: not declared
- Published: 2026-09-11 · Updated: 2026-09-11 · Language: en
- Canonical page: https://hysenlabs.com/projects/xiaoshideta-streaming-dllm

## Suffix pruning keeps a sliding window of the following region

The mechanism is described as approximated suffix pruning. For each block, the nearest neighbouring region that follows the current block is retained using a sliding window, shown as a red dashed box in the figure, and that region is concatenated with the trailing position to form an approximate suffix.

So the whole idea is that a diffusion language model re-reads a suffix at every step, and most of that suffix is context you already have. Keeping a bounded approximation of it is what removes the repeated cost.

The paired component is dynamic decoding, described in the title alongside suffix pruning. Together they are what the framework applies to Dream, LLaDA and LLaDA-1.5, and the results are reported as accuracy and throughput rather than a single number.

## Four directories, and Other is a placeholder

The project structure is the model list:

```shell
|-- <Dream>
|-- <LLaDA-1.5>
|-- <OpenPangu>
|-- <Other>
```

Three are real backbones with scripts in them. The fourth is a literal angle-bracketed placeholder meaning a model you add yourself, which tells you the intent is extensibility without shipping the extension.

There is no package to install and no import path. The top level is Dream/, LLaDA-1.5/, OpenPangu/, README.md, requirements.txt, visual_attention.py and fig/, plus the paper PDF. So the framework is a set of patches applied to each model's own inference code rather than a wrapper you configure once.

## The speedup range is 3.7x to 13.3x, and it depends on the length

The stated headline is 3.7x to 13.3x speedup across all benchmarks over the vanilla backbone, with a further 1.5x to 2.3x over the state-of-the-art acceleration method at a generation length of 512, with accuracy described as comparable or slightly better.

The per-benchmark table breaks that down. On Dream, HumanEval at 256 generation length moves from 49.4 accuracy and 20.4 tokens per second to 54.3 and 74.7, which is the 3.7x. At 512 length, GSM8K-CoT goes from 74.2 and 7.1 tokens per second to 74.7 and 94.1, the 13.3x.

The pattern is that longer generations gain more, because the suffix work compounds per step. Note also that the accuracy winner is not always the accelerated method: on HumanEval at 512, Prefix-Cache reaches 54.9 against 54.6 here.

## LLaDA-1.5 shows the largest ratios and the weakest accuracy case

The LLaDA-1.5 table is where the numbers get extreme. MBPP at 512 generation length goes from 38.2 accuracy and 0.9 tokens per second to 38.4 and 61.4, a 68.2x ratio. GSM8K at 512 goes from 81.0 and 2.5 to 81.2 and 69.8, which is 28.0x.

The reason the ratios are so much larger is in the baseline column. Vanilla LLaDA-1.5 runs at 0.9 and 2.5 tokens per second on those rows, so almost any caching strategy looks dramatic against it. Dream's baseline is an order of magnitude faster to begin with.

The accuracy story is also weaker here. On HumanEval, vanilla LLaDA-1.5 leads at 43.9 while the accelerated result is 39.0, and at 512 the figures are 40.5 against 40.2. Those are not gains.

## Open Pangu gets only the temporal half of the method

The block-causal extension is an explicit special case, and the reason is stated plainly: for block-causal diffusion models, the spatial redundancy reduction in Streaming-dLLM degenerates into a block-topology-aware special case. So the temporal decoding module is applied to Open Pangu instead.

That is a narrower claim than the rest of the framework, and the results match it. Reported throughput gains run 1.4x to 1.6x rather than the tens seen elsewhere, because only one of the two mechanisms applies.

Accuracy moves in both directions across the six benchmarks. GSM8K goes from 69.29 to 75.82 and MATH from 41.14 to 41.46, while CMMLU drops from 75.46 to 74.72, and MMLU-Pro is identical at 51.65.

## Weights come from four different places, one of them not on Hugging Face

Model weights are downloaded separately, and the run command takes a local path:

```shell
cd OpenPangu/inference
python generate_streaming.py --model-path /path/to/openPangu-7B-Diffusion-Base --prompt "introduce the china"
```

Dream comes from a Dream-org repository, LLaDA from GSAI-ML, LLaDA-1.5 from GSAI-ML, and Open Pangu from a gitcode address rather than Hugging Face.

For the other three backbones the entry point is a shell script rather than a Python call: cd Dream then bash eval_dream.sh, or cd LLaDA-1.5 then bash eval_llada.sh. Running plain LLaDA means editing the script to replace the llada-1.5 path with llada first.

## The environment is a full CUDA lock, pinned with equals signs

Setup is a conda environment and a requirements file:

```shell
$ conda create -n stream-dllm python=3.10.19
$ pip install -r requirements.txt
$ conda activate stream-dllm
```

Python 3.10.19 exactly, not a floor. And the requirements file is a complete frozen environment rather than a minimal list: every entry uses == rather than >=, and it runs past a hundred packages including the CUDA runtime set, nvidia-cublas-cu12, nvidia-cudnn-cu12, nvidia-nccl-cu12 and the rest, alongside torch's peers, gradio, fastapi, lm-eval, transformers' dataset library and evaluate.

That is a lockfile, not a dependency list. It describes one machine that was captured, which makes reproducing the published numbers easier and diverging from them harder. There is no release, no version tag and no licence statement in the top-level entries.

## Conclusion

Use it if you are already running a diffusion language model such as Dream or LLaDA and want throughput without retraining, since the method needs no weight updates. Read the tables rather than the headline before you rely on the speedup, because the ratios differ by an order of magnitude across backbones and generation lengths, and the accuracy column sometimes favours another method. Check your hardware against the requirements file, which is a full CUDA and PyTorch lock rather than a minimal list, and plan on picking one model directory rather than treating the repository as a library.

## FAQ

### What does Streaming-dLLM actually accelerate?

Diffusion language models, training-free. It is documented as supporting inference for Dream, LLaDA and LLaDA-1.5, with a lightweight extension for block-causal diffusion models using Open Pangu on Ascend 910B as its example.

### How does suffix pruning work in Streaming-dLLM?

For each block, the nearest neighbouring region following the current block is retained through a sliding window and concatenated with the trailing position to form an approximate suffix region, so the model stops re-reading the full suffix on every step.

### How much speedup does Streaming-dLLM report?

3.7x to 13.3x over the vanilla backbone across all benchmarks, and 1.5x to 2.3x additional over the state-of-the-art acceleration method at a generation length of 512. On LLaDA-1.5 at 512 length, MBPP reports a 68.2x ratio against a 0.9 tokens per second baseline.

### How do I run the Open Pangu extension of Streaming-dLLM?

Download the weights, then run generate_streaming.py with a --model-path argument and a --prompt from inside the OpenPangu/inference directory. Only the temporal decoding module is applied, since spatial redundancy reduction degenerates for block-causal models, so the reported gains are 1.4x to 1.6x.

### What environment does Streaming-dLLM need?

A conda environment on Python 3.10.19 with the pinned requirements file installed. That file is a full lock of over a hundred packages using exact versions, including the CUDA runtime set, so it describes one captured machine rather than a minimal dependency set.

## Sources

- [Issues](https://github.com/xiaoshideta/Streaming-dLLM/issues)
- [README](https://github.com/xiaoshideta/Streaming-dLLM/blob/main/README.md)
- [xiaoshideta/Streaming-dLLM on GitHub](https://github.com/xiaoshideta/Streaming-dLLM)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/xiaoshideta-streaming-dllm