# SDAR runs autoregressive training and diffusion decoding in one model

> SDAR, Synergy of Diffusion and AutoRegression, is a large-scale diffusion language model published by JetAstra in 1.7B, 4B, 8B dense and 30B MoE variants, with weights on Hugging Face and inference code plus two third-party engines. The README states plainly that the project is still an early experimental state.

**JetAstra/SDAR** — SDAR (Synergy of Diffusion and AutoRegression), a large diffusion language model（1.7B, 4B, 8B, 30B）

- Repository: https://github.com/JetAstra/SDAR
- Stars: 532 · Forks: 43
- Language: Python
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/jetastra-sdar

## What SDAR merges and why that matters for decoding

SDAR stands for Synergy of Diffusion and AutoRegression, and the README describes it as a large-scale diffusion language model that unites the complementary strengths of autoregressive and discrete diffusion modeling. The claimed combination is the training efficiency of autoregressive methods with the highly parallel decoding ability of diffusion models, with performance described as competitive with state-of-the-art open-source AR models. The highlights are listed as low-cost AR-to-BlockDiffusion, inference reported as 2 to 4 times faster, and advanced performance on science reasoning benchmarks such as GPQA and ChemBench. The interesting engineering claim is the first one. Converting a pretrained autoregressive model into a block diffusion model, described in the highlights as AR-to-BlockDiffusion and described in the news as low-cost, is what makes the rest practical: you do not have to train a diffusion language model from scratch to get one. The project page is at jetastra.github.io/SDAR/, the technical report is on arXiv at 2510.06303, and the model weights are collected on Hugging Face under the JetLM organisation.

## Model zoo: dense sizes, one MoE, and the block size axis

The weights came out in stages, and the sequence is worth reading. On 2025-07-20 the project open-sourced the weights for the 1.7B, 4B and 8B dense models along with a 30B MoE model, published as SDAR-1.7B-Chat, SDAR-4B-Chat, SDAR-8B-Chat and SDAR-30B-A3B-Chat. On 2025-08-18 the SDAR-30B-A3B-Sci model was released separately, a science-focused variant of the MoE model. On 2025-09-09 the weights for models with various block sizes were published, so alongside the default model with block size 4 you can find models with block sizes of 8, 16, 32 and 64 on Hugging Face. Block size is the central knob of the method, since it is the number of tokens the model denoises or predicts as a unit, and the fact that the default is 4 while larger sizes exist separately means the choice of checkpoint is a real decision rather than a download detail. The repository itself is small: generate.py, training, evaluation, thirdparty, assets and a .gitmodules file, with no GitHub releases.

## Running the built-in inference script

The built-in path needs a recent transformers release, and the README gives the version as a requirement line on its own.

```text
transformers>=4.52.4
```

Then the bundled script generates from a model directory.

```bash
python generate.py \
  --model_dir=JetLM/SDAR-1.7B-Chat \
  --trust_remote_code
```

The trust_remote_code flag is required, and that is worth understanding rather than copying. It tells transformers to fetch and execute the modelling code that ships alongside the weights on Hugging Face, which is the normal mechanism for a model with a custom architecture but also means the code is not the audited library you install from PyPI. For a research checkpoint you have read the paper for, that is an acceptable trade. For anything else, it is the thing to look at first. The inference code itself was released on 2025-08-13, together with a built-in script and a third-party inference engine, JetEngine.

## JetEngine, the throughput path and its dependencies

JetEngine is described as a lightweight inference engine for the SDAR series built on nano-vllm, supporting both dense and MoE models and Tensor Parallel distributed inference. The README gives a benchmark of the 4B model with block size 4 at batch size 128: on an NVIDIA A800 it reached more than 1800 tokens per second, and on an NVIDIA H200 it achieved more than 3700 tokens per second using FlashAttention-2 with Triton kernels. The installation is a sequence of commands that pulls the repository, initialises submodules and installs the engine.

```bash
pip install flash-attn --no-build-isolation #Install fa2
git clone https://github.com/JetAstra/SDAR.git
cd SDAR
git submodule update --init --recursive
cd third_party/JetEngine
pip install .
```

The load and generate call shows the parameters that differ from autoregressive generation. The LLM constructor takes mask_token_id, described as optional and only needed for masked or diffusion models, plus block_length. The sampling parameters carry the diffusion-specific fields: a remasking_strategy set to low_confidence_dynamic, a block_length, a denoising_steps count, and a dynamic_threshold. The token count budget is expressed as max_tokens of 256 in the example. For anyone porting from an autoregressive setup, the fields that matter are denoising_steps, dynamic_threshold and the remasking strategy, because those control how the model revises a partially generated block rather than extending a prefix.

## LMDeploy and the fine-tuning framework

The second production path is LMDeploy, and the README describes it as an industrial-grade inference solution providing deployment infrastructure for production environments, open-sourced on 2025-10-10. Its usage example imports pipeline, PytorchEngineConfig, GenerationConfig and ChatTemplateConfig from lmdeploy, along with Timer and visualize_pipe_out from the pytorch tools, and drives a model path of JetLM/SDAR-8B-Chat with a list of chat-style prompts. The prompts in the example are reasoning-style, including a calculus minimisation question with a step-by-step instruction and a boxed final answer. For training, the README points to the training directory and its training/README.md for detailed instructions on fine-tuning on your own dataset. On 2025-10-29 the project open-sourced its downstream task fine-tuning framework, powered by LLaMA-Factory, described as a toolkit for adapting SDAR to specific needs. So there are three distinct paths to a working system: the bundled script for evaluation, JetEngine or LMDeploy for throughput, and the LLaMA-Factory based flow for adaptation.

## The preliminary experiments and how to read them

The README's own table of contents is honest about the state of the evidence, because the experiments section is titled Preliminary Experiments and split into two parts. Part I covers scaling the Qwen3 series with SDAR for general, non-reasoning tasks, and Part II covers applying SDAR to Qwen3-30B-MoE for reasoning benchmarks. That structure tells you the method is applied on top of an existing base rather than trained from scratch, which is consistent with the AR-to-BlockDiffusion claim and with the released checkpoint names carrying Qwen lineage. The reasoning claims are attached to science benchmarks, GPQA and ChemBench, and the model zoo includes a Sci variant, so the science specialisation is a deliberate branch of the work rather than an accident of the training data. Two cautions apply to anyone reading the results. The experiments are labelled preliminary by the project itself, and the SDAR-30B-A3B-Sci checkpoint is a separate release from the general 30B model, so a number quoted for one is not a number for the other.

## Where SDAR is the wrong choice

Three limits, and the first one is stated by the authors. The README says SDAR is still an early experimental state, that they are actively developing more systematically, and that they welcome collaborations in this direction. That is not a disclaimer about warranty, it is a statement that the method and the tooling are moving. For a team that needs a model with a stable API, a documented recipe and predictable quality, that is disqualifying on its own, and there is no GitHub releases list to give you a version history to pin against. The second limit is the trust_remote_code requirement, which as noted means the modelling code executes from the model repository rather than from your installed library. The third is the hardware and dependency weight. FlashAttention-2 has to be built for your platform, and the JetEngine path expects an A800 or H200 class accelerator to reach the throughput figures in the README, since the numbers quoted are for those two cards at batch size 128. On a single consumer GPU the same code path will produce the same outputs but nothing like the same speed, so the 2 to 4 times faster inference claim depends on what you are comparing against and on what hardware you run it on. The pragmatic alternative for most teams is a well-supported open autoregressive model with a mature serving stack, and for researchers who specifically need diffusion decoding the choice is between this repository and the growing set of other diffusion language model implementations, where SDAR's contribution is the block-diffusion conversion path and the released checkpoints rather than the framework itself.

## Conclusion

Adopt SDAR if you are doing diffusion language model research and want open weights at several sizes plus two production inference paths. Skip it if you need a stable, documented model for a product today, since the README says the project is still an early experimental state and the published results are preliminary. Before you evaluate it, read the model zoo for the block size you intend to use, since the default is 4 and larger block sizes were released separately, and check which base series each checkpoint was scaled from.

## FAQ

### What is SDAR?

SDAR is Synergy of Diffusion and AutoRegression, a large-scale diffusion language model from JetAstra that combines the training efficiency of autoregressive methods with the parallel decoding ability of diffusion models. The README describes performance as competitive with state-of-the-art open-source autoregressive models.

### Which SDAR model sizes are available?

The dense 1.7B, 4B and 8B chat models, a 30B MoE model, and a separate science variant of the MoE model, SDAR-30B-A3B-Sci. Weights for different block sizes were also published, with a default of 4 and additional sizes of 8, 16, 32 and 64 on Hugging Face.

### How do I run inference with SDAR?

Three options. The built-in script with `python generate.py --model_dir=JetLM/SDAR-1.7B-Chat --trust_remote_code` after installing transformers 4.52.4 or later, the JetEngine engine for batch inference and throughput, which needs flash-attn and the repository submodules, or LMDeploy for production deployment.

### Is SDAR ready for production use?

The README states that SDAR is still an early experimental state, that the team is actively developing more systematically, and that collaborations are welcome. The experimental results are also presented as preliminary, and the repository publishes no GitHub releases.

## Sources

- [Issues](https://github.com/JetAstra/SDAR/issues)
- [JetAstra/SDAR on GitHub](https://github.com/JetAstra/SDAR)
- [License: MIT](https://github.com/JetAstra/SDAR/blob/main/LICENSE)
- [README](https://github.com/JetAstra/SDAR/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/jetastra-sdar