# LLaDA2.0-Uni puts image understanding and generation in one model

> LLaDA2.0-Uni is a diffusion large language model from the AGI Research Center at Inclusion AI that handles visual question answering, image generation and image editing through a single mask-token-prediction interface. The trade is hardware: the documented setup is CUDA 12.4 with Flash Attention 2, and the checkpoints are large enough that the quantised FP8 versions exist for a reason.

**inclusionAI/LLaDA2.0-Uni** — LLaDA2.0-Uni: Understanding and Generation the World. 

- Repository: https://github.com/inclusionAI/LLaDA2.0-Uni
- Stars: 698 · Forks: 42
- Language: Python
- License: not declared
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/inclusionai-llada2-0-uni

## Mask token prediction is the interface, not a bolt-on

The problem this project attacks is that multimodal models usually split the job. A vision-language model reads images, a diffusion model draws them, and an editing model modifies them, and a team ends up maintaining three checkpoints and three inference paths. LLaDA2.0-Uni is presented as a unified dLLM-based Mixture-of-Experts model that integrates multimodal understanding and generation, and the phrase the README uses is Unifying Multimodal Understanding and Generation with Diffusion Large Language Model. The architectural summary names a unified dLLM-MoE backbone built on LLaDA 2.0 that unifies understanding and generation into a mask token prediction paradigm. The intended user is a researcher who wants one model to answer a question about an image and then draw the answer, and the technical report is on arXiv as 2604.20796 with the checkpoints published under the inclusionAI organisation on Hugging Face and ModelScope.

## A discrete semantic tokenizer sits between pixels and tokens

The mechanism that makes one model do both jobs is that vision goes through a tokenizer. The README describes a discrete semantic tokenizer that uses SigLIP-VQ to convert visual inputs into discrete semantic tokens, and says this significantly enhances multimodal understanding. The point is that once an image is a sequence of discrete tokens, it can live in the same sequence space as text, and the mask token prediction machinery that generates text can generate image tokens too. On the other side sits an efficient diffusion decoder, which pairs the discrete tokens with a specialized diffusion decoder for high-fidelity generation and enables rapid 8-step inference via distillation. The decoder lives in the decoder/ directory of the repository and the tokenizer in encoder/, with the scripts/ directory holding the runnable pieces. This is the same design pattern as an autoregressive image generator, with the diffusion decoder doing the rendering work at the end.

## Eight steps of generation, and a distilled decoder that does eight instead of fifty

The inference cost is visible in the code examples, and the numbers are worth understanding before you plan hardware around them. The generation call looks like this:```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from decoder import decode_vq_tokens

model_path = "inclusionAI/LLaDA2.0-Uni"
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_path, device_map="cuda", torch_dtype="bfloat16", trust_remote_code=True
).eval()
model.tokenizer = tokenizer

# Generate image tokens
result = model.generate_image(
    "A modern Scandinavian kitchen with white cabinetry, marble countertops, and a single orchid on the island. A Nordic woman with sleek blonde ponytail, wearing an oversized sweater and dainty silver necklaces, stirs a matcha bowl with a bamboo whisk, eyes sparkling with quiet joy. Shot with 50mm, f/2.5, diffused window light, cool white balance, low saturation, clean skin retouch. Mood: serene, wholesome, hygge.",
    image_h=1024, image_w=1024,
``` The image generation call passes steps of 8 and a cfg_scale of 2.0, and the decode step that turns tokens into a picture defaults to a 50-step ODE. The README then describes a faster decoding option, the decoder-turbo variant which is a distilled decoder, giving around ten times faster image decoding with 8 steps instead of 50 and minimal quality loss, and the second example passes decode_mode set to decoder-turbo along with num_steps of 8. The tradeoff is explicit in the documentation rather than hidden: you can run the slower, higher-step decoder or the distilled one, and the code makes the choice a parameter. The image size is set through image_h and image_w, and the examples use 1024 by 1024.

## Thinking mode spends tokens before it draws

The second text-to-image example adds a mode called thinking, which is a genuinely different mode of operation rather than a prompt modifier. The call passes mode set to thinking along with thinking_steps of 32 and thinking_gen_length of 4096, and the result dictionary then carries a thinking field that the example prints before decoding. The idea is that the model reasons about the image before generating it, spending a bounded number of reasoning steps and a bounded generation length, and the code gives you the trace if you want to inspect it. The core capability list describes this as interleaved generation and reasoning, enabled by the unified discrete representations, which handles interleaved generation and interleaved reasoning. The distinction that matters in practice is between the plain mode, where the model goes straight to image tokens, and the thinking mode, where the extra tokens cost you latency before you see anything.

## Image editing takes one or many reference images

The third capability is editing, and the README describes it as flexible image editing that supports single or multi-reference editing, enabling precise modifications while preserving the original details. The image understanding path is what makes that possible, and the repository layout shows how it is split, with encoder/image_tokenizer.py providing the ImageTokenizer class and encoder.smart_img_process providing a smart_resize_images helper, and the understanding example in the README imports both alongside the usual transformers AutoModelForCausalLM and AutoTokenizer pair. The whole model is loaded the same way in every example, with trust_remote_code set to true and the tokenizer attached to the model object manually, which is worth noting because it means the checkpoint carries its own Python implementation rather than a stock transformers class.

## The install expects a CUDA box with Flash Attention 2

Installation is four steps and none of them is optional on the hardware side. You clone the repository, create a conda environment with Python 3.10, install PyTorch and torchvision from the CUDA 12.4 wheel index, install flash-attn with build isolation disabled because the README calls it required for efficient inference, and then install the remaining dependencies from requirements.txt:```bash
git clone https://github.com/inclusionAI/LLaDA2-Uni && cd LLaDA2-Uni
conda create -n llada2_uni python=3.10 -y
conda activate llada2_uni
``` The PyTorch step and the Flash Attention step are each one command:```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
``` and ```bash
pip install flash-attn --no-build-isolation
``` That file pins a floor for each dependency, with torch at 2.4.0, torchvision at 0.19.0, transformers at 4.45.0, safetensors, diffusers at 0.30.0, flash-attn at 2.5.0, Pillow, numpy, tqdm, accelerate and torchdiffeq, the last one being the differential equation solver the diffusion decoder needs. The environment name in the command is llada2_uni and the clone is of LLaDA2-Uni, which is a slightly different spelling from the repository name in the URL.

## Checkpoints come in four flavours across two model hubs

The weights are published in two variants and on two platforms, and the FP8 one is the reason the project is usable on smaller hardware. The README links checkpoints for the full model and for the FP8 quantised version on both Hugging Face, under inclusionAI/LLaDA2.0-Uni and inclusionAI/LLaDA2.0-Uni-FP8, and mirrors both on ModelScope. The news section dates the FP8 release to 2026-05-06, and the initial release with text-to-image, image understanding and image editing inference code to 2026-04-23. The quantised versions matter because a diffusion model with an MoE backbone is not a small download, and the README does not give a parameter count or a file size, so you have to check the checkpoint card yourself before committing a machine to it.

## Diffusers, ComfyUI and SGLang all landed within a month

The integration story has been built out quickly, and the TODO list in the README shows all four items ticked, with a quantised model, Diffusers support, ComfyUI support and SGLang support. The news entries give the dates, ComfyUI and Diffusers on 2026-05-12 and SGLang Omni support on 2026-05-29, with the SGLang installation details in a cookbook page in the sgl-project repository. The apps/ directory at the root of this repository holds the ComfyUI and Diffusers material. What is absent from the list is a production serving story: there is no release, and the serving path that is documented is SGLang Omni, which is the route to consider if you want to put this behind an API rather than run it in a notebook.

## Conclusion

LLaDA2.0-Uni suits a research team that wants a single model covering visual question answering, text-to-image and image editing rather than three separate models, and that has NVIDIA hardware with the CUDA toolchain to run it. It does not suit a team without a GPU, since the documented install targets CUDA 12.4 and requires Flash Attention 2, and it does not suit a production deployment that needs a stable, versioned API, because the project publishes no GitHub releases and the checkpoint is loaded with trust_remote_code enabled. Before adopting, read the technical report on arXiv at 2604.20796, check the checkpoint size and the FP8 variant against your memory budget, and note that the last push to the repository was on 2026-09-23.

## FAQ

### What is LLaDA2.0-Uni?

It is a unified diffusion large language model, a Mixture-of-Experts model built on LLaDA 2.0, that handles multimodal understanding and generation in one model through a mask token prediction paradigm. The checkpoint and technical report are published by the AGI Research Center at Inclusion AI, with the report on arXiv as 2604.20796.

### How do I install LLaDA2.0-Uni?

Clone the repository, create a conda environment with Python 3.10, install PyTorch and torchvision from the CUDA 12.4 wheel index, install flash-attn with --no-build-isolation because it is required for efficient inference, then install the rest from requirements.txt. The requirements pin torch at 2.4.0 or newer and transformers at 4.45.0 or newer.

### What hardware does LLaDA2.0-Uni need?

The documented setup is CUDA 12.4 with Flash Attention 2, and the inference examples load the model with device_map set to cuda and torch_dtype set to bfloat16. The README does not state a minimum GPU memory, so you have to check the checkpoint size yourself, and an FP8 quantised version of the weights is published alongside the full one.

### How do I generate an image with LLaDA2.0-Uni?

Call model.generate_image with a prompt, image_h and image_w, steps of 8 and cfg_scale of 2.0, then pass the resulting token ids, height and width to decode_vq_tokens to get a PIL image. The decode step defaults to a 50-step ODE, and passing decode_mode set to decoder-turbo with num_steps of 8 gives roughly ten times faster decoding with minimal quality loss.

### Can LLaDA2.0-Uni edit an existing image?

Yes, the README describes flexible image editing supporting single or multi-reference editing, enabling precise modifications while preserving the original details. Understanding an image goes through the discrete semantic tokenizer in the encoder/ directory, which uses SigLIP-VQ to convert visual input into discrete tokens.

## Sources

- [inclusionAI/LLaDA2.0-Uni on GitHub](https://github.com/inclusionAI/LLaDA2.0-Uni)
- [Issues](https://github.com/inclusionAI/LLaDA2.0-Uni/issues)
- [README](https://github.com/inclusionAI/LLaDA2.0-Uni/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/inclusionai-llada2-0-uni