Lumina-DiMOO: a discrete diffusion model that generates and understands images in one pass

One discrete diffusion model behind generation and understanding

The design choice that separates Lumina-DiMOO from an autoregressive multimodal model is stated first in the project introduction: it uses fully discrete diffusion modeling to handle inputs and outputs across various modalities. There is no AR decoder producing image tokens one at a time. Instead, the whole sequence, image or text, is treated as discrete tokens that get refined through a diffusion process.

That choice is what lets one checkpoint serve both directions. The listed capabilities span text-to-image generation at arbitrary and high resolution, image-to-image generation for editing, subject-driven generation and inpainting, and image understanding. A conventional text-to-image model answers none of the second set. A single diffusion model that also reads images removes the need to chain a separate vision encoder and a separate captioner around your pipeline.

The claimed benefits are efficiency and quality. The project states that compared to previous AR or hybrid AR-diffusion paradigms it shows better sampling efficiency, and that it reaches state of the art results on several benchmarks against other open source unified models. The benchmark panels published in the repository cover GenEval, DPG, OneIG-EN, TIIF, an image-to-image set and an understanding set, with the numbers themselves shown as images rather than text, and the project also reports a first place generation ranking on the UniGenBench leaderboard for open source unified models. Those are the project's own claims, and the underlying tables are in the README as pictures, so you would need to read them from the assets to check any specific number.

Generation samples 64 steps, understanding samples 128 in blocks of 256

The sampling configuration differs sharply between the two tasks, and this is the detail most likely to surprise someone who assumes one model and one setting.

For image generation, the project samples 64 steps. For image understanding, it sets the block length to 256 and the number of sampling steps to 128. The reason given is structural: text generation is performed in a block-wise manner, unlike image generation which uses a single global decoding step. The speed of understanding output is therefore influenced by both the number of blocks and the number of steps, and the speedup measured for image generation does not carry over to understanding in the same proportion.

That asymmetry has a practical consequence. A pipeline tuned around 64 global denoising steps cannot be reused unchanged for the text side, and the two paths have different latency characteristics. If you are building anything with a latency budget, treat the understanding path as the expensive one and size it separately.

The repository layout supports this reading. Alongside the model code there is a `pre_tokenizer/` directory, a `generators/` directory, an `inference/` directory, `configs/` and a `config.py`, which is the shape of a project that treats preprocessing, generation and configuration as separate concerns rather than a single script.

The Max Logit cache trades a little memory for a shorter wait

Sampling acceleration in this project comes from a cache, and the project describes it as a bespoke caching method that speeds up sampling by 2x. The implementation is open sourced as the Max Logit-based Cache, and it is listed in the project's own open-source plan as completed.

The mechanism is named in the release note but not explained in prose, so the practical guidance comes from the project's own measurement notes. In the notes, a run at 1536x768 resolution on an A800 GPU is recorded with a latency of 58.2 s and peak GPU memory of 38.9 GiB without the cache, and 32.2 s with 45.9 GiB peak GPU memory. Read that as what one configuration cost rather than as a general result, since the notes sit inside a collapsed details block in the README, but the shape of the trade-off is clear: the cache cuts wall clock time and raises peak memory.

That is the wrong way round if you are memory constrained and the right way round if you are waiting on a user. On a single accelerator, trading roughly seven additional gibibytes of peak memory for a large cut in latency is a decision you should make explicitly per workload, not inherit from a default.

Installing with conda and a pinned requirements file

The quick start is short, and it is worth following exactly rather than improvising, because the dependency pins are old enough that a modern resolution will likely conflict.

First clone the repository and create a Python 3.10 conda environment:

git clone https://github.com/Alpha-VLLM/Lumina-DiMOO.git && cd Lumina-DiMOO
conda create -n lumina_dimoo python=3.10 -y
conda activate lumina_dimoo

Then install the dependencies:

pip install -r requirements.txt

The requirements file pins the stack tightly: torch 2.3.1, torchvision 0.18.1, torchaudio 2.3.1, transformers 4.46.2, diffusers 0.34.0, gradio 4.19.0 and torchao 0.11.0, alongside accelerate, bitsandbytes, fairscale, einops, h5py, pytorch_lightning, sentencepiece and Ninja.

Two observations from that list. The torch pin is what makes a real installation a decision rather than a formality: you need a CUDA build of that exact torch version, and the environment cannot simply be pointed at a newer PyPI index to get a recent release. The presence of diffusers and gradio also tells you the intended entry point is a library integration and an interactive demo rather than a command line tool with a documented serving interface.

The model weights themselves are hosted on HuggingFace, and the project site carries the demo. Nothing in the README describes an inference server, an API endpoint, or a container image, so plan on running the code yourself in that conda environment.

Fine-tuning starts from pre-extracted discrete codes

The training path is documented in two steps rather than one, and the first step is where most of the work sits.

Before training, the discrete codes of the training images have to be pre-extracted. The final format after processing is described by two sample files, `assets/mmu_sample.json` and `assets/t2i_sample.json`, and the README sketches the extraction step and leaves the exact invocation to the training scripts. Expect to read the training scripts in `train/` to recover the intended invocation rather than to copy a complete line from the documentation.

The repository contents confirm the rest of the pipeline. There is a `train/` directory, a `data/` directory, `utils/`, a `ht_grpo/` directory, and an `xllmx/` directory. The `ht_grpo/` folder ties to the HT-GRPO paper the project announced on 2026-05-16, and the project has also published a test-time scaling algorithm for diffusion MLLMs under the name dMLLM-TTS, dated 2025-12-23.

That last pair of releases is worth flagging against the project's own checklist. The open-source plan still shows Test-Time Scaling unchecked while the news list above it announces a dMLLM-TTS paper and the tree carries an `ht_grpo/` directory. The checklist and the news feed disagree, so treat the plan list as stale documentation and judge availability from the directories that are actually present.

Diffusers and ComfyUI support, plus a separate evaluation codebase

The project's reach outside its own repository is wider than a typical research release. Support for Diffusers was added on 2025-10-21, with a parallel integration for ComfyUI, and a dedicated Lumina_DiMOO README was published in a Diffusers fork for that path. Evaluation code was released separately on 2025-11-27 using VLMEvalKit, and the repository ships that evaluation setup in its own `VLMEvalKit/` directory.

The Diffusers and ComfyUI integrations matter more than they might appear. If you already run a Diffusers based image pipeline, dropping this model in means reusing your existing sampling, scheduler and offload code rather than writing a bespoke inference script. The same goes for ComfyUI users, who can compose the model into a node graph instead of maintaining a Python environment. The pin on diffusers 0.34.0 in the requirements file is the constraint to watch, since the integration is tied to a particular Diffusers generation.

The training code was released on 2025-10-06, the inpainting and extrapolation code on 2025-09-12, and the technical report on 2025-09-25, with the report also bundled in the repository as Technical-Report.pdf. The open-source checklist is nearly complete, and the single unchecked box, Test-Time Scaling, is the one area where the paper trail runs ahead of the code, if the `ht_grpo/` directory does not cover it.

No releases, one branch, and a last push in May

The maintenance picture is the weakest part of this project, and any evaluation should start here. The repository has no GitHub releases at all. There are no tags to pin, no changelog entries to diff, and no versioned artefact to point a dependency at. The default branch is `main` and the last push was on 2026-05-19.

For a codebase where the model weights live on a separate hosting platform, that creates an awkward split. The code you check out and the checkpoint you download are versioned by nothing shared. The HuggingFace model card is the closest thing to a version reference for the weights, and the repository README does not document a matching commit.

The licence is Apache-2.0 for the code in this repository, which is permissive and includes an explicit patent grant. Two caveats are worth stating without giving legal advice. First, the licence covering the weight files on HuggingFace is a separate question from the licence of this source tree, and you should read the model card for that. Second, the institutions behind the work span several universities and laboratories, so if you need a different grant for a commercial product, the project points at its own pages and the project page is where to look.

One last practical note for anyone planning an evaluation. The README's benchmark results are published as image files under `assets/`, and the sampling speed comparison is likewise an image. If you need a numeric table rather than a picture, the VLMEvalKit evaluation code in the repository is the route to producing one, and running it is your own task rather than something the project ships as a result.

Editorial conclusion

Lumina-DiMOO is worth a look if you need one checkpoint that both generates and interprets images, and if you are comfortable working at the level of discrete codes, a fixed torch stack and a weights file from HuggingFace. It is a poor fit if you need a stable versioned release, a documented serving endpoint, or a model that runs on a single consumer GPU, since the repository publishes no GitHub releases and the sampling settings alone assume a research class accelerator. Before you commit, verify that the checkpoint on HuggingFace carries a licence you can use, that the torch 2.3.1 and transformers 4.46.2 pins work with your CUDA build, and that the Max Logit cache does not change the images it produces on your own prompts.

Frequently asked questions

Official sources

1. Alpha-VLLM/Lumina-DiMOO on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README