LucidFlux: restoring images without a caption to condition on

The conditioner is the contribution

LucidFlux is a caption-free universal image restoration framework, and the one-sentence description of the mechanism is the most useful thing in the README.

It leverages a lightweight dual-branch conditioner and adaptive modulation to guide a large diffusion transformer, Flux.1, with minimal overhead, achieving robust and high-fidelity restoration without relying on text prompts or MLLM captions.

The last clause is the idea. Image restoration models that use a diffusion prior have usually been conditioned on a text prompt describing the clean image, which is a strange requirement for a restoration job, because the clean image is exactly the thing you do not have. If you are restoring a photograph because it is degraded, nobody has written a caption for the high quality version. A method that also leaned on a multimodal captioning model would compound the problem. LucidFlux drops that requirement by deriving its conditioning from the image itself, through a dual-branch conditioner whose output is fed into the transformer as an adaptive modulation.

The rest of the framing is that it is universal, meaning the same framework is not tied to a particular degradation, and that the overhead of the conditioner is small relative to the diffusion transformer it guides, which matters when the transformer is the expensive part.

The project was accepted by ICLR 2026, the paper is on arXiv at 2509.22414, there is a project website, the weights are on Hugging Face under the `W2GenAI` organisation, and the model is available through the Fal AI Playground and API. The authors are Song Fei and Tian Ye with equal contribution, Lujia Wang, and Lei Zhu as corresponding author, from the Hong Kong University of Science and Technology in Guangzhou and at HKUST.

The deployment story is a series of memory reductions

The news section is the most useful part of the README for somebody who wants to run this, because it is a record of the model getting cheaper to run.

In October 2025 the model added offload and precomputed prompt embeddings, which eliminates the need to load T5 or CLIP during inference. The README says these improvements reduce memory usage significantly and that inference can now run with as little as 28 GB VRAM, which it frames as greatly enhancing deployment efficiency.

A day earlier, on 2025-10-05, the model had been officially added to the Fal AI Playground, with the online demo and the Fal API available directly.

And in the same month, a community contributor, credited by name, developed the ComfyUI integration, which enables LucidFlux to run with as little as 8 GB to 12 GB of memory through ComfyUI.

So the trajectory is 28 GB down to 8 to 12 GB, and the way it got there is worth noticing. The first step was removing work from the inference path by precomputing the prompt embeddings, which is a change to what the model loads rather than to what it computes. The second was a third-party integration that made the model usable inside a tool many people already run. Neither step is a change to the architecture, and both are what turns a paper into something a person can run on a machine they own.

The practical advice that follows is that a buyer of this model should decide between the API and a local run on the basis of those numbers, and that the ComfyUI route is the one to check first, because it is the cheapest by an order of magnitude and it is maintained outside the lab.

The release came in pieces, and the data came with it

The sequence of releases in 2026 shows a project moving from a paper towards a reproducible artefact, one piece at a time.

On 2026-02-06 the work was accepted by ICLR. On 2026-03-10 the metadata for the clean images used in LucidFlux was released, together with the filtering pipeline, and the pipeline path is named in the README as `tools/filtering_pipeline.py`. On 2026-03-13 the model integrated the VAE from the group's own UltraFlux project to enable 2K image restoration. On 2026-03-19 the training code was released. And on 2026-05-26 the model integrated PiD to enable 4K image restoration.

The order matters. Publishing the metadata and the filtering pipeline before the training code is the order of a reproducible release, because a training set is only usable if you know how the clean images were chosen. The filtering pipeline is the piece that decides which images count as clean, and shipping it alongside the metadata is what lets somebody rebuild the dataset.

The two resolution integrations are the other story, and they are integrations with siblings rather than new work in this repository. UltraFlux is described as data-model co-design for high-quality native 4K text-to-image generation across diverse aspect ratios, and it is the same group's CVPR 2026 project, so borrowing its VAE is a division of labour between projects that share authors. PiD comes from a different group and enables the 4K path.

The third sibling project, LucidNFT, applies LR-anchored multi-reward preference optimization to generative real-world super-resolution. So the group is working on the generator, the data, the resolution and the preference optimisation, which is a fairly complete research programme around one problem.

The requirements file is three dependency sets

The `requirements.txt` in the repository is short enough to read and is organised in three labelled blocks, which is more informative than a single flat list.

The first block is the inference dependencies, and it is the one that matters to somebody running the model. It pins `accelerate==0.30.1`, `diffusers==0.32.2`, `transformers==4.43.3`, `huggingface_hub==0.24.5`, `safetensors==0.6.2`, `timm==1.0.22`, `opencv_python==4.11.0.86`, `Pillow==10.4.0`, `numpy==1.26.4`, and the usual set of `matplotlib`, `tqdm`, `einops` and `fire`. Two entries stand out: `liger_kernel==0.6.1`, which is a set of fused kernels for transformer models and is the kind of dependency that determines whether a memory budget works, and `cv_bridge`, which is a bridge to OpenCV.

The second block is the training dependencies, and it is three lines: `deepspeed==0.18.8`, `datasets==4.0.0` and `omegaconf==2.3.0`. Deepspeed is the distributed training engine, and the fact that the training path uses it rather than plain PyTorch distributed says something about the scale of the training runs, and about the fact that this is a multi-node job.

The third block is the PiD decoder dependencies, with `hydra-core` for configuration and a long list of tooling packages: `loguru`, `termcolor`, `fvcore`, `iopath`, `pynvml`, `wandb`, `pandas`, `imageio`, `boto3` and `botocore`.

The inference and training blocks pin exact versions, down to the patch releases, but several entries in the PiD block are left unpinned. And the presence of `boto3` and `botocore` in the decoder set says the 4K path reads its data from object storage.

The benchmarks are published as figures, not as numbers

The results section of the README is worth a close look, because it is where a reader is most likely to be misled.

The quantitative results are two images, a comparison against other methods and a comparison against commercial models. There is no table of numbers in the text of the README, no metrics with values, no list of the datasets the model was evaluated on, and no statement of what the axes of those figures are. The same is true of the gallery and of the comparison tables with the open source methods, which are LQ, SinSR, SeeSR, SUPIR, DreamClear and the authors' own output, and a separate table against commercial models.

So the claims are visual. A reader can see that the authors' column looks better in the rows that were chosen, and cannot check whether the metric is perceptual, how the competitors were run, or what the numbers were.

That is normal for a project README of this kind, and the figures are the published artefact of the paper, where the tables and the metrics live. But it does mean that the README is not the place to evaluate the method, and a reader who wants to know whether the restoration is better has to go to the arXiv paper.

The rest of the repository supports the same conclusion. There is no `tests/` directory and no test script in the requirements or elsewhere, which is consistent with a repository whose artefact is a model and a training pipeline rather than a library with a test suite. The `image_datasets/` and `train_configs/` directories hold the data references and the training configurations, `assets/` and `images/` hold the figures, `scripts/` and `tools/` hold the running code, and `src/` holds the model.

Where LucidFlux is the wrong tool

The limits are visible in the numbers the README volunteers, and they are substantial.

The first is the memory. A large diffusion transformer guided by a dual-branch conditioner is a big model, and the README's own floor is 28 GB of VRAM for inference even with offload and precomputed embeddings. Only the community ComfyUI integration brings that to 8 to 12 GB, and that route is a third-party project with its own maintenance. A team without a card in that range, or without the appetite to use a ComfyUI-based deployment, is looking at the Fal API.

The second is that the evaluation cannot be checked from the repository. The quantitative results are figures, so a reader is asked to trust a picture. That is a poor basis for a procurement decision, and it is a good reason to read the paper before spending anything on the model.

The third is the shape of the method. Conditioning on a caption is a real constraint that this work removes, and for a restoration job it is the right constraint to remove, since the clean image is by definition unavailable. But a caption-free conditioner is a design that was chosen for a use case with no captions, and a team whose restoration jobs do have good captions, because they are working from a scanned book with a known text or a set of matched frames, is paying for flexibility it does not need.

The alternative, then, is the plain one: an API call, or a smaller restoration model that runs on the hardware you have, or a classical approach for the cases where the degradation is known and regular. The difference in approach is that LucidFlux spends a large model's capacity on a conditioning problem that a smaller model solves with a prompt, and the trade pays off when there is no prompt available and the model has to work from the degraded image alone.

Editorial conclusion

LucidFlux fits a team that has photographs to restore and no captions for them, which is the normal state of a restoration job, and that is the contribution: the conditioning comes from the image rather than from a description of it. It is the wrong choice for a team with modest hardware, because the model is a large diffusion transformer and even with offload and precomputed embeddings the README quotes 28 GB of VRAM for inference. Before adopting it, look at the training data and the filtering pipeline in `tools/` rather than at the gallery, and read the quantitative comparison as a figure rather than as a table, because the numbers in this repository are published as images and cannot be checked from the text of the README.

Frequently asked questions

Official sources

1. Issues
2. README
3. W2GenAI-Lab/LucidFlux on GitHub