RDDM: the official code for Residual Denoising Diffusion Models

A paper implementation, not a library

RDDM is described as the official implementation of Residual Denoising Diffusion Models, and the repository is linked to the CVPR 2024 paper of the same name, with the arXiv entry, a video, a blog post and Chinese versions of the paper, the video and the blog linked alongside the English ones.

That framing settles what kind of artefact this is. There is no package name in the readme, no pip install instruction and no import to write. The top level of the repository is a readme, an eval directory, an experiments directory, an img directory, an install.yaml and a poster image, and that is the whole surface.

What it is good for is reproduction and extension. The paper's results table is Table 3 in the main paper, and the readme links to result sets for each restoration dataset, so a reader can compare a run against published numbers. The pre-trained models are hosted on a Chinese academic sharing service, with two unets for the deresidual and denoising stages, and they are provided for the partially path-independent generation process.

There is also a detail that makes the repository more useful than a typical research drop. For image generation on the CelebA dataset, the readme says a pre-trained DDIM can be converted to RDDM by coefficient transformation, and there is a dedicated experiment directory for that. Converting an existing checkpoint rather than retraining from scratch is a difference between a weekend and a week of compute.

For the contact, the readme asks readers to get in touch with Jiawei Liu, who is the first author of the paper.

One conda file, with a warning attached

The requirements section is unusually short, and the warning under it is the useful part.

The whole environment is created from a single file.

conda env create -f install.yaml

The readme adds that if an error occurs, you may need to install the packages one by one, and it links a specific issue comment for that. That is an admission worth taking at face value, because an environment file that fails on a given machine is a common experience with research code, and the remedy the maintainers suggest is a manual fallback rather than a fix.

The reason is usually the CUDA side of the stack. A diffusion model implementation depends on a particular PyTorch build, and the pinned versions in an environment file are often tied to the machine the author used. A reader who is on a different GPU, a different CUDA version or a different driver is the one who hits the failure, and the one by one install path is a sign that the pinned combination is not portable.

The practical advice is therefore to create the environment, try the import that pulls in the model code, and if it fails, expect to reconcile the package versions yourself rather than waiting for a fix. The maintainers are reachable through the issue tracker, and the linked comment shows they have answered this kind of question before.

The datasets are the standard restoration benchmarks

The dataset section lists six sources, and the choice tells you what the model was built for.

Raindrop comes from the DeRaindrop repository, and the readme points at the test-a split for testing. GoPro is the deblurring dataset, ISTD points at the ST-CGAN project, SID-RGB is available either through a personal page or a shared drive, and LOL is the low light dataset. The sixth is CelebA, and it is the one used for the image generation experiments rather than for restoration.

Five of the six are image restoration benchmarks, and they cover the degradation types the model was designed to remove: rain streaks in the Raindrop case, motion blur in GoPro, shadows and uneven lighting in ISTD, sensor noise in SID-RGB, and low light in LOL. The experiments directory names two of them directly, with a directory called 2_Image_Restoration_deraing_raindrop_noise1 and another for the automatic objective selection algorithm on the LOL dataset.

Two practical points about the data. First, none of it is downloaded for you: the repository links out to the original sources, and in several cases to shared drives, so the datasets have to be fetched and placed according to the convention each experiment directory expects. Second, the results links are also hosted on the same academic sharing service, which means a reader outside that network may need to work around the download.

For a first run, the LOL and GoPro experiments are the ones with a clearly stated purpose in their names, and the Raindrop one is the one the readme highlights for the deraining and denoising case.

Training and the two evaluation paths

Training is run from inside an experiment directory, which is the shape of the whole repository.

cd experiments/xxxx
python train.py

The alternative is the same script launched through Accelerate, which is the Hugging Face launcher for distributed and mixed precision training.

accelerate launch train.py

Evaluation splits in two, and the split is by task rather than by convenience.

For image generation, the evaluation is a Python script that computes the FID and Inception Score.

cd eval/image_generation_eval/
python fid_and_inception_score.py path_of_gen_img

For image restoration, the readme says the MATLAB evaluation codes are in the eval directory, and no Python alternative is given. That is a real constraint rather than an oversight, and it comes from the measurement conventions of the restoration literature, where the standard metrics have historically been implemented in MATLAB. A team that wants to reproduce Table 3 therefore needs a MATLAB licence or a compatible reimplementation, and that is worth knowing before the first night of training is planned.

The readme also documents the citation for the work in BibTeX form, with the authors, the title, the CVPR proceedings, the month, the year and the page numbers, which is the detail a paper repository should get right and many do not.

The experiment names are the configuration

The experiments directory is where the design decisions are actually written down, because each directory name encodes the settings of the run.

The first is the partially path-independent generation process, which is the setting the pre-trained models are provided for. The second is the conversion of a pre-trained DDIM to RDDM, the coefficient transformation route described earlier. The third is image restoration, with a name that specifies deraining a raindrop with noise level 1. The fourth is the automatic objective selection algorithm, tested on the LOL dataset with a timestep of 5, a learning rate of 1e-2 and a reinitialised weight.

The fifth is image inpainting, and its name records the image size of 64, a batch size of 64, that the prediction is of the residual, that noise is at the centre with a mask, and that both the mask and the input are withheld. The sixth is image translation from dog to cat, again with the input withheld. The seventh is an image generation run whose name says it decreases alpha and increases beta, referring to Table 2 of the paper.

Read together, those names are a summary of what the method was demonstrated to do, and of the knobs that mattered: what is predicted, what is conditioned on, and which sampling parameters are varied.

The naming is also a practical annoyance. A directory called 4_Image_Inpainting_imgsize64_batch64_pred_res_noise_centermask_wo_mask_wo_input is a specification rather than a title, and reading the code inside one of them takes longer than the name takes to read. For a reader trying to work out which experiment to copy, the experiments directory listing is the documentation that matters, and there is no shorter index of it in the repository.

Where RDDM is the wrong tool

Four reasons, and they all come from the same fact, which is that this is a paper's code rather than a maintained package.

The first is that there is nothing to depend on. No package on PyPI, no import to add, no version constraint to express. Adopting RDDM means copying a directory, which means you own it, which means you also own the bug fixes, and there is no upgrade path when the authors publish one.

The second is the environment. The conda file is the documented setup and the readme warns that it may fail, which in practice means a machine with a different GPU stack will spend its first afternoon on dependency reconciliation.

The third is the evaluation. The generation metrics are a Python script, and the restoration metrics are MATLAB code, so reproducing the published numbers requires a MATLAB environment that most Python focused teams do not have.

The fourth is the licence, and it is the one to raise first with the authors. The repository records the licence as unknown and there is no licence file in the top level listing, so there is no stated permission for reuse, modification or redistribution of this code, which for a research implementation intended for others to build on is an odd position to be in. Ask, and get it fixed in the repository, before you put it inside anything.

On maintenance, the last push was on 2026-06-09 and the repository publishes no GitHub releases, so there is no version history to point at and no changelog to read.

Editorial conclusion

RDDM is the right starting point for a researcher who wants to reproduce or extend the paper, and it is unusually well organised for that, with named experiment directories, pre-trained models for the restoration benchmarks and both generation and restoration evaluation paths. It is the wrong tool for a production image pipeline, since there is no package to install, no stable API and no documented licence in the repository listing. Before building on it, read the experiments directory to see which configuration matches your task, check the licence position with the authors because the repository records it as unknown, and confirm the MATLAB based restoration evaluation is something you can run in your environment.

Frequently asked questions

Official sources

1. Issues
2. nachifur/RDDM on GitHub
3. Project website
4. README