DiffSinger: the AAAI 2022 singing voice synthesis codebase, and what it actually ships with

The problem DiffSinger was written to solve

Singing voice synthesis is harder than speech synthesis for a specific reason: pitch is not an inferred property, it is a constraint. A singer's F0 must follow a score. Early diffusion vocoders produced convincing timbre but were slow, because they denoise from pure noise over many steps. DiffSinger's contribution, as the paper title states, is a shallow diffusion mechanism: instead of starting the reverse process from Gaussian noise, the model starts it from an intermediate noised version of the target mel spectrogram, which shortens the number of denoising steps needed.

The repository is for people who work at the model level: researchers reproducing the AAAI 2022 results, students training an SVS system on a small dataset, and engineers who want to swap the pitch predictor or the vocoder and measure the difference. The README's overview table is the clearest statement of scope. It lists seven pipelines across three datasets (Ljspeech, PopCS, OpenCpop), three pitch input modes (none, ground truth F0, MIDI), and three acceleration methods (shallow diffusion, PLMS via PNDM, none). If you want a product that turns a MIDI file into a WAV with one command, this is the wrong layer of the stack.

How the shallow diffusion mechanism is wired into the pipelines

The README's table is effectively the architecture diagram. DiffSpeech takes text and produces F0, then mel, then waveform through HiFiGAN. DiffSinger on PopCS skips the F0 prediction stage entirely and takes ground truth F0 as input, so it is lyric plus F0 to mel, then mel to waveform through NSF-HiFiGAN. The OpenCpop cascade variant is the fuller system: lyric plus MIDI to F0, lyric plus F0 to mel, mel to waveform.

The end-to-end variant collapses that: lyric plus MIDI goes straight to mel, with F0 handled implicitly, and the vocoder stage becomes a pitch extractor followed by NSF-HiFiGAN. The PNDM variant replaces shallow diffusion with PLMS, which is a different acceleration strategy from the same laboratory. A separate row, FFT-Singer, shares the OpenCpop cascade pipeline but marks the acceleration method as invalid, meaning it is the non-diffusion baseline for comparison.

That table is also the honest limit of what the README tells you. It names the components and the datasets but does not describe the loss, the noise schedule, or how many diffusion steps each configuration uses. For those you have to read the paper or the per-pipeline documents under docs/.

Installing DiffSinger and running a first inference

The README gives two environment paths. The conda path pins the dependency set to a GPU generation, which tells you how old the stack is:

conda create -n your_env_name python=3.8
source activate your_env_name
pip install -r requirements_2080.txt

The comment in the README marks requirements_2080.txt as GPU 2080Ti with CUDA 10.2, and requirements_3090.txt as GPU 3090 with CUDA 11.4. Choose by your card, not by preference. The virtualenv path is the same idea without conda:

python -m venv venv
source venv/bin/activate
pip install -U pip
pip install Cython numpy==1.19.1
pip install torch==1.9.0
pip install -r requirements.txt

Note the ordering: Cython and numpy are installed before torch, and requirements.txt pins numpy==1.19.2 while the manual step installs numpy==1.19.1. Read requirements.txt before you run it. It pins pytorch-lightning==0.7.1, h5py==3.1.0, librosa==0.8.0, numba==0.53.1, and scipy==1.5.4, among others. Those are not version ranges, they are exact pins, and they are the main source of install friction on a modern machine.

Once the environment exists, the README points to docs/README-SVS.md for the singing pipeline and docs/README-TTS.md for the speech pipeline. The July 2022 update added two inference paths for OpenCpop, described in docs/README-SVS-opencpop-cascade.md and docs/README-SVS-opencpop-e2e.md, both under a section titled inference from raw inputs. Those documents, not the README, carry the actual commands. Training runs are monitored with TensorBoard:

tensorboard --logdir_spec exp_name

There is also a pretrained model release, tagged pretrain-model and dated 2022-01-17, which the release name describes as codes with pretrained models. The README does not state which of the seven pipelines those checkpoints correspond to, so match them against your config before assuming they load.

Where DiffSinger breaks, and where it is the wrong tool

The dependency pins are the first failure mode. pytorch-lightning==0.7.1 predates the 1.x API, so any code you write against current Lightning documentation will not run here. numpy==1.19.2 and numba==0.53.1 conflict with the numpy versions most modern scientific stacks expect. Installing this alongside another ML project in the same environment is asking for a resolution failure.

The second limitation is documentation coverage. The README's update log mentions MIDI-A-version and MIDI-B-version support in January and March 2022, but does not explain what distinguishes them. The overview table does not state diffusion step counts, training time, or hardware requirements beyond the two GPU-specific requirements files. If your card is not a 2080Ti or a 3090, the README does not tell you which file to use.

The third is scope. This is not a voicebank format, it does not ship an editor, and it has no mobile target. People searching for DiffSinger voicebanks, DiffSinger mobile, or DiffSinger in OpenUTAU are looking for the ecosystem that formed around the model, not this repository. The README acknowledges that directly in its acknowledgements, thanking Team OpenVPI for maintenance of a separate DiffSinger repository. If your goal is to sing through an editor, this repo is upstream of what you want.

DiffSinger versus the OpenVPI fork

The most useful comparison is the one the README itself makes. MoonInTheRiver/DiffSinger is the official research implementation: seven pipelines, three datasets, the paper's shallow diffusion mechanism, and a dependency set frozen around 2022. Its purpose is to make the published results reproducible and to expose the model internals for experiment.

OpenVPI's DiffSinger, credited in the acknowledgements for maintenance, is the community continuation. The difference in approach is not a matter of quality, it is a matter of what each is optimized for. The official repository keeps the paper's configurations intact, which is why the pins stay at pytorch-lightning==0.7.1 and numpy==1.19.2. A maintained fork can move those pins forward, drop the pipelines nobody uses, and add the tooling that turns a model into something a user can operate. If you are training from scratch to study the mechanism, the frozen pins are a feature. If you are building on top of DiffSinger for anything else, they are a tax you pay for nothing.

That split also explains the search traffic. Queries about voicebanks, OpenUTAU, and mobile all describe the fork's ecosystem, not this codebase.

Licence, maintenance, and the cost of upgrading

The repository is MIT licensed. For a research codebase that is permissive: you can reuse the code in commercial work, and you inherit no copyleft obligation. The MIT grant covers the code in this repository. It does not automatically cover the pretrained checkpoints, and it says nothing about the datasets the pipelines are built on. PopCS is released by this project, Ljspeech is an external corpus, and OpenCpop is hosted at wenet.org.cn. If you intend to ship a trained voice, the terms attached to the training data are a separate question, and the README does not address them. That is not legal advice; it is a pointer to where the unanswered question sits.

Maintenance is best judged from the facts. The repository is not archived, and the last push was on 2026-07-24. The most recent release is the pretrain-model tag from 2022-01-17. The README's update log stops at September 2022. So the code receives occasional pushes, but the documented surface has not moved in years, and the newest tagged artifact is a 2022 checkpoint.

Upgrade cost is therefore not about pulling a new version. It is about whether you can build the pinned environment at all, and whether you want to unpin it. Moving to a current PyTorch Lightning means rewriting the training loop against an API that changed substantially after 0.7.1. The repository provides no migration guide, because it never migrated.

Editorial conclusion

Adopt DiffSinger if you are reproducing the AAAI 2022 shallow diffusion results, training on OpenCpop or PopCS, or comparing cascade and end-to-end SVS pipelines under one codebase. Do not adopt it if you want a maintained synthesis tool with an installer, a GUI, or a voicebank format, because the last push was on 2026-07-24 and the README points users to the OpenVPI fork for that. Before you commit, verify that pip can still resolve pytorch-lightning==0.7.1 and numpy==1.19.2 on your platform, and check whether the pretrain-model release from 2022-01-17 matches the config you intend to run.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. MoonInTheRiver/DiffSinger on GitHub
4. README
5. Releases