open-mmlab/Amphion: a reproducible toolkit for audio, music and speech generation

What Amphion actually solves, and for whom

Audio generation research has a fragmentation problem. A text-to-speech paper ships one repository, a singing voice conversion paper ships another, and a vocoder is usually a third. Each has its own preprocessing script, its own expected directory layout, and its own evaluation code, so comparing two methods fairly means reimplementing one of them. Amphion's stated purpose is to support reproducible research and to help junior researchers and engineers get started in audio, music and speech generation. The repository groups its work by task: TTS, SVS, VC, AC, SVC and TTA are marked as supported in the README, while TTM (text to music) is marked as developing. Alongside the generation tasks the repository ships vocoders and evaluation metrics, and the README argues that shared metrics matter for consistent comparison. The audience is narrow and honest about it: this is a toolkit for people who will read a paper, download a checkpoint and run training, not an SDK for shipping a voice feature into a product.

How the repository is laid out and how a run flows

The top-level layout is the clearest statement of the design. There is models/, which holds the architectures; egs/, which holds the per-task recipes and their configuration; preprocessors/ and processors/ for turning raw audio and text into training features; modules/, optimizer/ and schedulers/ for the shared training machinery; evaluation/ and egs/metrics/ for scoring outputs; and visualization/, which the README calls out as a distinctive feature intended to help junior researchers understand a model or architecture. A typical run therefore moves through four stages: data lands in a task directory under egs/, a preprocessor extracts the features the model expects, training reads a config from config/ and writes checkpoints, and evaluation computes the metrics. The pretrained/ directory is where released checkpoints live. Two things follow from this. First, the unit of work is the recipe, not the library, so you will spend most of your time in egs/<task>/README.md rather than in a single API reference. Second, because preprocessing and training are separate steps, a mismatch between the feature configuration used to train a checkpoint and the one you preprocess with will surface as a shape error deep in the model rather than as a clear configuration warning.

Installing Amphion with Docker and running a first task

The repository ships a Dockerfile that builds the whole environment, which is the least ambiguous install path. It starts from nvidia/cuda:11.8.0-cudnn8-devel-ubuntu18.04, installs ffmpeg, espeak-ng and the usual build dependencies, downloads Miniconda3-py39_23.3.1-0-Linux-x86_64.sh, and creates a conda environment named amphion with Python 3.9.15. The Dockerfile declares PYTORCH='2.0.0' and CUDA='cu118' as build arguments, so those are the versions the image is built around. The Dockerfile's own build sequence is the reference here: it copies env.sh into the image, makes it executable with chmod +x, and then invokes conda run against the amphion environment with the -n flag. Those are the only environment commands the repository files show, so follow them rather than inventing a wrapper. From there the README does not give a single canonical first command. It links to per-task READMEs, for example egs/tts/README.md for text to speech, egs/svc/README.md for singing voice conversion, egs/tta/README.md for text to audio, egs/vocoder/README.md for vocoders and egs/metrics/README.md for evaluation. The practical first run is to open the README for the task you care about, follow its data preparation step, and check that the pretrained checkpoints referenced there are reachable from your network. If you cannot pull the checkpoint, the recipe will not run, and no amount of environment tuning will change that.

The dataset and licence problem is bigger than the code problem

Amphion's own news entries make the licensing situation concrete. The Emilia-Large dataset is described as combining the original 101k-hour Emilia dataset, licensed under CC BY-NC 4.0, with the 114k-hour Emilia-YODAS dataset, licensed under CC BY 4.0. Those two licences are not interchangeable. CC BY-NC 4.0 forbids commercial use, so a model trained on the combined corpus inherits a restriction that a model trained only on the YODAS portion would not. This is the limitation that matters most in practice, and it is the one most likely to be missed by a junior researcher who follows a training recipe without reading the dataset card. The repository code itself is MIT, which is permissive, but the MIT licence on the code says nothing about the weights you produce or the data you train on. Treat the dataset licence, not the repository licence, as the binding constraint on what you can do with a trained model.

Where Amphion is the wrong tool

Amphion is a research toolkit, and several of its own choices make it unsuitable for production serving. The install path assumes an NVIDIA GPU with a CUDA 11.8 capable driver, because the Dockerfile is built on nvidia/cuda:11.8.0-cudnn8-devel-ubuntu18.04 and pins PyTorch 2.0.0. There is no documented CPU-only path. The recipes are organised per task under egs/, which means upgrading across tasks is not a single version bump, and the release tags in the repository stop at v0.1.1-alpha from 2024-02-23 even though the news entries run through 2026-03-25, so the tagged releases do not describe the current state of the code. If you need a stable, versioned inference API with a compatibility promise, or you need to run on CPU or on a laptop, this is the wrong repository. It is also the wrong choice if your goal is a quick demo: the entry cost is a working CUDA environment plus a dataset you are licensed to use.

How Amphion differs from a single-model speech repository

The obvious alternative for many readers is a single-purpose speech repository, for example one that implements only text to speech with a fixed, well-documented inference interface. The difference in approach is scope versus depth. A single-model repository optimises one pipeline end to end: one preprocessing script, one checkpoint, one inference entry point, and a narrow dependency surface. Amphion optimises breadth and comparability: multiple tasks sharing preprocessors, vocoders and evaluation metrics, so that a TTS result and a singing voice conversion result can be scored the same way. That breadth is exactly what makes the repository harder to adopt, because the shared machinery has to accommodate every task's assumptions. If you are evaluating one method against a baseline, the shared metrics in evaluation/ and egs/metrics/ are the reason to pick Amphion. If you are shipping one voice, the narrower repository will get you there with less environment work.

Maintenance, releases and what upgrading costs

The repository is not archived, and the last push was on 2026-03-25, so work is ongoing. The release tags tell a different story: v0.1.0-alpha, v0.1.0 and v0.1.1-alpha were all published on 2024-02-23, and nothing has been tagged since. In practice that means the code you get from the default branch is ahead of the newest tag by a wide margin, and there is no tagged artefact that corresponds to the models announced in 2025 and 2026, such as Vevo2, DualCodec, Vevo1.5 and Metis. Upgrading therefore means tracking the default branch and reading the per-task READMEs for the models you use, not bumping a version pin. The cost of that is real: a change in a shared preprocessor or in config/ can affect several tasks at once, and because the recipes are separate directories, you will not discover the breakage until you run the affected task. Budget for pinning the repository at a commit rather than at a tag. The MIT licence on the code is permissive and imposes no copyleft obligation, but it grants no rights over the datasets or the released checkpoints, and the README does not state a licence for the pretrained models themselves. That is a question for whoever owns the deployment, not something the repository answers.

Editorial conclusion

Adopt Amphion if you are a research group or an engineer who already has a GPU box and a licensed speech corpus, and you want one repository that covers TTS, SVC, VC, TTA and vocoders rather than five separate codebases. Do not adopt it if you need a supported product with a stable API and a release cadence: the last push was on 2026-03-25, the newest release tag is v0.1.1-alpha from 2024-02-23, and the README points to per-task READMEs rather than a single versioned interface. Before you commit, verify three things on your own machine: that the Dockerfile's CUDA 11.8 and PyTorch 2.0.0 combination matches your driver, that the pretrained checkpoints for your target task are actually downloadable, and that the licence of the dataset you intend to train on permits your intended use.

Frequently asked questions

Official sources

1. License: MIT
2. open-mmlab/Amphion on GitHub
3. Project website
4. README
5. Releases