Matcha-TTS: Conditional Flow Matching for Fast Non-Autoregressive Speech Synthesis

The sampling-cost problem Matcha-TTS was built to remove

Diffusion-based TTS produces good speech, but it does so by running a learned denoising network many times per utterance. The README frames the motivation directly: the authors propose conditional flow matching, described as similar to rectified flows, in order to speed up ODE-based speech synthesis. The stated properties are that the method is probabilistic, has a compact memory footprint, sounds highly natural, and is very fast to synthesise from. Those four claims come from the project's own description, not from an independent measurement, and the ICASSP 2024 paper is the place to check the numbers. The practical consequence for an engineer is that the number of solver steps becomes a tunable knob rather than a fixed architectural cost. The CLI exposes it as --steps, and the README's own example passes --steps 10. The ONNX exporter treats the same quantity as a hyper-parameter fixed at export time, defaulting to 5. This is the target audience: people who want a non-autoregressive acoustic model they can train on a single-speaker dataset, run on modest hardware, and ship as a graph rather than as a Python training loop.

What the architecture actually consists of, and what it does not

Matcha-TTS is an acoustic model. It maps text to a mel spectrogram, and a separate vocoder turns that spectrogram into a waveform. The README is explicit about this boundary in the ONNX section, where the exporter optionally accepts vocoder-name and vocoder-checkpoint arguments so that the vocoder can be embedded in the exported graph and waveforms generated in a single run, similar to end-to-end TTS systems. That option exists precisely because the default arrangement is two stages. Anyone evaluating Matcha-TTS should therefore budget for a vocoder separately, including its licence, its download size and its own inference cost. The text front end is likewise not described in the README beyond the mention of piper_phonemizer in the v0.0.5 release title, so the phonemisation path is something to inspect in the source rather than something the documentation explains. The repository is primarily Jupyter Notebook according to its language metadata, which is consistent with a research release: the training and inference code exists, but the notebook is a first-class entry point, and the README points readers at synthesis.ipynb alongside the CLI and the Gradio app.

Installing and running the command line interface

The README gives a conda environment with Python 3.10 as suggested but optional, then a plain pip install matcha-tts, or an install from source via pip install git+https://github.com/shivammehta25/Matcha-TTS.git followed by cd Matcha-TTS and pip install -e . The badges at the top of the README advertise PyTorch 2.0+, Lightning 2.0+ and Hydra 1.3, so Hydra is the configuration layer and you should expect experiment configs rather than a flat settings file. Synthesis is one command: matcha-tts --text "<INPUT TEXT>". The README states that pretrained models are downloaded automatically by the CLI or the Gradio interface. There is a file mode, matcha-tts --file <PATH TO FILE>, and a batch mode that adds --batched. Three synthesis parameters are documented: --speaking_rate 1.0, --temperature 0.667, and --steps 10 for the Euler ODE solver. A Gradio interface is available as matcha-tts-app, and a HuggingFace Space is linked for browser use. One caveat worth flagging: the v0.0.5 release title mentions fixing a gdown bug, which tells you the automatic checkpoint download depends on Google Drive links. Those links can change or throttle, and the README offers no mirror.

Training on your own corpus: the LJSpeech recipe as a template

The training path is documented through LJSpeech and is specific enough to follow. Extract the dataset to data/LJSpeech-1.1, prepare file lists in the style of the NVIDIA Tacotron 2 setup, then edit configs/data/ljspeech.yaml so that train_filelist_path and valid_filelist_path point at your lists. Before training you must compute normalisation statistics with matcha-data-stats -i ljspeech.yaml, which prints a dictionary of mel_mean and mel_std. The README's example output is {'mel_mean': -5.53662231756592, 'mel_std': 2.1161014277038574}, and those values go back into the same YAML under the data_statistics key. Skipping this step means training against unnormalised targets. The training entry points are make train-ljspeech, or python matcha/train.py experiment=ljspeech. Two variants are documented: experiment=ljspeech_min_memory for a minimum-memory run, and multi-GPU via trainer.devices=[0,1]. The min-memory config is a useful signal that the default recipe assumes a reasonably large card. After training, synthesis from your own checkpoint uses matcha-tts --text "<INPUT TEXT>" --checkpoint_path <PATH TO CHECKPOINT>.

ONNX export and the torch version trap

ONNX support is credited in the README to an external contributor, @mush42, and covers both export and inference. Export requires pip install onnx, then python3 -m matcha.onnx.export matcha.ckpt model.onnx --n-timesteps 5. The README states plainly that n_timesteps is treated as a hyper-parameter rather than a model input, so it must be chosen at export time and cannot be varied at inference; if you omit it, the default is 5. This is a real design constraint. You cannot ship one ONNX file and let callers trade latency against quality, which is exactly the knob the Python CLI exposes as --steps. The second constraint is the torch version: the README says torch>=2.1.0 is needed for export because the scaled_product_attention operator is not exportable in older versions, and that at the time of writing users had to install that version manually as a pre-release. Inference uses onnxruntime or onnxruntime-gpu, with python3 -m matcha.onnx.infer model.onnx --text "hey" --output-dir ./outputs. Synthesis parameters carry over: --temperature 0.4, --speaking_rate 0.9, and --spk 0 for speaker selection. The presence of --spk implies multi-speaker checkpoints are anticipated, even though the training walkthrough is single-speaker LJSpeech.

Where this project is the wrong tool

Matcha-TTS is a research release, not a maintained inference library. The most recent tagged version in the supplied material is v0.0.7 from 2024-08-09, and the repository is not archived, so commits continue, but there is no evidence of a stability or deprecation policy. Treat the Python API as unstable. The documentation is thin in specific places: the README never explains the phonemisation pipeline in prose, never lists the available pretrained voices, and never states the output sample rate or the mel configuration of the released checkpoints. Anyone integrating this into a product will have to read the configs and the source to answer those questions. The checkpoint distribution through Google Drive is a second fragility, and the gdown fix in the release history shows it has already broken once. There is also no documented streaming or incremental synthesis path, so low-latency interactive use is unproven from this material. Finally, if your requirement is cloning an arbitrary voice from a few seconds of reference audio, nothing in the README describes that capability; the documented adaptation route is fine-tuning on a prepared dataset with file lists and normalisation statistics, which is a much heavier workflow.

How it differs from a full end-to-end TTS toolkit

The obvious comparison is a toolkit such as Coqui TTS or Piper, which package text front end, acoustic model, vocoder and a serving story behind one interface. The difference in approach is structural. Those projects hand you a complete pipeline with a fixed model zoo; Matcha-TTS hands you one acoustic model with a training recipe and lets you choose the vocoder, either by supplying vocoder-name and vocoder-checkpoint at ONNX export time or by wiring the two stages yourself. That is more work and more decisions, but it also means you are not locked into a particular vocoder or a particular text normalisation scheme, and the acoustic model itself is small enough that the README's memory-footprint claim is plausible for the design. The trade is maintenance surface. With a toolkit you inherit someone else's integration and upgrade path. With Matcha-TTS you own the glue, and given the v0.0.7 tag date you should assume you will be pinning versions yourself. If your priority is getting acceptable speech out of a pip install today, the toolkit wins. If your priority is controlling the acoustic model and exporting a graph, Matcha-TTS is the more direct route.

Licence, upgrades and what to check before you commit

Matcha-TTS is MIT licensed, which is permissive and places few restrictions on modification and redistribution. That licence covers this repository. It does not automatically cover the pretrained checkpoints hosted on Google Drive, the vocoder you pair with it, or the LJSpeech dataset used in the training example, each of which carries its own terms. This is not legal advice; check each artefact you redistribute. On upgrade cost, the picture is mixed. The Python dependency floor is stated as PyTorch 2.0+, Lightning 2.0+ and Hydra 1.3, but ONNX export raises the torch floor to 2.1.0, so an export pipeline pins you higher than an inference-only pipeline. Hydra configs mean experiment settings live in YAML under configs/, and the LJSpeech data config is the file you will fork for your own corpus. The version history is short and the gaps are uneven: v0.0.5 in January 2024, v0.0.6 in May 2024, v0.0.7 in August 2024. If you adopt this, pin the commit you validated rather than tracking main, and record the mel_mean and mel_std you computed with matcha-data-stats alongside the checkpoint, because a mismatch between normalisation statistics and the checkpoint they were trained with is a silent quality regression rather than a crash.

Editorial conclusion

Adopt Matcha-TTS if you need a small, permissively licensed acoustic model you can fine-tune on your own single-speaker corpus and export to ONNX for CPU or GPU inference, and if you are willing to accept a gdown-based checkpoint download and a vocoder you supply yourself. Do not adopt it if you need multilingual coverage, voice cloning from a short reference clip, or a maintained library with a stable inference API, because the repository is a research release whose last tagged version is v0.0.7 from August 2024. Before committing, verify three things: that the Google Drive checkpoint link still resolves, that your torch build is at least 2.1.0 if you intend to export, and that the licence terms of your chosen vocoder checkpoint are compatible with your distribution.

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. shivammehta25/Matcha-TTS on GitHub