UTMOSv2 predicts speech naturalness from a backbone borrowed from image classification

The backbone comes from an image model zoo

UTMOSv2 is the official implementation of the T05 system from the VoiceMOS Challenge 2024, published as a paper accepted at the IEEE Spoken Language Technology Workshop in 2024.

The paper's title is the interesting part, because it names the mechanism rather than the result: transfer learning from a deep image classifier to naturalness MOS prediction of high-quality synthetic speech. MOS here is mean opinion score, the human rating a naturalness scale is built from.

That choice leaves a trace in the dependency list. Alongside torch, torchaudio, transformers, librosa and numpy, the package requires timm at 1.0.7 or newer, which is a library of pretrained image models. An image backbone inside an audio scorer is not a mistake or a leftover; it is the transfer the paper is about, and a reader looking for the architecture should start there rather than in the audio stack.

The competition result is stated as first place in 7 of 16 metrics and second place in the remaining 9 in the VoiceMOS Challenge 2024 Track 1, which is the kind of claim worth attributing to the challenge rather than treating as a general accuracy figure.

Two lines to a MOS, and three different return types

The library path is short. Installation comes straight from the repository:

uv add git+https://github.com/sarulab-speech/UTMOSv2.git

Then predictions are made with a model created once and reused:

import utmosv2
model = utmosv2.create_model(pretrained=True)
mos = model.predict(input_path="/path/to/wav/file.wav") # Returns a float value

The same `predict` call returns three different things depending on which argument you pass, which is the API detail to know before you write the calling code. Given in-memory `data` as a torch tensor or numpy array with shape (batch_size, sequence_length) or (sequence_length,), it returns a tensor or array of shape (batch_size,), or of shape (1,) for a single item. Given an `input_path`, it returns a float. Given an `input_dir`, it returns a list of dicts with `file_path` and `predicted_mos` keys.

Two precedence rules come with it. When `data` is provided, `input_path` and `input_dir` are ignored, so a stale path in your call will not warn you. And when `data` is None, exactly one of `input_path` or `input_dir` must be set, not both.

Sampling rate defaults to 16 kHz and is not validated

The `sr` argument is the sampling rate of the input audio, and if it is not provided it defaults to 16000 Hz.

That default is the quiet risk in the API. A caller with 22.05 kHz or 44.1 kHz material who omits `sr` gets a confident number computed on audio the model was not given, and the return value looks exactly like a correct one. Nothing in the documented behaviour warns about it, so the rate has to travel with the data in your own code, whether that means a required argument or a wrapper that always passes it.

The same caution applies to the in-memory path, where the array's own sample rate is invisible to the function. You pass `data` and `sr` separately, and only one of them can be wrong silently.

For a metric that exists to compare synthetic speech against a reference, a wrong sample rate is not a small error. It produces a number that looks comparable across runs and is not.

The quick path and the accurate path are not the same path

The project draws the distinction itself, in a note under the quick prediction section: those methods provide quick and simple predictions, and for more accurate predictions and detailed usage of the inference script you should read the inference guide in docs/inference.md.

The script path is the other half of the interface, and it is what you would reach for in a pipeline. You clone the repository, install the package with its optional extra, and call a script that writes CSV:

python inference.py --input_dir /path/to/wav/dir/ --out_path /path/to/output/file.csv

The same script takes `--input_path` for a single file, and again exactly one of the two is required. If `--out_path` is omitted the results go to standard output, which the documentation recommends when the number of files is small.

Two practical notes about the install. The extra called optional contains pandas, so `uv sync --extra optional` is what the script path needs, and under zsh the bracket syntax has to be quoted, as in `pip install -e '.[optional]'`, or the shell eats it.

There is also a hosted demonstration on Hugging Face Spaces and a Colab notebook for the quickstart, which are the fastest ways to see a score before deciding whether to install anything.

Training is documented in three separate documents

Training your own model is handled by documentation rather than by a script in the README, and the three documents have distinct purposes.

The training guide in docs/training.md is where to start if you want to fine-tune UTMOSv2. The reproduction document, docs/reproduction.md, exists specifically to reproduce the training as described in the paper or as used in the competition, which is a different and stricter target than getting a working model. The datasets document, docs/datasets.md, holds the details of the data the project used.

The repository itself carries the entry points: train.py at the top level, inference.py beside it, the utmosv2 package, a tests/ directory, docs/, a poster.pdf, CITATION.cff, a quickstart notebook and a uv.lock.

The optional dependency groups mirror that structure. A train extra brings scikit-learn, wandb and python-dotenv, which is the experiment-tracking stack rather than the model itself. A check extra brings ruff, mypy and type stubs. A test extra brings pytest, and the optional extra brings pandas. So the split is deliberate: the runtime stays small and each workflow adds only what it needs.

One caveat visible in the packaging file: there is a TODO comment about reducing dependencies, above a runtime list of eight packages that includes both transformers and librosa. For a scorer that runs over a folder of wav files, that is a heavier import than you would choose deliberately.

MIT, Python 3.9 to 3.13, and a last commit in April 2026

The licence is MIT, held as a file referenced from the project metadata, with Kaito Baba named as author. The paper's citation is a BibTeX entry for the SLT 2024 workshop, with authors Baba, Nakata, Saito and Saruwatari, and the repository also ships CITATION.cff so citation tooling can pick it up without parsing the README.

Supported Python versions are declared as 3.9 through 3.13, and the version number itself is dynamic, read from `utmosv2.__version__` rather than written into the packaging file. Type checking is configured for Python 3.11 semantics, with missing imports ignored and untyped definitions disallowed.

Release tags are v1.3.0 in February 2026, v1.2.1 in July 2025 and v1.2.0 in May 2025, so the project ships a handful of releases a year rather than continuously.

The last commit landed on 2026-04-02. That matters for one specific reason rather than as a general maintenance signal: this system downloads pretrained weights, so what you want to know is when the weights and the inference path were last changed together, and the commit date is the only figure the repository gives you. Check the release notes for v1.3.0 against the model you actually load.

One gap in the visible documentation: the citation block itself is cut off partway through the page range in this copy, so the pagination has to come from the paper.

Editorial conclusion

Adopt UTMOSv2 if you need a repeatable naturalness score for synthetic speech and want one dependency-light call rather than a hosted service. Do not adopt it as a research reproduction without reading the reproduction guide, because the quick prediction path is explicitly a simplification of the competition setup. Pin the sampling rate yourself, since it defaults to 16000 Hz and a wrong value scores a resampled file, and check the last commit date before assuming the model weights are being refreshed.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. sarulab-speech/UTMOSv2 on GitHub