voxceleb_trainer: the metric learning trainer behind two speaker papers

A research trainer, tied to two named papers

This repository contains the framework for training speaker recognition models described in two papers, In defence of metric learning for speaker recognition and Pushing the limits of raw waveform speaker recognition. That framing is the important part. It is not a general purpose library and it is not a service; it is the code that produced the numbers in those papers, kept in a form that lets someone else run the same experiments.

The licence file in the repository is MIT, with a NOTICE file alongside it and a copyright line for NAVER Corp. The code is from a company research group, and the MIT terms mean you can take it, change it and ship it without a copyleft obligation attached.

The layout follows the experiment structure. SpeakerNet.py and trainSpeakerNet.py are the model wrapper and the training entry point, dataprep.py handles the dataset, and there are directories for configs, loss, models, optimizer, scheduler, lists and utils. DatasetLoader.py and tuneThreshold.py sit at the top level, and References.md holds the full list of methods used in the trainer.

The dependency list is small and modern enough to be unremarkable: torch and torchaudio at 2.0 or newer, asteroid-filterbanks, numpy, scipy, scikit-learn, tqdm, pyyaml and soundfile. Python 3.8 or newer is required.

There are no GitHub releases, so there is no version to pin. The last push was on 2026-04-22.

Data preparation is four separate commands and two system tools

The dataset is VoxCeleb, and preparing it is a sequence of stages in dataprep.py rather than one command. The first stage downloads the archive and needs an access key, and the rest unpack and convert it.

python ./dataprep.py --save_path data --download --key ACCESS_KEY
python ./dataprep.py --save_path data --concatenate
python ./dataprep.py --save_path data --extract
python ./dataprep.py --save_path data --convert

If you want data augmentation, there is a fifth stage on top of that.

python ./dataprep.py --save_path data --augment

The system requirements are easy to overlook because they sit in a sentence after the commands: in addition to the Python dependencies, wget and ffmpeg must be installed on the system. The extraction and conversion stages depend on them, and the failure mode is a broken stage rather than a helpful error.

The installation of the Python side itself is the usual one.

pip install -r requirements.txt

Once the data is in place, the trainer expects a list file where each line carries an identity and a file path, one utterance per line.

id00000 id00000/youtube_key/12345.wav
id00012 id00012/21Uxsk56VDQ/00001.wav

The README points at downloadable train and test lists for VoxCeleb2 and VoxCeleb1, and it flags a trap for the VoxCeleb1-E and VoxCeleb1-H test lists, which mix the dev set and the test set, so you have to merge both directories or adjust the paths in the trial list before evaluating.

The config file overrides the command line

Training runs go through trainSpeakerNet.py, and the usual invocation points at a config file in configs/.

python ./trainSpeakerNet.py --config ./configs/ResNetSE34L_AM.yaml
python ./trainSpeakerNet.py --config ./configs/RawNet3_AAM.yaml

Individual arguments defined in trainSpeakerNet.py can be passed as --{ARG_NAME} {VALUE}, and the boolean flags --eval, --distributed, --mixedprec, --augment and --log_input take no value, you just pass them.

Then there is the line that catches people. The README states that the configuration file overrides the arguments passed via the command line, which is the opposite of what almost every command line tool does, and it is the kind of detail that produces a two hour mystery when a flag appears to be ignored. The behaviour is deliberate, since the config files encode the exact settings of the paper's experiments, but it means the config is the source of truth and the command line is a convenience for the values the config does not pin.

It also means the config files are worth reading before you run anything, since they are the record of what the published models were actually trained with.

Seven loss functions and three model families

The trainer's core content is a set of interchangeable objectives and encoders, and the README lists them compactly.

The loss functions are Softmax, AM-Softmax, AAM-Softmax, GE2E, Prototypical, Triplet and Angular Prototypical. The model and encoder combinations are ResNetSE34L with SAP and ASP, ResNetSE34V2 with SAP and ASP, and VGGVox40 with SAP, TAP and MAX.

The interesting design decision is that objectives and encoders are decoupled, so the same backbone can be trained with a plain softmax, with the additive angular margin variants, or with a metric learning objective such as angular prototypical or triplet. That is what makes the paper's central argument testable inside the framework, since the claim is about which objective, not about which architecture.

Extending it is a documented, small job. New models go in the models directory and new loss functions in the loss directory, and the README simply points you at the existing definitions as examples. There is no plugin system to learn and no registry to register with.

Online data augmentation is switched on with --augment, and the augmentation approach is the one described in [2], the VoxSRC 2020 paper.

Reproducing a paper number means matching the batch arithmetic

The README has a section on replicating the results from the paper, and it is the most practically useful part of the documentation because the details it fixes are the details that decide whether your run matches.

First, the names differ. VGG-M-40 in the paper is VGGVox in the repository, Thin ResNet-34 is ResNetSE34, Fast ResNet-34 is ResNetSE34L, and the H and ASP variant from the second paper is ResNetSE34V2. If you follow a paper's model name into the code without translating, you will train the wrong thing or fail to find it.

Second, and more awkward, the batch size in the paper is nPerSpeaker multiplied by batch_size in the code. The paper's batch of 800 is therefore --nPerSpeaker 2 --batch_size 400, or --nPerSpeaker 3 --batch_size 266, and so on. The code does not take 800 directly.

Third, the models were trained with --max_frames 200 and evaluated with --max_frames 400.

The README closes that section with a configuration it describes as a good balance between speed and performance.

python ./trainSpeakerNet.py --model ResNetSE34L --trainfunc angleproto --batch_size 400 --nPerSpeaker 2

One more ordering rule ties the sections together: the configuration file wins over the command line, so if a config file pins max_frames, passing --max_frames on the command line will not change it.

Pretrained checkpoints let you skip the training entirely

If your goal is to evaluate rather than to train, the repository points at pretrained models and gives the exact commands. The first is a model described in the first paper, and the README states that the script should return EER 2.1792.

python ./trainSpeakerNet.py --eval --model ResNetSE34L --log_input --trainfunc angleproto --save_path exps/test --eval_frames 400 --initial_model baseline_lite_ap.model

A larger model trained with online data augmentation is the second one, and the README states the script should return EER 1.0180.

python ./trainSpeakerNet.py --eval --model ResNetSE34V2 --log_input --encoder_type ASP --n_mels 64 --trainfunc softmaxproto --save_path exps/test --eval_frames 400 --initial_model baseline_v2_smproto.model

The third is a pretrained RawNet3, which is not a file you download but a git submodule you initialise, and the README states the script should return EER 0.8932.

git submodule update --init --recursive

Those expected equal error rates are the reproduction check. They are also a useful sanity test of your own environment, because a mismatch means something in the data, the frames or the checkpoint is not what it should be, and the evaluation command is fast enough to run before you commit a week of GPU time to a training run.

Multi GPU training needs a visible device list and a unique port

Two flags cover the hardware side of training. The --mixedprec flag enables mixed precision training and is recommended for Tesla V100, GeForce RTX 20 series or later models. The --distributed flag enables distributed training.

Distributed training has two setup details that are easy to miss. The GPU indices have to be set before training starts.

export CUDA_VISIBLE_DEVICES=0,1,2,3

And if you run more than one distributed training session at the same time, you have to change the --port argument, otherwise the sessions collide on the same port.

The documentation also names a companion script, tuneThreshold.py, which belongs to the evaluation side of the pipeline: the --eval runs report an equal error rate, and the decision threshold that goes with it is something you tune separately for your own data rather than inherit from the paper.

On the state of the project, the honest summary is that this is a research codebase with a permissive licence, no published releases, and a last push on 2026-04-22. It runs, the checkpoints are downloadable, and the reference numbers in the README are the way to check that your environment reproduces the setup. What it does not have is a release cadence, so treat the branch as a moving target and record the commit you trained from.

Editorial conclusion

voxceleb_trainer fits a research group that wants to reproduce or extend the VoxSRC line of speaker recognition work, and that values having the loss functions, the encoders and the evaluation protocol in one place with the pretrained checkpoints attached. It does not fit a production team that needs a maintained speaker verification service, since the repository is a research trainer with no releases and its most recent push was on 2026-04-22. Verify three things before you start a long run: that the data preparation actually finished, because the four dataprep.py stages are separate commands and a missing stage shows up as a broken training run much later, that your batch arithmetic matches the paper, where nPerSpeaker multiplied by batch_size gives the effective batch of 800, and that the config file you pass with --config does not silently override the flags you thought you were setting, since the README states the configuration takes precedence.

Frequently asked questions

Official sources

1. clovaai/voxceleb_trainer on GitHub
2. Issues
3. License: MIT
4. README