FireRedVAD: voice and audio event detection across a hundred languages

Three tasks, two of them streaming

FireRedVAD is described as a state-of-the-art industrial-grade Voice Activity Detection and Audio Event Detection solution, and the task split is what defines the scope. It supports non-streaming VAD and streaming VAD, plus non-streaming AED, and it detects speech, singing and music across 100 plus languages. The distinction between streaming and non-streaming is the one that matters for a production integration, because a streaming detector has to decide as the audio arrives with a bounded look-ahead, while a non-streaming detector sees the whole file and can use future context. A streaming detector therefore has a latency versus accuracy trade that the non-streaming one does not have to make, and the difference is visible in the API surface, where the streaming result is a sequence of per-frame objects carrying the frame index, whether it is speech, the raw probability, the smoothed probability, and the speech start and end flags. The AED task is a different shape again, since it returns a mapping of event type to a list of time ranges plus a ratio per event, so a single recording can be reported as mostly speech, mostly singing and mostly music at once.

The benchmark is 102 languages and the numbers are per metric

FireRedVAD is evaluated on FLEURS-VAD-102, a multilingual VAD benchmark covering 102 languages, and the README's table gives four metrics against four baselines, which is worth reading in full because the metrics disagree with each other.

The four rows are AUC-ROC, where FireRedVAD scores 99.60 against 97.99 for Silero-VAD and 97.81 for TEN-VAD, with no figures listed for FunASR-VAD or WebRTC-VAD. F1 score is 97.57 against 95.95, 95.19, 90.91 and 52.30 in the same order. False Alarm Rate is 2.69 against 9.41, 15.47, 44.03 and 2.83. Miss Rate is 3.62 against 3.95, 2.95, 0.42 and 64.15.

The construction of the test set is given in the footnote, which is the part to read before quoting the headline figure. They randomly selected around 100 audio files per language from the FLEURS test set, resulting in 9,443 audio files with manually annotated binary VAD labels where speech is 1 and silence is 0, and the README says this VAD test set will be open sourced, coming soon. The reading of the table is more interesting than the ranking. FunASR-VAD has the lowest miss rate at 0.42 but a false alarm rate of 44.03, and the README notes that this indicates over-prediction of speech segments, so a detector that says everything is speech scores well on misses and badly on false alarms. WebRTC-VAD is the opposite extreme, with a 64.15 miss rate. FireRedVAD sits in the middle on both, which is the practically useful place to be.

Installing it, and the model weights are a separate step

The setup starts with a clean Python environment, and the recommended path is conda.

    conda create --name fireredvad python=3.10
    conda activate fireredvad
    git clone https://github.com/FireRedTeam/FireRedVAD.git
    cd FireRedVAD  # or fireredvad

Then you choose either the pip route or the source route. The pip route is two commands.

    pip install fireredvad

    # With PyTorch (if not already installed)
    pip install fireredvad[gpu]

The source route depends on your platform, with a separate requirements file for macOS on arm64 and another for Windows and Linux with Nvidia GPUs, and it needs PATH and PYTHONPATH set up.

    # for macos users
    pip install -r requirements-macos-arm64.txt

    # for windows/linux users with Nvidia GPUs
    pip install -r requirements.txt 

    export PATH=$PWD/fireredvad/bin/:$PATH
    export PYTHONPATH=$PWD/:$PYTHONPATH

The model weights are downloaded separately, and the README gives two routes, ModelScope, recommended for users in China, and Hugging Face.

    # Download via ModelScope (recommended for users in China)
    pip install -U modelscope
    modelscope download --model xukaituo/FireRedVAD --local_dir ./pretrained_models/FireRedVAD

    # Download via Hugging Face
    pip install -U "huggingface_hub[cli]"
    huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD

The last setup step is the one that catches people out, because the model expects a specific audio format, and the README asks you to convert your audio to 16kHz 16-bit mono PCM if needed.

    ffmpeg -i <input_audio_path> -ar 16000 -ac 1 -acodec pcm_s16le -f wav <output_wav_path>

The CLI, the scripts and the tunables

Three ways to run it are documented, and the CLI is the most compact. If you installed with pip you call the fireredvad command directly, with a task, a wav path and a model directory.

$ fireredvad --task vad --wav_path assets/hello_zh.wav --model_dir pretrained_models/FireRedVAD/VAD
# #param of DetectModel <class 'fireredvad.core.detect_model.DetectModel'> is 588417 = 2.2 MB (float32)
# Result: {'dur': 2.32, 'timestamps': [(0.44, 1.82)], 'wav_path': 'assets/hello_zh.wav'}

The model directory differs per task, with VAD, Stream-VAD and AED as separate paths, and the printed parameter count tells you the model is small, around 2.2 MB in float32 for each of the three shown. The AED result shows the multi-event structure, with speech, singing and music each carrying their own time ranges and a ratio.

$ fireredvad --task aed --wav_path assets/event.wav --model_dir pretrained_models/FireRedVAD/AED
# #param of DetectModel <class 'fireredvad.core.detect_model.DetectModel'> is 588931 = 2.2 MB (float32)
# Result: {'dur': 22.016, 'event2timestamps': {'speech': [(0.4, 3.56), (3.66, 9.08), (9.27, 9.77), (10.78, 21.76)], 'singing': [(1.79, 19.96), (19.97, 22.016)], 'music': [(0.09, 12.32), (12.33, 22.016)]}, 'event2ratio': {'speech': 0.848, 'singing': 0.905, 'music': 0.991}, 'wav_path': 'assets/event.wav'}

The script route runs three shell scripts in the examples directory, inference_vad.sh, inference_stream_vad.sh and inference_aed.sh. The source installation path uses vad.py, stream_vad.py and aed.py directly, and their flags are the tuning surface. The non-streaming VAD and AED scripts share a common set, a smooth window size of 5, a speech threshold of 0.4, min and max speech frame counts, a min silence frame count, a merge silence frame count, an extend speech frame count, a chunk max frame of 30000 and a write_textgrid flag that emits TextGrid output, plus a save_segment_dir option that writes the segments to disk. The AED script adds singing and music thresholds, both at 0.5 by default, and a min and max event frame count.

The Python API and the packaging underneath

The Python API mirrors the CLI, with a config object and a detector class.

from fireredvad import FireRedVad, FireRedVadConfig

vad_config = FireRedVadConfig(
    use_gpu=False,
    smooth_window_size=5,
    speech_threshold=0.4,
    min_speech_frame=20,
    max_speech_frame=2000,
    min_silence_frame=20,
    ...

The package is published as fireredvad with a version of 0.0.2 in the pyproject file, a development status of Beta, and a console script entry point of fireredvad pointing at fireredvad.bin.fireredvad_cli:main. The runtime dependencies are audio focused, with numpy, kaldiio, kaldi-native-fbank, soundfile and textgrid, and the optional extras split torch into a gpu group and a cpu group that both require torch 2.0.0 or newer, plus a dev group with build, twine and ruff. The source requirements file pins more tightly, with torch at 2.1.0 for the cu118 build and an extra index URL pointing at the PyTorch wheel index, plus exact pins for numpy, kaldiio, kaldi_native_fbank and soundfile. The presence of kaldi-native-fbank and kaldiio is the giveaway that the feature extraction is Kaldi-style, which is the convention across speech tooling, and textgrid as a dependency is what makes the write_textgrid output possible. The repository tree has a runtime/ directory, an examples/ directory with a rust-onnx-example/ inside it, and separate examples folders for wav/ and wav4aed/.

Where FireRedVAD is the wrong choice

The first limit is that the headline benchmark is the authors' own, on a test set they constructed, and the README says the labels were manually annotated and that the set will be open sourced, coming soon. Until it is, the F1 figure cannot be reproduced independently, and the construction matters, since the set samples around 100 files per language from a read speech corpus, which is clean, single-speaker and evenly distributed between speech and silence in a way that call centre audio, meeting audio and noisy street audio are not. The second limit is the format constraint, since the model wants 16 kHz 16-bit mono PCM, which means any pipeline that works in 44.1 or 48 kHz stereo has a resampling stage, and that stage is lossy and is where a surprising amount of quality in a voice pipeline goes. The third is the model size and task shape, because a 2.2 MB float32 detector with a speech threshold and frame counts is a small, adjustable model, and adjusting it is manual work: a threshold tuned on clean read speech will produce false alarms on a noisy channel, and the README does not describe an automatic calibration. The fourth is that streaming and non-streaming are not the same detector, so a system that prototypes on non-streaming and then goes to production on streaming will need re-tuning and re-validation, and the AED task is non-streaming only, so an event-driven application cannot use it. The fifth is project maturity, with a version of 0.0.2 on PyPI, a beta development status, and the code having been split out of the larger FireRedASR2S release as a standalone repository in March 2026, so the API is likely to move.

Editorial conclusion

Adopt FireRedVAD if you need speech segmentation across many languages, or speech, singing and music detection in the same pass, and you want a low false alarm rate rather than a permissive detector. Skip it if your audio is a single language and your needs are modest, since a smaller model may be enough, and read the benchmark construction before you trust the headline number. Verify first that your audio is 16 kHz 16-bit mono PCM, which the model expects, that you download the weights for the task you are running, since the VAD, streaming VAD and AED models are separate directories, and that your own evaluation on real recordings matches the reported figures.

Frequently asked questions

Official sources

1. FireRedTeam/FireRedVAD on GitHub
2. Issues
3. License: Apache-2.0
4. README