FireRedASR2S: an all-in-one ASR stack with VAD, LID and punctuation in one repository
A SOTA Industrial-Grade All-in-One ASR system with ASR, VAD, LID, and Punc modules. FireRedASR2 supports Chinese (Mandarin, 20+ dialects/accents), English, code-switching, and both speech and singing ASR. FireRedVAD supports speech/singing/music in 100+ langs. FireRedLID supports 100+ langs and 20+ zh dialects. FireRedPunc supports zh and en.
At a glance
- What is it?
- FireRedASR2S bundles four separately trained models (FireRedASR2, FireRedVAD, FireRedLID, FireRedPunc) behind one Python package. The evaluation tables are the strongest part of the release; the deployment documentation is the thinnest.
- Who is it for?
- Adopt FireRedASR2S if your audio is Mandarin, a Chinese dialect, English, code-switching or singing, and you want VAD, language identification and punctuation from the same vendor instead of wiring four projects together. Do not adopt it if you need a stable pinned dependency set today: pyproject.toml and requirements.txt disagree on torch, torchaudio, transformers and numpy versions, and nothing in the repository resolves that.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 108 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The four problems FireRedASR2S collapses into one repository
A production transcription pipeline is rarely one model. You need a voice activity detector to cut silence and music out of long recordings, a language identifier to route each segment to the right recogniser, the recogniser itself, and a punctuation model to turn a lowercase word stream into something a human can read. FireRedASR2S ships all four as separate checkpoints under one Apache-2.0 repository: FireRedASR2 for recognition, FireRedVAD for voice activity, FireRedLID for spoken language identification, and FireRedPunc for punctuation restoration. The stated audience is teams working with Chinese audio. The model table lists Mandarin plus more than 20 dialects and accents (Cantonese in both the Hong Kong and Guangdong variants, Sichuan, Shanghai, Wu, Minnan, Anhui, Fujian, Gansu, Guizhou, Hebei, Henan, Hubei, Hunan, Jiangxi, Liaoning, Ningxia, Shaanxi, Shanxi, Shandong, Tianjin, Yunnan), English, and code-switching. Singing is covered too: the evaluation includes the opencpop singing test set, where FireRedASR2-LLM reports 1.12% CER against 4.36% for Doubao-ASR. If your problem is a single clean English speaker, the dialect coverage is dead weight and a smaller model will do.
Two recogniser architectures and what each one gives you
FireRedASR2 exists in two variants, and the difference matters more than the CER gap between them. FireRedASR2-LLM is the LLM-based recogniser; the release notes state that vLLM added support for it in pull request 35727. FireRedASR2-AED is the attention-based encoder-decoder variant, and the README states it additionally supports word-level timestamps and confidence scores. That is the practical dividing line. If you need to align a transcript to audio, or filter low-confidence segments before they reach a downstream system, AED is the variant that exposes the data. If you only need the transcript and want the lower error rate, LLM wins on the published averages: 2.89% versus 3.05% on the four Mandarin test sets, and 11.55% versus 11.67% across the 19 dialect sets. The release notes also state that NVIDIA contributed TensorRT-LLM acceleration for FireRedASR2-AED, with a benchmark on the AISHELL-1 test set showing 12.7x speedup over the PyTorch baseline on a single H20. That number is specific to one GPU, one test set and one comparison, so treat it as a direction rather than a capacity plan.
FireRedVAD, FireRedLID and FireRedPunc under the same roof
The supporting modules are not afterthoughts. FireRedVAD is evaluated on FLEURS-VAD-102, a benchmark the repository describes as roughly 100 audio files per language drawn from the FLEURS test set, giving 9,443 files with manually annotated binary VAD labels. On that benchmark it reports 97.57% F1 against 95.95% for Silero-VAD, 95.19% for TEN-VAD, 90.91% for FunASR-VAD and 52.30% for WebRTC-VAD. The false alarm column is the interesting one: FireRedVAD reports 2.69 against 9.41 for Silero-VAD and 15.47 for TEN-VAD. A lower false alarm rate means fewer chunks of music or noise sent to the recogniser, which is where the compute goes. FireRedVAD also supports non-streaming and streaming operation and a multi-label mode, mVAD, for speech, singing and music. FireRedLID reports 97.18% accuracy across 100+ languages and 20+ Chinese dialects, positioned against Whisper and SpeechBrain. FireRedPunc reports 78.90% average F1 for Chinese and English against 62.77% for FunASR-Punc. Every one of these numbers comes from the project's own evaluation, and none of the benchmarks is described as independently reproduced.
Installing FireRedASR2S and running the CLI
The package is on PyPI as fireredasr2s and requires Python 3.11 or later. The pyproject.toml declares a console script named fireredasr2s-cli pointing at fireredasr2s.fireredasr2s_cli:cli_main, and it configures an extra index URL for the CUDA 11.8 PyTorch wheels. The README does not walk through a first transcription, so the example below is the shape of the install rather than a verbatim tutorial. Check the command's own help output before relying on any flag.
pip install fireredasr2s
fireredasr2s-cli --helpModel weights are not bundled. The README points to a Hugging Face collection at FireRedTeam/fireredasr2s and a ModelScope collection, with separate repositories for FireRedASR2-LLM, FireRedASR2-AED, FireRedVAD, FireRedLID and FireRedPunc. Download the checkpoints you need from those two sources before the first run. The repository also carries an examples_infer/ directory and a runtime/ directory at the top level; the README does not document what either contains, so read the files there directly. Note the dependency conflict before you install anything: pyproject.toml pins torch==2.10.0, torchaudio==2.10.0, transformers==5.1.0 and numpy==2.4.2, while requirements.txt pins torch==2.1.0+cu118, torchaudio==2.1.0+cu118, transformers==4.51.3 and numpy==1.26.1. Those two sets cannot both be satisfied, and the repository does not say which one is current.
Where the published numbers stop being useful
Read the per-test-set rows, not the averages. The Mandarin average of 2.89% CER for FireRedASR2-LLM is carried by aishell1 at 0.64% and aishell2 at 2.15%; on ws-meeting it is 4.32%. The dialect picture is more uneven. On md-shanghai-conv FireRedASR2-LLM reports 28.70% CER and on md-shanghai-daily 24.94%, and on md-changsha 23.18%. Those are hard sets for every system in the table, but the point stands: a 11.55% dialect average does not mean 11.55% on your audio. There are also sets where a competitor wins outright. On md-yue-conv, Fun-ASR reports 33.71% while FireRedASR2-AED reports 11.66%, but on md-wuhan Fun-ASR reports 4.34% against 9.60% for FireRedASR2-LLM. On md-chuan-conv and md-chuan-daily, Doubao-ASR is ahead of both FireRedASR2 variants. The honest reading is that FireRedASR2S is strong across the aggregate and not uniformly best per dialect.
FunASR and the alternative approach
FunASR is the closest comparison, and it appears in three of the tables. The architectural difference is packaging. FunASR is a toolkit: you assemble a pipeline from its model zoo, and the components are released and versioned separately. FireRedASR2S is a system: the four modules are trained to work together, share an evaluation methodology, and ship from one repository with one version number, currently 0.0.1. That matters for the interfaces between stages. FireRedVAD's multi-label output distinguishes speech, singing and music, which is exactly the signal a routing layer needs before it decides whether to send a segment to the recogniser or drop it. FireRedLID then labels the language, and FireRedPunc handles the tail. You can build the same chain from FunASR components, and FunASR has the larger surrounding ecosystem. The trade-off runs the other way too: a single-vendor chain means you inherit that vendor's release cadence, its dependency pins, and its bugs as a set.
Licence, maintenance and the upgrade question
The repository is Apache-2.0, and pyproject.toml declares license = "Apache-2.0" with the LICENSE file at the top level. Apache-2.0 permits commercial use and modification and includes a patent grant; it also requires that you preserve notices and state changes. That is a description of the licence text, not legal advice, and the model weights are distributed through Hugging Face and ModelScope rather than inside the repository, so check the terms attached to each checkpoint separately. On maintenance: the last push to the default branch was on 2026-06-02, which is more than three months before this article, and the repository is not archived. The release history in the README runs from 2026-02-12, when FireRedASR2-AED, FireRedVAD, FireRedLID and FireRedPunc were released with weights and inference code, through 2026-02-25 when the FireRedASR2-LLM weights appeared, with the technical report following on 2026-03-12. The README states that finetuning code is coming soon; it is not in the repository. Version 0.0.1 with a dependency set that contradicts requirements.txt suggests the packaging is still settling, so pin your own environment and expect to re-resolve it on upgrade.
Editorial conclusion
Adopt FireRedASR2S if your audio is Mandarin, a Chinese dialect, English, code-switching or singing, and you want VAD, language identification and punctuation from the same vendor instead of wiring four projects together. Do not adopt it if you need a stable pinned dependency set today: pyproject.toml and requirements.txt disagree on torch, torchaudio, transformers and numpy versions, and nothing in the repository resolves that. Before committing, download FireRedASR2-AED and FireRedVAD, run them on a slice of your own dialect audio, and compare the CER against the per-test-set rows in the README table rather than the averages.
Frequently asked questions
Which STT model is best?
There is no single answer, and FireRedASR2S's own tables show why: FireRedASR2-LLM leads the aggregate at 9.67% CER across 24 test sets, but Fun-ASR beats it on md-wuhan and md-chuan-daily, and Doubao-ASR beats it on md-chuan-conv. Pick by the test set closest to your audio.
What is the fastest speech-to-text model?
The README does not publish a latency comparison between FireRedASR2-LLM and FireRedASR2-AED. The only speed figure in the release notes is a TensorRT-LLM benchmark on the AISHELL-1 test set, where FireRedASR2-AED runs 12.7x faster than the PyTorch baseline on a single H20.
How does ASR work?
FireRedASR2S splits the job across four models: FireRedVAD detects where speech, singing or music occurs, FireRedLID identifies the spoken language, FireRedASR2 transcribes the segment, and FireRedPunc restores punctuation. The repository describes this as an all-in-one system rather than a single end-to-end model.
What is the latest speech recognition technology?
The release notes list FireRedASR2-LLM, the LLM-based recogniser whose weights were released on 2026-02-25 and which vLLM added support for in pull request 35727. FireRedASR2-AED is the attention-based encoder-decoder variant and additionally exposes word-level timestamps and confidence scores.
Community notes