Fun-ASR: Tongyi Lab's ASR Models, Native Transformers Support and llama.cpp Runtimes
Fun-ASR speech recognition models, with native Hugging Face Transformers support for Fun-ASR-Nano and separate FunASR, vLLM and llama.cpp deployment paths.
At a glance
- What is it?
- Fun-ASR is a family of end-to-end speech recognition checkpoints from Tongyi Lab, split between a Chinese/English/Japanese Nano model and a 31-language MLT variant. This article covers the two checkpoints, the Transformers quickstart, the FunASR and llama.cpp deployment paths, and where the documentation goes quiet.
- Who is it for?
- Fun-ASR-Nano-2512 is the checkpoint to try if you need Chinese dialect and accent coverage, or Japanese, and you want a single 800M model that runs through plain Transformers without a serving stack. Fun-ASR-MLT-Nano-2512 is the one to pick when your audio spans the 31 listed languages and Mandarin-heavy accuracy is not the priority.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 6 days ago.
- What is it written in?
- Mainly C, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
Two 800M checkpoints with different language coverage
The repository ships two distinct Nano checkpoints, and the README is explicit that their capabilities are not interchangeable. Fun-ASR-Nano-2512 is trained on tens of millions of hours and covers Chinese, English and Japanese. The Chinese side is unusually broad: 7 dialects (Wu, Cantonese, Min, Hakka, Gan, Xiang, Jin) plus 26 regional accents covering Henan, Shanxi, Hubei, Sichuan, Chongqing, Yunnan, Guizhou, Guangdong, Guangxi and more than twenty other regions. English and Japanese cover multiple regional accents as well. The README also lists lyric recognition and rap speech recognition as additional features. Fun-ASR-MLT-Nano-2512 is the multilingual sibling, trained on hundreds of thousands of hours and covering 31 languages: Chinese, English, Cantonese, Japanese, Korean, Vietnamese, Indonesian, Thai, Malay, Filipino, Arabic, Hindi, Bulgarian, Croatian, Czech, Danish, Dutch, Estonian, Finnish, Greek, Hungarian, Irish, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian and Swedish. Both are 800M parameters. The practical consequence is that you choose one. Picking MLT for a Mandarin call-centre workload trades away the dialect and accent training that the base Nano checkpoint was built around, and picking base Nano for European-language audio simply does not work, since those languages are not in its training set. The naming is the only signal in the repository that distinguishes them, so read the table before downloading weights.
How the Transformers path works without the FunASR toolkit
The README's headline claim for the Transformers route is that no toolkit installation and no remote Python code are needed. That second point matters: the example passes trust_remote_code=False to both AutoProcessor.from_pretrained and AutoModelForSpeechSeq2Seq.from_pretrained, which means the model runs through the stock transformers classes rather than executing custom modelling code shipped alongside the weights. The processor exposes an apply_transcription_request method that takes the audio source and a language argument, and returns tensors ready for model.generate. The example sets sampling_rate to 16000 in audio_kwargs, so the pipeline expects 16 kHz input. Decoding is greedy: do_sample=False with max_new_tokens=128. The generated tensor includes the prompt tokens, so the example slices off the input length with generated[:, inputs.input_ids.shape[1]:] before calling processor.batch_decode. The README pins a specific revision string for the model, d93b302ee7fd505e1b3576120fc142fc6f7820e1, and a separate revision for the example audio file. Pinning both is the right instinct for a model whose weights live outside the repository, but it also means the quickstart breaks silently if a hub revision is ever removed. The example runs on CPU with dtype=torch.float32 and torch.set_num_threads(4). Nothing in the README states a GPU requirement for this path.
Installing Fun-ASR-Nano and running your first transcription
The Transformers quickstart is the shortest path to a transcript. The README gives this exact install line, which pins every dependency to a specific version:
python -m pip install 'transformers==5.17.0' 'torch==2.10.0' 'torchaudio==2.10.0' 'librosa==0.11.0' 'soundfile==0.13.1'Note that this is not the same set as the repository's requirements.txt, which lists looser lower bounds (torch>=2.9.0, transformers>=4.51.3) plus funasr, zhconv, whisper_normalizer, pyopenjtalk-plus, compute-wer and openai-whisper. The requirements.txt is the toolkit path; the pinned pip line is the standalone Transformers path. Install one or the other, not both, unless you know why you need the overlap.
Once installed, the README's inference recipe loads the processor and model, then transcribes a remote MP3:
import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
torch.set_num_threads(4)
model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf"
revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1"
audio = "https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512/resolve/272c57b82523ada6fd87095e955f8e29100979ab/example/en.mp3"
processor = AutoProcessor.from_pretrained(
model_id, revision=revision, trust_remote_code=False, token=False
)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id, revision=revision, trust_remote_code=False, token=False,
dtype=torch.float32,
).to("cpu").eval()The processor call and generation step follow, with language set to "en" and sampling_rate at 16000. What you should see is a decoded string printed to stdout. The first run downloads roughly 800M parameters, so budget time for that before you conclude anything about latency. For local files, batches and keyword handling, the README points to examples/transformers/; for a notebook walkthrough it points to examples/colab/fun_asr_nano_transformers.ipynb. To swap in the multilingual checkpoint, change model_id to FunAudioLLM/Fun-ASR-MLT-Nano-2512 and drop the revision pin, since the README gives that revision only for the base Nano repository.
FunASR, vLLM and llama.cpp are separate deployment paths
The repository description states that Fun-ASR-Nano has native Hugging Face Transformers support while FunASR, vLLM and llama.cpp are separate deployment paths. These are not layers on top of one another. The FunASR path is the Python toolkit: the README calls FunASR 1.4.15 the current Python release for source installs, MOSS discovery, and realtime or industrial deployment, installed with python -m pip install -U "funasr==1.4.15". The vLLM path targets batch and streaming serving, with a guide at docs/vllm_guide.md and an example at examples/vllm_batch.py. The llama.cpp path is distributed as versioned runtime packages: the releases listed in the repository include runtime-llamacpp-v0.1.9, runtime-llamacpp-v0.1.10 and runtime-llamacpp-v0.2.1, and the README's news section points at a runtime-llamacpp-v0.2.6 tag with verified llama.cpp and GGUF packages for Linux, macOS and Windows. The repository topics include gguf, llama-cpp and on-device, which matches that packaging. The version numbering across those two families is worth reading carefully before you pick a tag: 0.1.10 and 0.2.1 are different lines, and the newer number is not always the one with the most patches. For realtime work the repository also contains serve_realtime_ws.py and a streaming SDK example at examples/streaming_sdk.py, and the news section describes realtime WebSocket serving as part of the production deployment story.
Where Fun-ASR is the wrong tool
Speaker diarization is the clearest boundary. The repository has topics for it and an example file, examples/speaker_diarization.py, so the capability exists somewhere in the FunASR ecosystem. But the README's own news section attributes offline long-form transcription with timestamps and anonymous speaker labels to MOSS-Transcribe-Diarize, described as a third-party OpenMOSS model, not to the Fun-ASR checkpoints. If your requirement is per-speaker labels on long recordings, the model you want is not the one this repository is named after. Timestamps are a similar gap. The Transformers recipe returns text via batch_decode and nothing else; the README does not document word-level or segment-level timestamp output for that path, and max_new_tokens=128 caps a single generation, which is a short utterance, not a meeting. Long-form audio therefore needs either chunking on your side or one of the toolkit paths, and the README does not spell out which one handles it. Language coverage is the third boundary. Base Nano is Chinese, English and Japanese only; if your audio includes Korean or Vietnamese, you need MLT, and if it includes both Cantonese dialects and European languages, neither single checkpoint covers the full set. Finally, the README does not document rollback behaviour, version compatibility between the llama.cpp runtime packages and the Python toolkit, or what happens when a pinned hub revision disappears. For a pipeline that must run unattended for years, that is an unanswered question rather than a solved one.
Fun-ASR and Whisper, and what the comparison actually turns on
The obvious alternative is OpenAI's Whisper, which appears in this repository's own requirements.txt as openai-whisper and in the topics as whisper-alternative. The architectural difference is the one that matters for deployment. Whisper is a single multilingual model with one fixed language set and a fixed encoder-decoder shape; you install it, download one checkpoint, and the same weights serve every language it knows. Fun-ASR splits the same 800M parameter budget across two checkpoints with different training data and different language lists. That split is a deliberate trade: base Nano concentrates its capacity on Chinese dialects, regional accents, lyrics and rap, at the cost of the 31-language breadth that MLT carries. Whisper gives you one artefact to manage and a language list that is neither of Fun-ASR's; Fun-ASR gives you a checkpoint choice and, per the README, verified GGUF packages for Linux, macOS and Windows through the llama.cpp runtime line, which is a packaging story Whisper's reference implementation does not ship in the same form. A second comparison people search for is against Qwen ASR. The repository does not include such a comparison, and the only concrete relationship visible here is shared lineage: the GitHub organisation is QwenAudio, and the README describes Fun-ASR as coming from Tongyi Lab. Anything beyond that would be speculation.
Licence, maintenance and the cost of the pinned stack
The repository is Apache-2.0 and the default branch is main. It is not archived, and the last push was on 2026-09-10, six days before this writing, so the code side is moving. The release history reinforces that: runtime-llamacpp-v0.1.9 on 2026-07-24, v0.1.10 on 2026-09-05 and v0.2.1 on 2026-08-27, with the README's news section pointing at a v0.2.6 tag. Three runtime releases in under two months across two version lines is a fast cadence, and fast cadences have a cost. Every one of those tags is a separate artefact you may need to validate against your audio. Apache-2.0 is permissive and imposes no copyleft obligation on your own code, but it covers the repository; the model weights live on Hugging Face and ModelScope under their own terms, and the README does not restate those terms in the repository itself. Check the model card for the checkpoint you actually deploy rather than assuming the repository licence carries over. On upgrades, the pinned Transformers recipe is the sharpest constraint: transformers==5.17.0, torch==2.10.0 and torchaudio==2.10.0 are exact versions, so any other package in your environment that needs a different Torch will conflict. The requirements.txt path is looser (torch>=2.9.0, transformers>=4.51.3), which makes it easier to integrate but harder to reproduce. Pick deliberately, and note that the README does not describe a migration path between the two.
Editorial conclusion
Fun-ASR-Nano-2512 is the checkpoint to try if you need Chinese dialect and accent coverage, or Japanese, and you want a single 800M model that runs through plain Transformers without a serving stack. Fun-ASR-MLT-Nano-2512 is the one to pick when your audio spans the 31 listed languages and Mandarin-heavy accuracy is not the priority. Do not adopt either if your pipeline depends on word-level timestamps or per-segment confidence scores: the README documents neither, and the offline long-form path it points at is a third-party model, MOSS-Transcribe-Diarize, not Fun-ASR itself. Before committing, verify the pinned revision d93b302ee7fd505e1b3576120fc142fc6f7820e1 still resolves on the Hugging Face hub, confirm transformers==5.17.0 installs cleanly on your Python version, and check whether the 800M weights fit the memory budget of the hardware you actually intend to run on.
Frequently asked questions
How does Fun-ASR compare with Whisper?
Fun-ASR splits its 800M parameter budget across two checkpoints: Fun-ASR-Nano-2512 covers Chinese, English and Japanese with 7 Chinese dialects and 26 regional accents, while Fun-ASR-MLT-Nano-2512 covers 31 languages. Whisper appears in this repository's requirements.txt as openai-whisper and in the topics as whisper-alternative, but the README does not publish a head-to-head comparison, so the concrete difference visible here is the checkpoint split and the llama.cpp / GGUF packaging for Linux, macOS and Windows.
How does Fun-ASR compare with Qwen ASR?
The README does not compare Fun-ASR with Qwen ASR. The only relationship the repository shows is shared lineage: the GitHub organisation is QwenAudio and the README describes Fun-ASR as a family of models from Tongyi Lab.
Can I use Fun-ASR for realtime transcription?
The repository includes serve_realtime_ws.py and examples/streaming_sdk.py, and the README's news section describes realtime WebSocket serving as part of the production deployment story alongside native vLLM batch and streaming paths. The README does not give latency figures for that path.
Which Fun-ASR checkpoint should I pick for Chinese dialect audio?
Fun-ASR-Nano-2512 is the checkpoint trained for that case, covering Chinese, English and Japanese with 7 dialects (Wu, Cantonese, Min, Hakka, Gan, Xiang, Jin) and 26 regional accents. Fun-ASR-MLT-Nano-2512 trades that dialect and accent training for 31-language coverage.
Community notes