Dolphin ASR covers 40 Eastern languages and 22 Chinese dialects

What Dolphin is actually for: Asian language coverage, not general ASR

Dolphin is a multilingual, multitask ASR model developed through a collaboration between Dataocean AI and Tsinghua University. Its stated coverage is 40 Eastern languages across East Asia, South Asia, Southeast Asia and the Middle East, plus 22 Chinese dialects, and it is trained on more than 210,000 hours of data combining DataoceanAI proprietary sets with open-source datasets. The same model performs four jobs: speech recognition, voice activity detection, segmentation and language identification. That combination is the reason to look at it. A system that must decide where speech starts, split it into segments, detect which language is being spoken and only then transcribe it can do all four from one set of weights, instead of chaining three separate models. The Chinese dialect coverage is the sharper differentiator. Most general purpose multilingual recognizers treat Mandarin as one language, and the dialect work here is substantial enough that a separate model family, Dolphin-CN-Dialect, exists for it.

Joint CTC-Attention with an E-Branchformer encoder

The architecture follows the design of Whisper and OWSM, and the project says so plainly rather than claiming an original structure. The specific choices are a joint CTC-Attention architecture, an encoder based on E-Branchformer and a decoder based on a standard Transformer. Joint CTC-Attention means the acoustic and language modeling paths are trained together, with CTC supplying the alignment signal that a purely attention-based decoder would otherwise have to infer. Two deliberate departures from the Whisper line matter for how you use the model. Dolphin does not support translation tasks, and it eliminates the use of previous text and its related tokens. Dropping previous-text conditioning is a real trade-off: it removes the ability to use preceding context to disambiguate, and it removes a whole task category. The README does not explain what is given to the decoder instead, so the practical consequence is that context comes from the hotword and prompting paths rather than from a transcript prefix.

The two-level language token system

The most distinctive mechanism in the project is a two-level language token system introduced to handle linguistic and regional diversity, which the project notes is especially needed for the Dataocean AI dataset. The first token names the language, for example <zh> or <ja>, and the second token names the region, for example <CN> or <JP>. Both are visible in the command-line interface, where language and region are passed separately with --lang_sym and --region_sym. This is what makes dialect selection a first class decision rather than a model choice. A user who wants Cantonese or a regional variety does not have to pick a different checkpoint; they pass the region token, and the model family that carries the region token is the one whose weights were trained for it. Two token levels also means two things to get right. If you pass a language symbol without a region, you are asking the model to guess which regional distribution to assume, and the accuracy of that guess is not documented.

Installing dataoceanai-dolphin and preparing the audio

The tool needs FFmpeg on the system, because it converts the input audio file to WAV format. The setup notes list the installation route per platform:

# Ubuntu or Debian
sudo apt update && sudo apt install ffmpeg

# MacOS
brew install ffmpeg

# Windows
choco install ffmpeg

Then install the package itself:

pip install -U dataoceanai-dolphin

The alternative route installs from source:

pip install git+https://github.com/SpeechOceanTech/Dolphin.git 

That second command points at a SpeechOceanTech repository path while the project URL elsewhere in the package metadata points at github.com/DataoceanAI/dolphin, so a source install may not fetch what you expect. The setup file declares python_requires=">=3.7" and registers a console script named dolphin bound to dolphin.transcribe:cli, which is where the command-line entry point lives. Dependencies are read from requirements.txt at install time and include modelscope, addict==2.4.0, pydub and funasr. Since models are fetched from ModelScope, Hugging Face and related hubs, expect the first run of a given checkpoint to download weights before any transcription happens.

Running the command line with hotwords, prompts and word timestamps

The default model is small, so a first invocation needs only an audio path:

dolphin audio.wav

Select a checkpoint and pin the language and region when the language is known:

dolphin audio.wav --model small.cn --lang_sym "zh" --region_sym "CN"

The hotword path takes a file of terms. The encoder-biased variant passes --use_deep_biasing true:

dolphin audio.wav --model small.cn --hotword_list_path hotwords.txt --use_deep_biasing true

A prompt-based checkpoint uses different flags, with a two stage filter:

dolphin audio.wav --model small.cn.prompt --hotword_list_path hotwords.txt --use_prompt_hotword true --use_two_stage_filter true

Word level timestamps are available on the same family of models:

dolphin audio.wav --model small.cn.prompt --word_timestamp true

The Python API mirrors these options. The hotword example in the docs passes a list directly rather than a file path, and a prompt model adds decoding_method='attention', so the flag combinations are not interchangeable between the command line and the library.

The Python entry points and the model table

The library surface is small: load a model, then transcribe. In Python that is two calls.

import dolphin
from dolphin import transcribe

model_name = 'small.cn'
model = dolphin.load_model(model_name, device="cuda")

result = transcribe(model, 'audio.wav')
print(result.text)

Language, region and hotword options are keyword arguments on transcribe, and use_deep_biasing and use_two_stage_filter enable the same behaviours as the command-line flags. The model table lists nine named variants. base is 0.1 B parameters and small is 0.4 B, medium is 0.9 B and large is 1.7 B, and the Chinese dialect line adds base.cn, base.cn.streaming, small.cn, small.cn.streaming and small.cn.prompt. The surrounding text says there are 8 models and that 6 are available now, while the table marks seven rows as publicly available, so the counts in the documentation do not line up. The medium and large rows carry no availability mark at all, which is the detail worth acting on if you were planning around a 0.9 B or 1.7 B checkpoint.

Where Dolphin is the wrong tool

The clearest wrong-tool case is translation. The project states that Dolphin does not support translation tasks, so a pipeline that needs to turn foreign speech into another language needs a different model. The second is context-dependent recognition. Because previous text and its tokens are eliminated, you lose the ability to feed the model a transcript prefix, which some ASR use cases rely on for names, product identifiers and numbers. The third is the model count. If the release you need is medium or large, the documentation gives you no way to obtain it. A fourth, quieter limitation is the audio path. FFmpeg is a hard requirement, not an optional convenience, and any deployment that feeds raw compressed audio into the tool needs a conversion step in front of it. The project also states that Dolphin does not support translation tasks while simultaneously reporting capability for segmentation, voice activity detection and language identification, so if your requirement list includes any task beyond the four named ones, verify it against the task list rather than the general description.

Licence, dependencies and the release history

Both the code and the model weights are released under the Apache 2.0 licence, which is the same terms the setup metadata declares in its license field. That covers weights as well as code, so a commercial deployment does not need a separate weight licence, though the licence carries no warranty. The dependency list is short but heavy: funasr pulls in a substantial ASR stack, modelscope handles model distribution, pydub handles audio slicing, and addict is pinned to exactly 2.4.0. That pin means an upgrade of addict elsewhere in your environment can collide with the install. On releases, the two tagged versions are v20250716 published on 2025-07-17 and v20250409 published on 2025-04-09, and the version numbering follows a date pattern rather than a semantic one. The last push was on 2026-06-11, and the changelog entry dated 2026-05-09 records the release of the Dolphin-CN-Dialect small and base models, including streaming and prompt variants, along with word timestamp prediction for all Dolphin models. The project is not archived and the recent push sits within the last few months.

Editorial conclusion

Dolphin fits teams that transcribe Mandarin and other East, South, Southeast Asian and Middle Eastern languages and need dialect coverage or hotword biasing in the same stack. It does not fit translation work, and it is not a substitute for a model with previous-text conditioning. Before deploying, check the availability of the exact checkpoint you want against the model table, since the medium and large rows carry no public release, and confirm that your audio path produces WAV input, because the tool requires FFmpeg for the conversion.

Frequently asked questions

Official sources

1. DataoceanAI/Dolphin on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases