Confucius4-R2T2: a streaming speech model that never revises what it said

Append-only output is the contract, not a detail

Confucius4-R2T2 is described as a low-latency and high-accuracy true streaming automatic speech recognition model, and the phrase that should decide whether you use it is append-only output mode: committing transcript text permanently without revising previous words.

Almost every streaming recogniser works by emitting a hypothesis and then revising it as more audio arrives. Early words get changed, sometimes several times, and the text you saw a moment ago is not the text you have now. For a live captioning display that means flickering text. For a downstream natural language pipeline that consumes tokens as they appear, it means a parser has to cope with words that change after they were acted on.

The README is explicit that this matters for applications where text must be processed or acted upon instantly, and names the cases: Real-Time Live Captioning and Subtitling, downstream NLP pipelines and LLM agents, and simultaneous speech translation. The result it claims is a smoother user experience that avoids disruptive text revisions and visual flickering.

The mechanism that makes the promise safe is a learning paradigm rather than a decoding trick. R2T2, short for Real Real-Time Transcription, is built upon the Qwen3-ASR model and trained with what the documentation calls a unique set of data construction techniques: stable-prefix data, forced time-alignment data and token-level audio segmentation. Combined with a Longest Stable Prefix learning paradigm, the model can dynamically determine when a stable prefix can be safely emitted and when more audio context is needed. By exposing only stable prefixes, the model provides high-quality context that conditions later predictions while guaranteeing that already emitted text does not change.

The claimed latency is 200 to 600 milliseconds on average, with decoding chunks configurable from 80 ms to 2 s.

The audience is anyone whose downstream consumer cannot tolerate retractions.

The package wraps qwen_asr and adds streaming on top

The repository structure tells you how the code is organised, and the packaging metadata is unusually explicit about it.

There is an r2t2/ directory for the Python package and a separate r2t2_llama/ directory for the llama.cpp backend, plus example.py, ws_server.py and ws_client.py for the WebSocket path, and two shell scripts, run_example.sh and run_start_server.sh. The r2t2 package is the primary entry point, and the llama.cpp directory is an alternative backend.

The pyproject.toml describes the relationship in its own comments. It says Confucius4-R2T2 is a multilingual streaming and one-shot ASR model built on the Qwen3-ASR architecture, and that the file describes how to build, install and distribute the accompanying r2t2 Python package, which wraps the upstream qwen_asr architecture and adds streaming inference on top.

That last clause is the key architectural fact. This is not a from-scratch model implementation. It is a wrapper around an existing Qwen3 ASR implementation with a streaming layer added, and the project keeps the model weights and the inference code together so you can run both.

The package metadata is conventional: the distribution is named confucius4-r2t2 at version 0.1.0, it requires Python 3.10 or newer, and the keywords list names the domain accurately with asr, speech-recognition, streaming, qwen3-asr, vllm and multilingual among them. The build backend is setuptools with a wheel requirement, and the development status is marked as Beta.

The classifiers also narrow the platform. The operating system classifier is POSIX Linux only, and the Python classifiers cover 3.10 to 3.12.

There are two licence files in the repository, and they are not the same thing, which is the single most important administrative detail in this project.

Two licences: Apache 2.0 for the code, something separate for the weights

The README carries a badge pointing at a code licence of Apache 2.0 and links to the LICENSE file, and the pyproject metadata declares the same thing with a license field of Apache-2.0 and the OSI Approved Apache Software License classifier.

But the repository root holds a second pair of files, MODEL_LICENSE and MODEL_LICENSE_zh, and the project metadata records the repository licence as a custom notice that GitHub cannot classify. So the code and the model weights are licensed separately, and the code licence does not automatically cover the weights.

That distinction is standard practice in the model release world, and it is worth understanding before you deploy. The weights are the part with terms attached: a model licence typically restricts how the model can be used, sometimes restricts commercial use, and often specifies attribution and redistribution conditions. The code is the part you can usually vendor, modify and ship inside your own application without asking.

The Chinese-language version of the model licence file, MODEL_LICENSE_zh, is a duplicate of the same terms in another language, which is the usual reason to ship a translated licence alongside the original.

For an organisation building a product on this model, the model licence is the document to read, and it is the one that the README's Apache badge does not describe. Read MODEL_LICENSE before you commit to a deployment, and treat the distinction between the code licence and the weights licence as a legal review item rather than a detail.

The repository has no GitHub releases, so the model checkpoints are distributed through the linked Hugging Face and ModelScope pages rather than as release assets, and the last push to the repository was on 2026-09-24.

Two backends, and the choice is about throughput versus portability

The documentation lists a vLLM backend for high-throughput inference, with a Hugging Face transformers backend also available, and the table of contents adds a llama.cpp backend alongside them. The Python API section is split into offline transcription and streaming transcription under the vLLM backend, with the llama.cpp backend as its own entry point.

That is three ways to run the same model, and they are not equivalent.

The vLLM backend is the one the project leads with, and it is aimed at serving. The vLLM path is the natural choice when the model runs on a GPU server and several streams or requests arrive at once, because vLLM's continuous batching is what makes the throughput claim meaningful. It is also the path with the heaviest memory requirement, since the weights sit in GPU memory.

The transformers backend is the fallback for anyone who does not want to run a vLLM server, at the cost of that throughput.

The llama.cpp backend, in its own r2t2_llama directory, is the portability option. It is the one that would let you run the model on hardware without the vLLM stack, and it is the path most likely to matter for edge or CPU deployments, though the repository documentation does not say what precision or performance to expect from it.

The README also lists context and hotword prompts as natively supported, and multilingual coverage optimised for Chinese and English while also supporting a broad range of additional languages. A supported languages section in the table of contents suggests a fuller list is documented further down than the excerpt available here.

The evaluation sections cover streaming performance and accuracy, with separate English and Chinese accuracy tables, and there is a note addressed to authors and maintainers of models included in the comparisons, which is a good sign about how the benchmark numbers are handled.

Installing the package and running the example

The packaging file is the best installation guide available, because its comments spell out the two install forms at the top.

  pip install .                # install the package and vLLM backend
  pip install -e .             # editable install with the vLLM backend

Those two lines are the summary of what the Docker path is for. The vLLM backend comes with the install, so a working environment is a pip install rather than a stack of individual components. The editable form is for working on the r2t2 package itself, which is what a contributor wants, since the package name in the metadata is confucius4-r2t2 and the version is 0.1.0.

The interpreter floor is visible in the same file.

requires-python = ">=3.10"

The documented path is to clone the repository, then set up the environment with either conda or uv, and the Docker route is the one the documentation recommends. The Docker steps are three: start a container, run the example inside the container, and manage the container.

Once the environment exists, the first thing to do is run example.py through run_example.sh, and then start the WebSocket service with run_start_server.sh if you want to consume the stream from another process. The weights themselves are not in the repository, they are on the linked Hugging Face and ModelScope pages, so a first run needs either a model download step or a mounted checkpoint directory.

A WebSocket server for anyone who does not want to write Python

The repository ships a WebSocket server, with a start and stop section, a WebSocket endpoint description, a message format section and an example client. The shell script run_start_server.sh is the wrapper for starting it, and ws_client.py is a ready-made client.

That is a meaningful part of the design for a streaming model, because the whole point of append-only output is that a consumer acts on partial text. A WebSocket server makes that consumable from a browser, a mobile client or another service without the consumer having to embed the Python package. The message format section in the documentation means the protocol is defined rather than improvised, which is what you need if you are going to write your own client.

The example client is useful for a second reason: it is the fastest way to see the streaming behaviour without wiring up a model client of your own.

The entry points are named after what they do. example.py is the minimal usage example the repository description refers to, run_example.sh runs it, and run_start_server.sh starts the WebSocket service.

The Docker path in the documentation is marked as recommended, with three steps: start a container, run the example inside the container, and manage the container. Given that the model weights are large and the vLLM backend has a real memory profile, containerising the environment is a reasonable default rather than a convenience, since it keeps the GPU driver and CUDA configuration consistent between your laptop and your server.

Where Confucius4-R2T2 is the wrong tool

The first wrong-tool case is a batch transcription job. If you have a recording and you want a transcript, the streaming constraint buys you nothing and the append-only promise actively costs you something, because the model will not go back and correct an early misrecognition. A conventional offline recogniser optimises for final accuracy; this model optimises for committing text safely and early. The README claims no loss in offline accuracy, and that adding streaming support does not degrade offline recognition, but even so the streaming-oriented training is a different objective.

The second case is a downstream consumer that wants to revise. If your pipeline would rather wait for a stable final transcript than act on partial text, the append-only mode removes an option you might have wanted, and you would be paying for a property you do not use.

The third case is a deployment on anything other than a GPU machine with substantial memory. The vLLM backend is the recommended path and it wants real memory, the transformers backend is the lighter but slower fallback, and the llama.cpp backend exists but is documented separately with its own directory, so you should check what it supports before assuming it is equivalent.

There is also a scale question that the streaming model does not answer. The README's claims are about latency and accuracy per stream. Whether the serving stack sustains that latency when several concurrent streams arrive is a capacity question that depends on your hardware and your batching configuration, and the repository does not state figures for concurrent throughput.

One more limitation is worth flagging plainly. The model is built on the Qwen3-ASR architecture, and the Qwen3-ASR tech report referenced in the README is described as something that will be released soon. A significant part of the model's training methodology is therefore not yet documented in the places you would look for it.

Editorial conclusion

Confucius4-R2T2 suits a team that needs subtitles, live captioning or an LLM agent fed by speech, and where revising already-emitted text would break the downstream consumer, since the append-only contract is the point of the model. It does not suit a project that needs a single offline transcript of a long recording with no streaming constraint, where a conventional offline recogniser is simpler. Before adopting, read the model licence in MODEL_LICENSE separately from the code licence in LICENSE, start from the Docker path the documentation recommends, and confirm that your hardware meets the memory needs of the vLLM backend before committing to the 2 s chunk setting.

Frequently asked questions

Official sources

1. Issues
2. netease-youdao/Confucius4-R2T2 on GitHub
3. README