qwen-asr: running Qwen3-ASR speech-to-text in plain C

What the project is: an inference engine, not a model port

The repository implements the inference pipeline for Qwen3-ASR speech-to-text models, both the 0.6B and the 1.7B, in C. That framing matters: this is not a reimplementation of the model, it is the code that runs someone else's weights.

The dependency situation is the headline. The CPU build has zero external dependencies beyond the C standard library and a BLAS implementation, Accelerate on macOS and OpenBLAS on Linux. Optional GPU backends accelerate inference on AMD GPUs with ROCm, built with make rocm, and on NVIDIA GPUs with CUDA, built with make cuda. Tokens stream to stdout as they are generated.

The performance claim in the README is that the implementation runs at a speed multiple of the file length even on modest hardware, such as a low end Intel or AMD processor. For a speech-to-text engine written in C with no Python runtime and no framework, that is the property that matters, because transcription is a batch or pipeline job where a per-request overhead of hundreds of milliseconds is often the difference between a usable service and an unusable one.

The project is MIT licensed, the primary language is C, the last push was on 2026-10-01, and it publishes no GitHub releases. The repository tree shows how the code is organised: main.c as the entry point, qwen_asr.c with its header, then a separate file per stage, qwen_asr_encoder.c, qwen_asr_decoder.c, qwen_asr_tokenizer.c, qwen_asr_audio.c and qwen_asr_safetensors.c, plus qwen_asr_kernels.c and a set of kernel implementations, a generic one, an AVX one and a NEON one, with a shared kernels_impl.h. The cuda/ and rocm/ directories hold the GPU backends, and samples/ holds test audio.

The MPS decision, and why it is stated so forcefully

The README has an important paragraph that is worth reading closely, because it is unusual for a project to tell contributors not to send pull requests about a feature.

The implementation explicitly avoids implementing support for MPS. The reasoning given is that transcription systems are very important pieces of infrastructure, and are often run on remote Linux servers, so adding the MPS target would focus the efforts too much on Apple hardware. The code runs very well on Apple hardware anyway, because it is NEON optimized. The instruction to contributors is to not send pull requests about the feature but to fork the code instead, in order to add MPS support, and the author says he will add it much later when the other optimizations are mature.

This is a scope decision made in public and defended on architectural grounds, and it is the kind of thing that is easy to get wrong in a different way. A project that accumulates every platform target ends up with four backends, each half-maintained, and the CPU path becomes the one nobody tests. By refusing one target outright, the author keeps the CPU path the reference implementation and the two GPU backends the ones with actual hardware behind them.

The trade-off is real and stated. If you are on Apple silicon and want Metal acceleration, the answer is that there is none, and the CPU path with NEON kernels is what you get. For most transcription workloads that is fine, because the CPU path is already faster than real time.

It is also a good illustration of the general shape of this repository. The decisions are made once, explained, and then defended against well-meant contributions.

Quick start: build, fetch a model, transcribe

The quick start is short enough to reproduce from the README alone:

# Build
make blas

# Download a model (interactive selector: small=0.6B, large=1.7B)
./download_model.sh

# Transcribe audio (tokens stream to stdout as generated)
./qwen_asr -d qwen3-asr-0.6b -i audio.wav

# Pipe any format via ffmpeg
ffmpeg -i audio.mp3 -f s16le -ar 16000 -ac 1 - 2>/dev/null | \
    ./qwen_asr -d qwen3-asr-0.6b --stdin

# Streaming mode (incremental output for live audio)
./qwen_asr -d qwen3-asr-0.6b -i long_recording.wav --stream

The build is make blas, which produces the CPU binary. The model comes from download_model.sh, which runs an interactive selector choosing between the small 0.6B and the large 1.7B, so you pick the weights at download time rather than discovering the difference later. Transcription then takes the model directory with -d and the input file with -i.

The ffmpeg pipeline is the part that makes this usable in a real pipeline. Audio can be piped from stdin with --stdin, making it easy to transcode and transcribe any format via ffmpeg, and the command in the README resamples to 16-bit signed little endian at 16kHz mono, which is what the model expects.

Two input formats are supported. WAV input handles 16-bit PCM files at any sample rate, auto-resampled to 16kHz, and stdin input does auto-detection between a WAV header and raw s16le 16kHz mono. That means the ffmpeg pipe and a WAV file on disk take the same code path, which is the sort of detail that determines whether a tool fits into an existing audio pipeline.

The streaming invocation at the end of the block uses --stream on a long recording, and the distinction between that and offline mode is the central design question in the project.

Offline versus streaming, and the segment size decision

Both normal, offline, and streaming, online, modes are supported, and the defaults differ by build. The CPU build defaults to full offline decode with -S 0, so the whole audio is encoded at once. The ROCm and CUDA builds default to 20-second segments with -S 20, to bound GPU attention and memory use, and -S 0 explicitly enables full-audio decode on those backends too.

Streaming mode processes audio in 2-second chunks with prefix rollback, keeping the last few decoded tokens as context for the decoder and the language model when transcribing the next chunk, and a sliding window bounds encoder and decoder context for indefinite streaming.

The README is candid about the cost of streaming, and it is a warning worth heeding. In this implementation, interactive --stream prioritises incremental token stability over throughput, and can be much slower than normal mode when you process an already-recorded file end to end. That is a sensible trade for live audio, where you want stable text rather than fast text, and a poor trade for a file you could have decoded in one pass.

The mode guidance is given by file length. Up to about 60 seconds, try -S 0, the CPU default, if full-context quality and speed are acceptable, and prefer -S 20 if full-context decoding produces empty or incomplete output. For large prerecorded files, use segmented offline mode such as -S 20 or -S 30 or more. For long live or continuous audio, or low-latency user interface needs, use --stream. For batch and offline file transcription, prefer -S 20 or -S 30, since it is usually much faster than interactive --stream.

Two boundary problems are called out with fixes. If segmented output drops or warps around boundaries, try a different segment size and keep the default --past-text auto. If you want stronger continuity across segments or chunks, try --past-text yes, which can help continuity and can also cause drift on some files.

The underlying reason for segmenting at all is stated plainly: for long files, memory and time grow with sequence length, and for very long files the decoder cost still grows with sequence history even in segmented mode, which is why --stream exists for the indefinite case.

Feature details that matter in production: monitoring, silence, memory mapping

A handful of the feature list items are less obvious than the rest and are worth pulling out for anyone running this as a service.

Monitor mode, --monitor, shows inline Unicode symbols on stderr for real-time pipeline diagnostics, and the normal and debug output modes give you a middle ground. By default timing information is printed to stderr with an Inference line and an Audio line showing the realtime factor:

./qwen_asr -d qwen3-asr-0.6b -i audio.wav --silent    # no stderr output
./qwen_asr -d qwen3-asr-0.6b -i audio.wav --debug      # per-layer/per-chunk details

So a run tells you both how long the inference took and how many times faster than real time it was, which is the number you need to size a deployment.

Optional silence skipping, --skip-silence, drops long silent spans before inference and is off by default, with the note that it may use less CPU for the same file. That is a cheap win on recorded audio with dead air, and the fact that it is off by default is the right call because on some recordings the detection itself is not worth the CPU.

Memory-mapped weights are the other one. CPU inference accesses decoder BF16 weights directly from safetensors files, while the GPU backends convert and upload weights at load time. So on CPU, loading a 1.7B model does not mean reading gigabytes into RAM, and the process start cost is closer to a page fault than to a copy.

Prompt biasing, --prompt, injects a system prompt to bias the model toward specific terms or spellings, with an unusually honest caveat: prompt biasing is very soft, the models may or may not care about your instructions, and usually spelling instructions are followed decently. Language is usually auto-detected from the audio and can be forced with --language.

Optional segment splitting is also configurable, with -S 20 or -S 30 for large files and -W 3 for the segment-cutting silence search.

The build system, and how the GPU backends are kept separate

The Makefile in the repository root is where the engineering layout becomes clear. The CPU build uses gcc with -Wall -Wextra -O3 -march=native -ffast-math, links against libm and libpthread, and compiles a source list that includes the generic, NEON and AVX kernel files alongside the core modules. The debug build adds -g -O0 -DDEBUG and the address sanitizer, and there is a separate target for it.

The two GPU backends are built in a way that lets them coexist with the CPU one, and the comment in the Makefile says so directly. The ROCm objects are compiled into a per-architecture directory, build/rocm/$(GPU_ARCH), with the architecture defaulting to gfx1151, and they link against rocblas with an rpath into the ROCm install, and the target binary is a separate qwen_asr_rocm. The CUDA objects go into build/cuda/$(CUDA_ARCH) with the architecture defaulting to sm_121, and the comment notes that sm_121 is the GB10 GPU in DGX Spark. That links against cuBLAS the same way, with the target qwen_asr_cuda.

So the CPU binary, the ROCm binary and the CUDA binary are three separate artefacts from the same source list, differing in the object directory and the BLAS library, and each has its own kernel-test target.

The repository also carries asr_regression.py, a Python script, and python_simple_implementation.py, alongside a tests/ directory. The Python file is a reference implementation to check the C against, which is a sensible way to build a port: write the simple version first, then compare.

The model documentation lives in MODEL.md, MODEL_CARD_OFFICIAL.md and AGENT.md and CLAUDE.md, and the samples directory holds jfk.wav with its expected jfk.txt, a test_speech pair and a night_of_the_living_dead_1968 directory, so a regression run has something to compare against.

Which model, and what to expect from the 0.6B

Both parameter sizes are supported, and the implementation detects which one it has from the weight files, so the -d argument and the actual checkpoint have to agree. The README's guidance is that while the 1.7B model is generally more powerful, the 0.6B model seems to be the sweet spot for CPU inference, and that the speed difference is not huge, so you may want to try both and decide depending on your use case.

That is a useful piece of advice precisely because it is unmeasured. The claim in the abstract of this project is not that one model is better, but that the implementation runs at a speed multiple of the file length even on modest hardware, and the 0.6B versus 1.7B choice sits inside that. If you are on a low end processor, the smaller model with a full-context decode is probably the configuration that keeps you comfortably faster than real time,.

The zero-dependency CPU build is the part that changes the deployment conversation. There is no Python, no container image to maintain, no CUDA toolkit on the machine if you do not want one, and the binary is one file. For a transcription service that has to run on many small machines, or inside a locked-down environment, that is a different operational profile from a Python inference stack.

The cost of that profile is that you own the model weights, the BLAS library, and the build. The download script handles the weights, and the make targets handle the rest, but the dependency on a BLAS implementation being present is the one thing you have to arrange yourself, and on Linux that means OpenBLAS rather than a system default.

Practical advice for trying it, and what this project is not

It is worth being explicit about the boundaries, because the name and the language make a couple of assumptions easy to form.

This is not a training framework. There is no fine-tuning path here, no dataset pipeline for speech, and no model weights in the repository. The weights come from the Qwen3-ASR project and the download script fetches them. The Python file in the tree is a reference implementation for comparison, and the regression script is for checking behaviour, not for producing models.

It is also not a Python package. The primary language is C, the build produces a command line binary, and the output is tokens on stdout. There is no binding here for the Python ecosystem, and the project description says C inference, which is the honest scope. If you want to call this from a Python service you are writing the binding.

And it is not a general audio tool. It takes WAV or raw PCM and produces a transcript. The ffmpeg pipe in the README is a convenience for getting arbitrary formats in, not a feature of the tool.

What it is, then, is a small, fast, dependency-light inference engine for two specific speech-to-text models, with a CPU path that runs anywhere a C compiler and a BLAS library exist, and with GPU backends for two specific vendors. The author's stated approach, visible in the MPS refusal and in the careful separation of the GPU object directories from the CPU ones, is to keep the thing small and the scope honest, and the repository delivers on that in a way that is easy to miss until you read the Makefile.

A few things to do in order will save time.

Build the CPU binary with make blas, then run download_model.sh and pick the 0.6B for a first pass, since the README calls it the CPU sweet spot. Transcribe the shipped sample first, samples/jfk.wav, and compare against samples/jfk.txt, because a working baseline on a known file tells you whether the build is sound before you point it at your own recordings.

For real recordings, use the ffmpeg pipe rather than converting files yourself, since the resampling to 16kHz mono 16-bit is exactly what the model wants and the pipe does it for you.

Then pick the mode based on length rather than by habit. Short clips under a minute, use -S 0. Large prerecorded files, use -S 20 or -S 30 and expect the output to be emitted per segment. Live or indefinite audio, use --stream and accept the throughput cost. If the output looks broken at segment boundaries, change the segment size before you change anything else, and leave --past-text on auto unless continuity is more important than drift for your recordings.

Finally, measure. The stderr output gives you the inference time and the realtime factor, so you can size a deployment from your own audio rather than from the general claim in the README.

The one thing to keep in mind throughout is that this is a personal project with a single author and no tagged releases, so the API is the source of truth rather than a published version, and the MPS path simply does not exist if you are on Apple silicon and want Metal.

Editorial conclusion

Adopt qwen-asr if you need speech-to-text throughput you cannot get from a Python stack, whether that means running on machines where a Python runtime is unwelcome, fitting the model on modest CPU hardware, or keeping the whole pipeline in one native binary with no container to maintain. Do not adopt it for training, for a Python integration without writing your own binding, or on Apple silicon expecting Metal acceleration, because the author has ruled MPS out of scope and asks for forks rather than pull requests. Before you commit, build with make blas, take the 0.6B weights from download_model.sh, and check the shipped jfk sample against its expected transcript, then choose your segment size from the length of your actual audio rather than from a default.

Frequently asked questions

Official sources

1. antirez/qwen-asr on GitHub
2. Issues
3. License: MIT
4. README