# onnx-asr: running Parakeet, Canary and Whisper speech recognition without PyTorch

> onnx-asr is a pure Python package that loads ONNX speech recognition models and supplies the preprocessing and greedy decoding they omit. It installs with pip, keeps NumPy and onnxruntime as its only required dependencies, and caps most models at 20 to 30 seconds of audio unless you add VAD.

**istupakov/onnx-asr** — A lightweight Python package for Automatic Speech Recognition using ONNX models

- Repository: https://github.com/istupakov/onnx-asr
- Website: https://istupakov.github.io/onnx-asr/
- Stars: 383 · Forks: 35
- Language: Python
- License: MIT
- Published: 2026-09-17 · Updated: 2026-09-17 · Language: en
- Canonical page: https://hysenlabs.com/projects/istupakov-onnx-asr

## The gap onnx-asr fills between an ONNX export and a working transcript

Exporting an ASR model to ONNX usually leaves you with an encoder and a decoder and nothing else. The README makes this explicit: when these models are saved in ONNX format, typically only the encoder and decoder are included, and running them requires the corresponding preprocessing and decoding implementations. That missing half is log-mel spectrogram feature extraction on the way in and greedy search decoding on the way out, plus the file reading and resampling that turn a WAV path into a NumPy array. onnx-asr supplies those pieces for each architecture it supports, so the ONNX graph becomes usable without you reimplementing a feature extractor from the original training code. The intended reader is a Python developer who already has a model in mind and wants inference, not a training pipeline. The package description calls it lightweight, and the dependency list backs that up: numpy and typing-extensions are the only hard requirements in pyproject.toml, with onnxruntime, onnxruntime-gpu and huggingface-hub moved into the cpu, gpu and hub extras. No PyTorch, no Transformers, no FFmpeg.

## Architecture support: NeMo, GigaAM, Zipformer, Wav2Vec2 and Whisper behind one load_model call

The supported list is broader than the package name suggests. It covers NVIDIA NeMo Conformer, FastConformer, Parakeet and Canary with CTC, RNN-T, TDT and Transformer decoders; GigaChat GigaAM v2, v3 and Multilingual with CTC and RNN-T decoders including E2E versions; Kaldi Icefall Zipformer with a stateless RNN-T decoder, which the README notes includes Alpha Cephei Vosk 0.52 and later; T-Tech T-one with a CTC decoder; HuggingFace Wav2Vec2 with CTC; and OpenAI Whisper. Each of those needs its own preprocessor and decoder, and that per-architecture work is the substance of the package rather than a thin wrapper over onnxruntime. The README also points to a model comparison page for how these stack up against the original implementations, which is the honest place to look before assuming an ONNX export behaves identically to the PyTorch checkpoint it came from. One architectural limit is stated plainly: T-one streaming is not yet supported. If your application needs partial hypotheses while audio is still arriving, this package as documented does not provide that.

## Installing onnx-asr and transcribing a first WAV file

The README gives a single install line with two extras. The cpu extra pulls onnxruntime, and the hub extra pulls huggingface-hub so the model can be fetched from the Hugging Face Hub. Run it in a virtual environment, since the onnxruntime pins in pyproject.toml are version-specific:

```bash
pip install onnx-asr[cpu,hub]
```

Swapping cpu for gpu installs onnxruntime-gpu instead. The README's quick start then loads a model and recognizes a file. The comment in that example warns the download may take a few minutes, which is worth planning for on a first run:

```python
import onnx_asr

# Load the Parakeet TDT v3 model from Hugging Face (may take a few minutes)
model = onnx_asr.load_model("nemo-parakeet-tdt-0.6b-v3")

# Recognize speech and print the result
result = model.recognize("test.wav")
print(result)
```

What you should see is the transcript printed to stdout; the README does not show the exact return type of recognize here, so check the API reference before writing code that indexes into it. The package also reads NumPy arrays directly, does its own file reading and resampling, and ships a command-line interface documented in the usage guide if you would rather not write Python at all. Models can come from Hugging Face or from a local directory, including quantized versions.

## The 20 to 30 second ceiling and why VAD is not optional for real recordings

The README carries a warning that the maximum audio length for most models is 20 to 30 seconds, and that longer audio needs VAD, short for Voice Activity Detection. This is the constraint that decides whether the package fits your input. A meeting recording, a podcast episode or a call-center export will not go through a single recognize call on most of these architectures. The documented path is segmentation through VAD, which the usage guide covers under a dedicated VAD section. Treat that as part of the integration, not an afterthought: segment boundaries affect transcript quality, and the package's own handling of them is what you are relying on rather than a model that natively consumes long context. For short utterances, voice commands, or pre-chunked data, the ceiling never comes up. For anything archival, budget the VAD work before you evaluate accuracy.

## Where onnx-asr stops: streaming, custom models and community checkpoints

Three boundaries are worth stating before adoption. Streaming is the first, as noted above for T-one, and the package's design centers on recognizing a complete WAV file or array rather than an audio stream. The second is custom architectures: the README says custom models are supported through a conversion guide, but that guide is about converting a model into the layout onnx-asr expects, and it is a separate piece of work from loading a checkpoint someone else exported. The third is the community models page. The README says the Hugging Face community publishes additional onnx-asr-compatible fine-tunes and optimized exports, and then says community models are maintained by their publishers and may not be tested by this project. That sentence matters when you pick a language fine-tune for production. The project's own supported list is the tested surface; anything else is a publisher's promise. There is also a development-status signal in pyproject.toml: the classifier reads Development Status :: 4 - Beta.

## onnx-asr versus faster-whisper and the PyTorch NeMo path

The obvious comparison is faster-whisper, which also runs Whisper through a CTranslate2 runtime rather than PyTorch and is built for Whisper specifically. The difference in approach is breadth against depth. faster-whisper optimizes one model family end to end and has no interest in Parakeet, Canary or GigaAM. onnx-asr is model-agnostic within its supported list and pays for that by shipping a preprocessor and decoder per architecture, which is why the README devotes a section to explaining that ONNX exports arrive incomplete. If Whisper is the only model you will ever run, the narrower tool is the simpler dependency. The other comparison is running the original NeMo or GigaAM code with PyTorch. That path gives you the authors' own inference code and any feature it has, at the cost of a much larger install and a heavier runtime. onnx-asr exists to avoid that install. The README links a model comparison page against the original implementations, which is where you should look for any accuracy gap rather than assuming there is none.

## Licence, maintenance and the cost of keeping onnxruntime pinned

The package is MIT licensed, and pyproject.toml declares license = "MIT" with license-files = ["LICENSE"]. MIT is permissive, so redistributing it inside a commercial product is not the obstacle; the models are. Parakeet, Canary, GigaAM and Whisper checkpoints each carry their own terms from their publishers, and nothing in the repository changes those. Check the model licence separately from the package licence. On maintenance, the last push to the default branch was on 2026-08-16, and the most recent release listed is v0.12.0 on 2026-07-15, following v0.11.0 in March and v0.10.2 in January. The upgrade cost lives in the dependency pins. The cpu and gpu extras exclude specific onnxruntime versions: the constraints in pyproject.toml skip 1.24.1, all of 1.25.x and 1.26.0, and cap onnxruntime below 1.24 on Python versions earlier than 3.11. If your environment pins onnxruntime for another reason, those exclusions can conflict. NumPy is pinned at 1.22.4 or later, and the README states support for NumPy 1.22 through 2.4 and Python 3.10 through 3.14.

## Conclusion

onnx-asr suits Python teams that want to run Parakeet, Canary, GigaAM or Whisper checkpoints through onnxruntime without pulling in PyTorch, Transformers or FFmpeg, and it fits edge boxes as well as CUDA, TensorRT, CoreML, DirectML, ROCm and WebGPU targets. Skip it if you need streaming recognition, since the README states T-one streaming is not yet supported and the package is built around whole-file recognition with VAD segmentation for long audio. Before committing, verify two things yourself: that your audio fits the 20 to 30 second limit the README gives for most models, and that the model you want appears in the supported model names list rather than only in the community models page, which the project says may not be tested by it.

## FAQ

### What is the difference between ASR and TTS?

onnx-asr only does the ASR direction: it takes a WAV file or NumPy array and returns a transcript. The README describes the package as Automatic Speech Recognition using ONNX models, and nothing in it covers speech synthesis.

### Is ONNX faster than PyTorch?

The README publishes an Inverse Real-Time Factor table for onnx-asr across a 9800X3D CPU, a Cortex A53, a T4 with CUDA and an RTX 5070 Ti with TensorRT, and links a benchmarks page for detail. It does not compare those numbers against PyTorch implementations of the same models.

### What is an ASR AI model?

In this project it is an encoder and decoder pair exported to ONNX, which the README says typically excludes the preprocessing and decoding steps. onnx-asr supplies the log-mel spectrogram preprocessors and greedy search decoding those exports need.

### What is the relationship between ASR and NLP?

onnx-asr stops at the transcript. The README documents recognition, token-level timestamps and log probabilities, and does not describe any downstream language processing on the text.

## Sources

- [istupakov/onnx-asr on GitHub](https://github.com/istupakov/onnx-asr)
- [License: MIT](https://github.com/istupakov/onnx-asr/blob/main/LICENSE)
- [Project website](https://istupakov.github.io/onnx-asr/)
- [README](https://github.com/istupakov/onnx-asr/blob/main/README.md)
- [Releases](https://github.com/istupakov/onnx-asr/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/istupakov-onnx-asr