TheWhisper: TheStageAI's Whisper Build for Streaming and On-Device Speech-to-Text
Optimized Whisper models for streaming and on-device use
At a glance
- What is it?
- TheWhisper packages fine-tuned Whisper weights with CoreML engines for Apple Silicon and TensorRT engines for NVIDIA GPUs, plus a streaming pipeline. The install is the hard part, and one accelerator path depends on a paid platform account.
- Who is it for?
- Adopt TheWhisper if you are targeting Apple Silicon or a listed NVIDIA GPU and you want chunked streaming transcription without writing your own VAD and buffer logic; the CoreML path is the one that needs no external platform account. Do not adopt it if your hardware is not in the support matrix, if you are on Python 3.13, or if you need a documented rollback story for the stage-optimized engines, because the README does not describe one.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 97 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What TheWhisper actually is, and who it is for
TheWhisper is a Python package, thestage-speechkit, that wraps Whisper speech recognition for two kinds of deployment: a desktop or edge device with an Apple Silicon chip, and a server or workstation with a specific NVIDIA GPU. The repository also publishes fine-tuned Whisper weights on Hugging Face, and those weights are the part that differs from stock Whisper. The README states that the fine-tuned models accept chunk lengths of 10, 15, 20 and 30 seconds, whereas the original models are fixed at 30 seconds. That single change is what makes streaming plausible: a pipeline can emit partial text every ten seconds instead of waiting for a full 30-second window.
The intended audience is narrow and identifiable. You are building real-time captioning, a live meeting transcriber, a voice interface, or an on-device note taker, and you care about latency and power draw rather than about squeezing the last point of word error rate out of a research checkpoint. The README names those cases directly. If you only need to transcribe a folder of files once a day on a CPU box, this is more machinery than the job requires.
Two engines, two platforms, one Python surface
The architecture splits at the accelerator. On Apple hardware the package uses CoreML engines; on NVIDIA hardware it uses TensorRT engines obtained through TheStage AI's ElasticModels. The README gives a figure of roughly 2W of power consumption and about 2GB of RAM for the CoreML path, and 220 tok/s on an L40s for whisper-large-v3 on the NVIDIA path. Those are the project's own numbers, published in its README, not measurements taken here.
The Python surface hides the split behind one class. `ASRPipeline` handles file input and returns text with optional word timestamps. `StreamingPipeline` handles a live source, and the README shows it paired with `MicStream`, `FileStream` and `StdoutStream` from `thestage_speechkit.streaming`. A `platform` argument selects the backend, and `model_size` selects a size variant of the fine-tuned model. The repository layout reflects this: `thestage_speechkit/` holds the package, `examples/` holds runnable scripts including `run_apple_asr.py`, `run_nvidia_asr.py`, `run_streaming.py` and `server.py`, and `asr_postprocess/` holds text cleanup code.
The package dependencies tell you what the streaming layer is doing under the hood. `silero-vad` is a voice activity detector, `librosa` handles audio loading, `sounddevice` captures a microphone, and `fastapi` plus `uvicorn` back the local REST API and the Electron frontend example. So the streaming path is not a single model call; it is VAD-gated chunking feeding a model that was fine-tuned to accept those shorter chunks.
Installing TheWhisper: clone, then pick an extra
The README's Quick Start is a clone followed by an optional-dependency install. The extras are defined in `pyproject.toml`, so the names below are the real ones.
git clone https://github.com/TheStageAI/TheWhisper.git
cd TheWhisperFor Apple Silicon, the extra pulls torch 2.7.0, torchaudio 2.7.0, transformers 4.52.3, mlx 0.25.2 and coremltools 8.3.0. Expect a large download.
pip install .[apple]For NVIDIA, the extra pins torch 2.9.0 and the TensorRT 10.13.3.9 packages instead.
pip install .[nvidia]The third path is the one that needs care. To use TheStage AI's optimized engines rather than the plain NVIDIA extra, you install `thestage-elastic-models` from a private index, then the package, then the `thestage` CLI, and you register an API token generated on the TheStage AI platform.
pip install 'thestage-elastic-models[nvidia]==0.1.7' --index-url https://thestage.jfrog.io/artifactory/api/pypi/pypi-thestage-ai-production/simple --extra-index-url https://pypi.nvidia.com --extra-index-url https://pypi.org/simple
pip install .[nvidia]
pip install thestage
thestage config set -t <YOUR_API_TOKEN>A first real use, on Apple, is the README's own example: construct `ASRPipeline` with the Hugging Face model id `TheStageAI/thewhisper-large-v3-turbo`, `model_size='S'` and `chunk_length_s=10`, call it on a WAV path with `return_timestamps="word"`, and read `result["text"]`. The README does not spell out what `model_size='S'` means in terms of accuracy, and that is a gap worth noting before you ship it.
Where the support matrix stops
The requirements table is explicit, and explicit tables are usually honest about their boundaries. On NVIDIA the supported list is RTX 4090, RTX 5090, L40s, H100, A100 and Jetson-Thor, on Ubuntu 20.04 or later, with CUDA 11.8 or higher, driver 520.0 or higher, and Python 3.10 to 3.12. A T4, a V100 or a consumer card outside that list is not covered. On Apple the supported chipsets are M1 through M4 in their Pro, Max and Ultra variants, on macOS 15.0 or later or iOS 18.0 or later, with Python 3.10 to 3.12.
Two constraints follow. First, Python 3.13 and 3.14 are outside the stated range even though `pyproject.toml` declares `requires-python = ">=3.9"`. The packaging metadata and the support matrix disagree, and the matrix is the one describing tested hardware. Second, the Jetson-Thor install requires `tensorrt==10.13.3.9` to already be present on the device, and it installs from a Jetson-specific index, so that path is not interchangeable with the desktop NVIDIA path.
The optimized-engine route has a non-technical dependency. It requires an account on TheStage AI's platform and a token set through `thestage config set`. The README describes ElasticModels as free for small organizations but does not define the threshold. If you cannot accept a third-party account in your deployment chain, the CoreML path and the plain `[nvidia]` extra are the routes that avoid it.
The licence has two layers, and only one is MIT
The repository is MIT licensed, and `pyproject.toml` declares `license = { text = "MIT" }`. That covers the code you clone. The README also carries a section titled Enterprise License Summary, which means there are terms beyond the MIT grant for some users. The README does not reproduce those terms in the sections available here, so the only responsible statement is that the summary exists and should be read before commercial deployment.
The weights are a separate question from the code. They are published on Hugging Face under `TheStageAI/thewhisper-large-v3-turbo`, and model weights carry their own terms independent of the repository licence. The README does not state what those terms are. If you are shipping a product, check the model card on Hugging Face as well as the repository LICENSE, and treat the Enterprise License Summary as the document that decides whether your organization owes anything. This is a description of what the repository says, not legal advice.
Maintenance is visible but should be judged by dates rather than adjectives. The repository is not archived, and the last push was on 2026-06-15. No releases were retrieved for this project, so version pinning in `pyproject.toml` (torch 2.7.0 on Apple, torch 2.9.0 and TensorRT 10.13.3.9 on NVIDIA) is currently the main signal of what the maintainers consider a working combination.
Faster-whisper, and what changes if you pick it
The obvious comparison is faster-whisper, a CTranslate2 reimplementation of Whisper that most people reach for when they want Whisper to run faster than the reference implementation. The difference in approach is where the speed comes from. faster-whisper rewrites the inference path in a different runtime and keeps the standard 30-second Whisper window; streaming on top of it is something you build, typically by pairing it with a VAD and a rolling buffer.
TheWhisper instead fine-tunes the model to accept 10, 15 and 20 second chunks, then ships accelerator-specific engines (CoreML, TensorRT) and a streaming pipeline that already includes `silero-vad`. That is a different bargain: you get a shorter path to a working streaming service, and you give up portability, because the engines are tied to the hardware list in the support matrix. faster-whisper runs on a much wider range of hardware, including plain CPUs, at the cost of assembling the streaming layer yourself.
A second reference point is the upstream openai/whisper package, which is the baseline both projects are measured against. It offers no streaming pipeline and no accelerator-specific compilation, and its models are fixed at 30-second windows. If your workload is batch transcription on a machine that is not in TheWhisper's matrix, upstream Whisper or faster-whisper is the correct choice and TheWhisper is the wrong one.
What to check before you depend on it
The README documents the happy path well and the operational edges poorly. There is no rollback procedure for the stage-optimized engines, no statement about what happens when the TheStage AI platform is unreachable during a `thestage config set` or a model fetch, and no description of how a streaming session behaves when the audio source stalls mid-chunk. The `model_size` parameter is used in every example without an explanation of what the size variants cost in accuracy.
The benchmark directory is where accuracy claims live. The README points to `benchmark/README.md` for latency, memory, power and OpenASR accuracy results, and that is the file to read rather than the marketing figures in the overview. The 220 tok/s and 2W numbers are project-published figures for specific hardware; treat them as the maintainers' measurements, not as guarantees for your machine.
One practical check that the README does support: the repository ships `examples/server.py`, which uses FastAPI and uvicorn, and `electron_app/README.md` for the desktop frontend. If you intend to self-host, those two files define the shape of the service you would be running, and reading them takes less time than discovering the constraints at deploy time.
Editorial conclusion
Adopt TheWhisper if you are targeting Apple Silicon or a listed NVIDIA GPU and you want chunked streaming transcription without writing your own VAD and buffer logic; the CoreML path is the one that needs no external platform account. Do not adopt it if your hardware is not in the support matrix, if you are on Python 3.13, or if you need a documented rollback story for the stage-optimized engines, because the README does not describe one. Before committing, verify three things: that pip install .[apple] resolves on your macOS and Python combination, that TheStageAI/thewhisper-large-v3-turbo loads through ASRPipeline, and that your intended use falls under the MIT grant rather than the Enterprise License Summary.
Frequently asked questions
What is TheWhisper used for?
It is used for low-latency streaming and on-device speech-to-text, including real-time captioning, live meetings, voice interfaces and edge deployments. The README also points to a local REST API with JS and Electron frontend examples for building a macOS note-taking app.
How do I install TheWhisper on Apple Silicon?
Clone the repository, then run pip install .[apple] from inside it. That extra installs torch 2.7.0, torchaudio 2.7.0, transformers 4.52.3, mlx 0.25.2 and coremltools 8.3.0, and the README lists M1 through M4 chipsets on macOS 15.0 or later as supported.
Does TheWhisper need a TheStage AI account?
Only for the optimized ElasticModels engines on NVIDIA and Jetson-Thor. That path requires installing thestage-elastic-models, then running thestage config set -t with a token generated on TheStage AI's platform. The Apple CoreML path and the plain [nvidia] extra do not involve that step.
Which Python versions does TheWhisper support?
The support matrix states Python 3.10 to 3.12 for both the NVIDIA and Apple requirements. The packaging metadata in pyproject.toml declares requires-python >= 3.9, which is broader than the matrix, so the matrix is the safer guide.
What chunk lengths does TheWhisper support for streaming?
The fine-tuned models accept 10s, 15s, 20s and 30s chunk modes, and the support matrix marks all four as available on both whisper-large-v3 and whisper-large-v3-turbo, on NVIDIA and Apple. Stock Whisper models are fixed at 30 seconds.
Community notes