whisper vs whisper.cpp: one model, two runtimes
These are not two competing recognisers. whisper is a PyTorch model packaged for Python, and whisper.cpp is a dependency-free C and C++ implementation of that same model with its own weight format, so the real question is which runtime your deployment needs and whether you are willing to convert the weights. Both carry the same MIT licence, so nothing here is a licensing decision.
At a glance
| Project | openai/whisper | ggml-org/whisper.cpp |
|---|---|---|
| Licence | MITPermissive: commercial use allowed | MITPermissive: commercial use allowed |
| Maintenance | Commits in the last six monthsLast push August 31, 2026 | Commits in the last six monthsLast push October 11, 2026 |
| Language | Python | C++ |
| GitHub stars | 110,321 | 54,298 |
| Read more | Our analysisGitHub | Our analysisGitHub |
Which one to choose
Choose whisper if you are writing Python, you already have ffmpeg and PyTorch available, and you want the reference package with its model table, its language selection and its translation task in one place. It is the shortest path to a first transcript, and it is the right pick for a service that already runs PyTorch and wants the large models on a GPU.
Choose whisper.cpp if the target is a device, an app or a language your service does not run: Apple Silicon, iOS, Android, a Raspberry Pi, a browser, a C or C++ host, or a machine with no PyTorch at all. It is also the right pick when integer quantization is the only way to fit the model in a fixed memory budget, and when you need the property the README states directly, that it allocates nothing at run time.
Two runtimes for one set of weights
The most useful thing to establish first is that the two projects implement the same recogniser. OpenAI's whisper repository is the reference: a transformer sequence-to-sequence model trained jointly on multilingual speech recognition, speech translation, spoken language identification and voice activity detection, represented as one sequence of tokens with special tokens acting as task specifiers, which is what lets a single model replace several stages of a traditional pipeline. It ships as a Python package with a command line entry point.
whisper.cpp is a plain C and C++ implementation of that same model, with no dependencies, sitting on the ggml machine learning library. The README puts the whole high-level model in two files and describes the rest as library support, and the point of the rewrite is portability: the same weights running on an iPhone, a Raspberry Pi, a browser, or a machine with no Python installed.
Because the model is shared, the weights are shared too, in a different container. The Python side loads a PyTorch checkpoint; the C and C++ side loads a model converted to the ggml format, and the repository ships the bridge as a model download script plus make targets for each size, so converting is a normal documented step rather than an obstacle. That also means the accuracy of a given model size is a property of the model, and the two runtimes differ in speed, memory and platform coverage rather than in what they hear.
Getting each one running
Both paths need ffmpeg, and both need it for a slightly different reason.
For whisper, the README installs the package with pip and then requires the ffmpeg command line tool on the system, with per-distribution instructions. It also depends on OpenAI's tiktoken for fast tokenisation, and it warns that you may need a Rust toolchain installed if tiktoken has no prebuilt wheel for your platform, along with a setuptools-rust fix for a specific install failure. The codebase is described as expected to work with Python 3.8 to 3.11 and recent PyTorch versions, and a model download happens on first use.
For whisper.cpp, the path is clone, download a converted model, configure with cmake, build, and run the command line example against a WAV file. The one condition worth reading twice is at the bottom of that section: the example currently runs only with 16-bit WAV files, so anything else has to be converted first, and the README gives the ffmpeg invocation to do it. There is a faster route for a first look, a make target that downloads one English model and transcribes the bundled samples.
The practical difference is where ffmpeg sits. With whisper, ffmpeg is a system dependency of the tool and it handles any input format for you. With whisper.cpp, ffmpeg is part of your input pipeline and you are responsible for the format. In a batch service that already deals in compressed audio, that is a small thing to arrange. In an embedded product with one fixed capture format, it is either free or impossible depending on what the hardware emits.
Memory, hardware and where each fits
The two projects publish memory tables, and reading them together is the fastest way to size a deployment.
Whisper's table lists six model sizes, four with English-only versions, with approximate VRAM requirements from about 1 GB for the smallest through about 2 GB, 5 GB and 10 GB, and it says the relative speeds are measured on one accelerator while real-world speed varies with language, speaking speed and hardware. It also notes that the English-only models tend to do better, most visibly at the smallest sizes, and that turbo is an optimised version of the large model trading a little accuracy for speed.
whisper.cpp's table is in megabytes and gigabytes of resident memory, from a few hundred megabytes for the smallest model to just under 4 GB for the largest, alongside the disk size of each converted file. The same repository adds integer quantization, which reduces both memory and disk and, depending on the hardware, can be faster, with a documented procedure to quantize a converted model and point the example at the result. It supports mixed precision, CPU-only inference, and a long list of accelerators: Metal and Core ML on Apple Silicon, Vulkan, NVIDIA and AMD GPU paths, several NPUs, AVX and NEON intrinsics, and BLAS for POWER machines.
The conclusion from that is the situation each project was built for. whisper wants an accelerator it can use through PyTorch and enough memory for the larger models. whisper.cpp wants a small, predictable footprint and a wide set of places to run, which is why its platform list includes mobile, embedded boards and WebAssembly.
Where whisper falls short
Three limits are documented and worth planning around.
Streaming is not something the package offers as a feature, and the third-party tools that do it live outside the repository. If your product needs a caption that appears as someone speaks, that is a project you have to add, and the analysis of whisper tells you not to adopt it for that case.
Word-level timestamps are not a documented output, which is the same boundary that separates it from a forced-alignment stack.
Translation has a specific trap. The README says the default model selection is fine for English transcription, but that the default model is not trained for translation, and to use a multilingual model for non-English speech. The analysis adds the sharper form of the problem: with the default model, asking for translation returns the original language. That failure mode is quiet, because the output is fluent text in the right script, and it will not be caught by a test that only checks that something came back.
There is also the maintenance question, and it is a real one for a dependency. The newest tag in the release list is from 2025-06-26, while the recorded last push is 2026-08-31, and the analysis asks you to confirm that this is recent enough for whatever policy you hold. What that buys you is a reference implementation and an ecosystem, which is a lot; what it does not buy you is a stream of releases.
Where whisper.cpp falls short
The trade here is portability against convenience, and the inconvenience is specific.
Input format. The command line example takes 16-bit WAV only, so a pipeline that receives MP3s or phone recordings needs a conversion step that does not exist in the other project. The analysis of whisper.cpp names this directly: you supply 16-bit WAV audio or transcode it first.
No pipeline around it. It is not a Python-first stack, and the analysis says to skip it when you want ready-made speaker diarization, which lives in the Python ecosystem rather than in this repository.
An unversioned C API. The analysis describes the C-style interface as documented as C-style rather than versioned, which is a real difference from a package with a published interface policy. It is manageable, and it is not the same as a stability guarantee.
Weight management is yours. The converted format is not the same artefact as the PyTorch checkpoint, so the first thing a new deployment has to do is obtain or convert the model, and the analysis tells you to check the model size against your memory budget and confirm your platform appears in the supported list.
And for a team that already runs Python everywhere, it can be the wrong tool for a defensible reason. Adding a C and C++ build, a conversion step and an unversioned API to a service that already has PyTorch, ffmpeg and a working call to the model is work with no corresponding gain, and the honest version of this comparison is that the majority of server-side Python projects should stay with whisper and treat the C and C++ port as something for the client, the device and the edge.
A choice for concrete situations
The scenarios below are the ones the two projects are actually used in.
A Python service that transcribes uploads, with a GPU available and occasional need to translate non-English audio into English. That is whisper, with a multilingual model rather than the default, and with a conversion step in the pipeline for word timings if the product needs them.
A desktop or mobile application, or anything shipping inside a C or C++ product. That is whisper.cpp, and the platform list is the argument: the same model on the phone, on a laptop with no Python, and on a server, with one integration instead of three.
An embedded product on a Raspberry Pi, or a browser build. Also whisper.cpp. The memory table and the quantization support are what make it fit, and the C-style API is what makes it callable from a native application.
A batch job on a machine with no GPU, or a research pipeline that needs a library rather than a binary. whisper on the first, and whisper on the second if you need the Python ecosystem, with a note that the alignment and diarization tools live in other projects either way.
A team that wants both. This is the ordinary case rather than a compromise: prototype and explore in Python with whisper, convert the model once, and run the same weights in production with whisper.cpp wherever the deployment target is not a Python host. The two are not rivals, they are the same model at two stages of a product, and the conversion is documented rather than hidden.
Bottom line
whisper is the reference implementation and the right default for a Python service that already has ffmpeg, PyTorch and possibly a GPU, and it is the one to pick if you need its translation task, its language selection or a quick first transcript without a build step. whisper.cpp is the right pick for a device, a mobile app, an embedded board, a browser or any C and C++ host, and it is the only one of the two that lets you quantize the model and run without allocating anything at run time. Before you commit, verify three things: which model size fits the memory you actually have, whether your audio can be produced as 16-bit mono PCM at 16 kHz if you choose the C and C++ port, and whether the release cadence of the Python package is acceptable to your dependency policy. The judgement worth stating plainly is that this is a runtime decision rather than a model decision, so pick the one that matches where the inference happens, and treat the weights as portable between them.