whisper.cpp vs whisperX: a runtime and a pipeline
These solve adjacent problems rather than competing. whisper.cpp is a runtime that reimplements the Whisper model in dependency-free C and C++ so it can run on a phone, a board or a browser, while whisperX keeps the model in Python and adds what a transcript lacks: word-level timestamps and speaker labels. Choose on the shape of the output you need, and note that the same model runs under both.
At a glance
| Project | ggml-org/whisper.cpp | m-bain/whisperX |
|---|---|---|
| Licence | MITPermissive: commercial use allowed | BSD-2-ClausePermissive: commercial use allowed |
| Maintenance | Commits in the last six monthsLast push October 11, 2026 | Commits in the last six monthsLast push September 26, 2026 |
| Language | C++ | Python |
| GitHub stars | 54,298 | 24,472 |
| Read more | Our analysisGitHub | Our analysisGitHub |
Which one to choose
Choose whisper.cpp if you need a transcript somewhere your service does not run: a device, a mobile app, a browser build, a Raspberry Pi, a C or C++ host, or a machine with no Python. It is also the one to pick when integer quantization is the only way to fit the model in a fixed memory budget, and the one that ships a C-style API for embedding inference in a native application.
Choose whisperX if what you need is not a transcript but a transcript with structure: word-level timings for subtitles, or speaker labels for a meeting or an interview. That is what it adds on top of a faster backend, and it is worth its extra dependencies when you have a CUDA machine and a Hugging Face token for the diarization model. It is not worth them when you only need the words.
One is a runtime, the other is a pipeline
The comparison is easier to make once you notice that the two projects are not answering the same question.
whisper.cpp is a reimplementation. It takes OpenAI's Whisper model and writes the inference path again in plain C and C++ with no dependencies, on top of the ggml library, and the README states that the whole high-level implementation lives in two files. Everything else in the repository exists so that one model can run in many places: integer quantization, mixed precision, voice activity detection, a C-style API, and acceleration paths for Apple Silicon, x86, POWER, Vulkan, NVIDIA, AMD, and several NPUs. Its platform list is the point: macOS, iOS, Android, Java, Linux, WebAssembly, Windows, a Raspberry Pi and Docker.
whisperX does not reimplement anything. It is a pipeline built around a recogniser, taking three components and composing them: a faster backend for batched inference, a wav2vec2 model for forced phoneme alignment, and a diarization model for speaker labels, with voice activity detection as preprocessing. The README is explicit about why, noting that Whisper's own timestamps are at the utterance level rather than per word and can be inaccurate by several seconds, which is a problem for subtitles and a bigger one for a meeting with three speakers.
So the outputs differ in kind. One gives you a string. The other gives you a string annotated with when each word was said and who said it. Comparing them on accuracy would be comparing a file format to a database.
Getting each running
The two installations are not comparable in effort, and that is the first thing to check.
whisper.cpp is a clone, a model download, a cmake configure and a build, then a command line example against a WAV file. There is a make target that downloads one English model and transcribes the bundled samples, which is the shortest possible first run, and a documented quantization procedure for when the model needs to be smaller. The one input condition is stated in the README: the example currently runs only with 16-bit WAV files, and anything else has to be converted first, with an ffmpeg command provided.
whisperX starts one step earlier, because it is a GPU-first pipeline. Its setup section puts CUDA toolkit installation as step zero and says to skip it only if you are using the CPU alone. After that, installation is a single pip or uvx command, with a from-source option and a developer install for people who intend to modify the code, and it also notes you may need ffmpeg and rust following the upstream instructions. Diarization adds a dependency that is not a library: the README's diarization section requires a Hugging Face token and acceptance of the upstream model's terms, so the first successful run needs network access to a model host as well as to the packages.
The difference in practice is that whisper.cpp has no account or token anywhere in its quick start, and whisperX has one.
Memory, hardware and how fast each one goes
The two publish numbers that let you size a machine, and they point in opposite directions.
whisper.cpp's memory table runs from about 273 MB for the smallest model to just under 4 GB for the largest, alongside the disk size of each converted file, and the same repository documents integer quantization as a way to reduce both. It also supports CPU-only inference, which is the mode that makes a Raspberry Pi or a browser build viable at all, and the README's supported-platform list is essentially a statement about which machines can run a model locally with no Python present.
whisperX is sized for a GPU. Its README states the faster backend requires under 8 GB of GPU memory for the large model at a beam size of five, and it claims batched inference at seventy times realtime with that model, a figure that comes from the project's own description and its paper. It will run on CPU, as our analysis notes, at a speed penalty.
There is also a platform exclusion worth knowing before you plan an ARM server. Our analysis of WhisperX records that the project's own packaging excludes a dependency on Linux aarch64, which rules out the one architecture where whisper.cpp is strongest. That single line is often the whole decision: a GPU workstation with hours of recorded audio wants whisperX, and anything ARM-shaped or device-shaped wants whisper.cpp.
Where whisper.cpp falls short
Three things, in order of how often they matter.
No alignment and no diarization. This is the big one. A plain Whisper transcript from this runtime carries no per-word timing and no speaker identity, and the output is a stream of text with segment boundaries at whatever granularity the model produced. If your product needs a subtitle that stays in sync or a meeting transcript that says who said what, this project does not solve that and does not pretend to.
Input handling is yours. The 16-bit WAV requirement is a real constraint on a pipeline that receives compressed audio, and unlike the Python project there is no ffmpeg integration in the tool: the conversion is a step in your own code or your own job.
The interface is not versioned. Our analysis of whisper.cpp describes the C-style API as documented as C-style rather than versioned, which is a materially weaker promise than a package with a published interface policy. It is workable, and it is worth flagging to a team that intends to build a product on top of it.
Two smaller items. Weights are in the project's own converted format rather than the original checkpoint, so the first deployment step is obtaining or converting a model. And the analysis lists the case to skip it for: a Python pipeline that wants ready-made speaker diarization.
Where whisperX falls short
The costs here are dependencies and platform coverage rather than features.
The install chain is longer and partly outside the project's control: a CUDA toolkit, Python packages, possibly ffmpeg and rust, plus a Hugging Face token and an acceptance of terms for the diarization model. Our analysis tells you to verify that the diarization model loads with your token before committing, which is the step that surprises people.
Architecture coverage is narrower. The packaging exclusion for Linux aarch64 is the one that decides real projects, and there is no mobile or browser target in the same sense the other project has.
You now track two inference backends. The pipeline uses a faster backend rather than the reference implementation, so a change in either that project or the alignment model can move your output, and the release list shows a release candidate alongside stable versions, with our analysis advising you to check that your installer does not pick up the candidate by default.
And the accuracy question is inverted. If you only need text, the extra stages are not free: alignment and diarization are additional passes, and the project's own framing is that it is worth adopting when subtitle sync or speaker attribution matters more than a single pip install. For a plain transcript it is the more complicated way to get the same string.
Licences and maintenance
Neither project is a licensing obstacle, and both are being pushed, so this section is short.
whisper.cpp is MIT licensed. whisperX is BSD-2-Clause. Both are permissive licences of the kind that let you embed the code in a proprietary product without an obligation to publish your changes, and neither imposes a network or a reciprocity condition. If your policy forbids copyleft, both pass, and if your policy requires a patent grant, that is a question for whichever upstream model you load rather than for these repositories.
On activity, the recorded last pushes are close together, 2026-09-28 and 2026-09-26. The release shapes differ. whisper.cpp's newest tag is v1.9.3 from 2026-08-20, with a second tag from earlier the same month, which reads like a project shipping versioned releases regularly. WhisperX's list has v3.8.6 from 2026-05-25 and a release candidate, 3.8.7rc1, from 2026-06-26, which reads like a project shipping fewer, larger releases and cutting candidates along the way.
That difference has one practical consequence and one non-consequence. The consequence is that a whisper.cpp dependency is a smaller version to track, and the non-consequence is that neither project is close to abandonment, so maintenance is not a reason to prefer either one.
A choice for concrete situations
The situations below are where the two are actually chosen.
Subtitles, podcasts, anything with a video track and a fixed runtime budget. whisperX, because per-word timings are the entire reason that project exists and doing it yourself means adding an alignment model anyway.
Meeting and interview transcription where the value is in knowing who said what. Also whisperX. Its diarization stage is not something the other project attempts, and our analysis of WhisperX names speaker attribution as a case to adopt it for.
An on-device feature: a phone app, a desktop client, a voice assistant that must work offline. whisper.cpp, and the platform list is the reason rather than the licence.
An embedded product on a single-board computer, or a browser build. whisper.cpp, with quantization to fit the memory and CPU-only inference as the operating mode.
A backend service that just needs text, on hardware you already own. Neither is a bad choice, and this is the case where the simpler one wins: a dependency-free C and C++ runtime with a model you control, or a pip install that also gives you timings. Pick the one whose dependencies fit your build, because the quality of the transcript is the same model either way.
And the combined case, which is more common than the comparison suggests. Prototype in Python, ship the device build on the C and C++ runtime, and use the alignment pipeline offline for the subtitles, because all three are running the same model on different schedules.
Bottom line
whisperX is the better tool whenever the output needs structure rather than text, because word-level timestamps and speaker labels come from components no single-model runtime provides, and it costs a CUDA toolkit, a Hugging Face token and an architecture limitation to get them. whisper.cpp is the better tool whenever the deployment is a device, a board, a browser or a native application, because that is what its dependency-free implementation and quantization exist for. Both licences are permissive and both projects are being pushed, so neither licensing nor maintenance should decide this. Verify first whether your platform is excluded from the pipeline's packaging, whether your audio can be produced as 16-bit mono PCM at 16 kHz if you choose the runtime, and whether a stable release rather than a candidate is what your installer will fetch. The concrete judgement is that this is a question about output shape, so answer it by looking at what your product displays rather than at which project is more capable.