nyrahealth/CrisperWhisper 2.0: verbatim or intended transcription, chosen per request

The decision CrisperWhisper makes explicit

Most speech-to-text systems do not expose a choice between writing down what was said and writing down what was meant. The README states the position plainly: systems "inherit that choice from their training data and apply it inconsistently." CrisperWhisper 2.0 turns that into a parameter. The same recording can produce a verbatim transcript that keeps fillers, repetitions, cut-offs, false starts and vocal events in a consistent format, or an intended transcript with numbers, dates and emails formatted the way a writer would set them. The README's own example shows the contrast: the verbatim line keeps `[um]`, the repeated "we we", the cut-off "th-" and `[laughter]`, while the intended line reads as clean prose with "March 3 at 9:30".

The audience follows from that. Teams building TTS training data need disfluencies preserved because removing them changes the prosody the model must learn. Clinical and speech-pathology work needs stutters and pauses marked, not smoothed away. Dataset construction and labeling pipelines need a transcript that matches the audio sample by sample. None of those groups is served well by a system that silently normalizes. The other group, meeting notes and subtitle generation, wants the opposite, and CrisperWhisper gives it the same model with `mode="intended"`.

Two backends, conditional continuation, and how verbatimize works

The package is inference code for CrisperWhisper models. The pyproject file is explicit that the core install pulls in no inference backend at all: dependencies are `tokenizers`, `numpy`, `soundfile`, `soxr` and `huggingface_hub`, and you choose a backend through extras. The `ct2` extra installs a CTranslate2 fork at version 4.7.1.post2 or later, described in pyproject as the first fork build with `generate_dual_greedy` (transcribe_dual) and a native speculative loop. The `transformers` extra is the pure PyTorch path.

The pyproject comment carries a real constraint that is easy to miss: do not add `faster-whisper` to the same environment. It depends on upstream ctranslate2, which installs into the same `site-packages/ctranslate2/` directory as the fork and clobbers it. The FeatureExtractor is vendored inside `crisperwhisper/features.py` instead. The fork wheels are built with CUDA_DYNAMIC_LOADING=ON and dlopen `libcublas.so.12` at runtime; `crisperwhisper/_nvidia_libs.py` preloads the pip copy of that library so GPU inference works without a system CUDA toolkit.

Longform handling is the other mechanism worth naming. Audio of any length is transcribed without chunk-boundary artifacts because each window continues from the words already transcribed, a scheme the README calls conditional continuation. The stated benefit is no duplicated or dropped words at the seams and no timestamp-token bookkeeping. The README does not document how the continuation state is reset after a decoding failure, and the truncated pyproject excerpt does not show the full extra list, so the `all` extra mentioned in a comment cannot be checked here.

Verbatimize is the feature with the least obvious mechanism. Given audio plus a trusted clean transcript, the model is asked to reproduce the content word-for-word and insert only the disfluencies and vocal events actually present in the audio. The README reports rare-word recall rising from 6.8% to 96.1% versus re-transcribing. That comparison matters because re-transcribing a clip is the obvious alternative, and the README's numbers say it is the wrong one when you already trust the text.

Installing CrisperWhisper and running a first transcription

Install from PyPI. Pick one extra. The `ct2` path is the fast one and needs only an NVIDIA driver on Linux, because CUDA libraries arrive through pip. The `transformers` path runs anywhere torch does, including macOS, Windows and CPU.

pip install "crisperwhisper[ct2]"

pip install "crisperwhisper[transformers]"

The first load of a model downloads it from HuggingFace. On the `ct2` backend it also converts the model once into a local cache, and that conversion needs torch and transformers, which the lean `ct2` extra does not install. The README gives this fix:

pip install "crisperwhisper[ct2,convert]"   # or add [convert] later

A CPU-only torch is enough for the conversion. The README states the dependencies can be uninstalled once the converted model is cached, and that loading a directory which already contains a converted CT2 model (`model.bin`) never needs them. For a first run, the default model is `nyralabs/CrisperWhisper2.0_large`; `turbo`, `medium` and `small` are also available as shorthands.

from crisperwhisper import CrisperWhisperModel

model = CrisperWhisperModel()          # nyralabs/CrisperWhisper2.0_large
result = model.transcribe("meeting.wav", language="en")
print(result.text)

That call returns the verbatim transcript, which is the default. To get the readable version instead, pass `mode="intended"`. To get per-word boundaries, pass `word_timestamps=True` and read `result.words`, where each entry exposes `start`, `end` and `word`. Audio longer than 30 seconds is handled automatically, according to the README, so no chunking code is needed on your side. If you already have a clean transcript you trust, `model.verbatimize("clip.wav", "I think we should ship it Friday.")` is the entry point for the upgrade path.

Timing accuracy and the benchmark's own caveats

The README's headline numbers come from the project's own benchmark, the Nyra Verbatim Speech Benchmark, and the project runs it. That is not a reason to dismiss the numbers, but it is a reason to read the footnotes. Disfluency F1 is averaged over ten languages. English and German use human-labeled evaluation sets; the other eight use synthetic verbatim sets. CrisperWhisper 1.0 was English and German only, and its 64.8 average covers just those two languages, which means the 1.0 row and the 2.0 row are not measuring the same thing. The README says so in a footnote, which is more than many benchmark tables do.

Word timing is the more directly checkable claim. The README reports 29.6 ms mean absolute word-boundary error on read speech (TIMIT) for CrisperWhisper 2.0, against 37.1 ms for xAI Grok Speech-to-Text, 49.3 ms for CTC-seg, 64.8 ms for WhisperX and 85.5 ms for Canary. The footnote states the scoring is done on exactly the words each system gets right, and that the timings are extracted from supervised cross-attention. That last detail is the mechanism: the aligner is the model's own attention, not a separate forced-alignment pass. The README does not give a per-language breakdown of boundary error in the table itself, and points to a separate post for conversational-speech results.

Treat the leaderboard as a claim with a stated methodology rather than an independent verdict. The comparison set is the one the project chose, and the eight synthetic-language sets are synthetic. For a decision, the more useful question is whether your audio resembles read speech or conversational speech, because the README reports 30 ms and 41 ms mean boundary error respectively for those two conditions.

Where CrisperWhisper is the wrong tool

The first limitation is licensing, and it is structural. The pyproject file declares `license = {text = "MIT"}` for the package, and the repository carries a LICENSE file. The model weights are separate artifacts on HuggingFace, and the README does not state their terms. The repository metadata's license field reads NOASSERTION, which is consistent with a package whose code and weights may not share a licence. If your legal review needs one permissive grant covering both the code and the weights, check the model card before you build on it. This is not a legal opinion; it is a pointer to the one document that would settle it.

The second limitation is the dependency situation on the fast path. The `ct2` extra depends on a fork, not on upstream CTranslate2, and the pyproject comment names a concrete collision: installing `faster-whisper` in the same environment silently replaces the fork's `site-packages/ctranslate2/` directory. If your stack already contains faster-whisper for another model, you cannot simply add CrisperWhisper beside it. You need separate environments or a different deployment shape.

The third is the one-time conversion. It needs torch and transformers even on the `ct2` path. In a container build that is fine, because the conversion happens once and the dependencies can be removed. In a serverless or read-only-filesystem deployment where the cache cannot persist, you pay that cost on every cold start, or you pre-bake a converted model directory containing `model.bin`.

Finally, if you only ever want clean, readable transcripts, the verbatim machinery is overhead you are not using. A conventional ASR system with a punctuation model will be simpler to operate, and the disfluency metrics that make CrisperWhisper interesting will not appear in your output at all.

CrisperWhisper against WhisperX and plain Whisper

The natural comparison is WhisperX, and the difference is architectural rather than a matter of tuning. WhisperX keeps Whisper's transcription and bolts on a separate forced-alignment stage using a phoneme model to recover word boundaries. CrisperWhisper derives timings from supervised cross-attention inside the model, which the README's aligner note describes, and reports 29.6 ms boundary error against WhisperX's 64.8 ms on TIMIT read speech. The practical consequence is one model to load and one pass to run, against WhisperX's two-stage pipeline where alignment failures are a separate class of bug.

The second difference is what the transcript contains. WhisperX inherits Whisper's normalization behavior, so fillers and repetitions appear or disappear depending on the recording. CrisperWhisper exposes `mode="intended"` and the default verbatim mode as a per-call choice, and adds `verbatimize` for audio you already have a clean transcript for. WhisperX has no equivalent of that last path: its answer to a clean transcript is to re-transcribe, which the README's rare-word recall comparison (6.8% versus 96.1%) argues against.

Against plain Whisper, the difference is the same choice plus longform handling. Whisper's chunked processing produces seam artifacts; CrisperWhisper's conditional continuation is designed to avoid duplicated or dropped words at window boundaries. The README also claims built-in mitigation of Whisper's looping-hallucination failure mode in the CTranslate2 runtime, though it does not describe the mitigation, so treat that as an unverified claim about a known failure mode rather than a documented mechanism.

Maintenance, upgrades and what to pin

The last push to the default branch was on 2026-09-14, and the most recent release, v2.0.3, was tagged the same day. That release is described as a transformers-only install fix plus word timestamps for space-less languages. The release before it, v2.0.1 on 2026-07-23, is labeled early-EOT recovery. v2.0.0 landed on 2026-07-21. Three releases in roughly two months, with the newest fixing an install path, suggests the 2.0 line is still settling rather than frozen. The package classifiers say Development Status 4 - Beta, which is consistent with that.

The upgrade cost concentrates in two places. First, the CTranslate2 fork: pyproject pins a minimum of 4.7.1.post2 and explains that this specific build is where `generate_dual_greedy` and the native speculative loop appeared. Upgrading the fork is not a routine dependency bump, because the fork's API surface is what the dual-greedy path calls. Second, the converted model cache. Moving between package versions may invalidate a cached CT2 model, and re-conversion needs torch and transformers present again. Pin both the package version and the model revision in production, and keep the conversion step in your build rather than in your startup path.

On licensing, the code side is MIT per pyproject and the LICENSE file. The weights are a separate grant. The README does not state what that grant is, and the NOASSERTION metadata does not resolve it. If you redistribute the models, or ship them inside a product, that is the question to answer first.

Editorial conclusion

Adopt CrisperWhisper if you need verbatim transcripts with word-level timings, or if you already hold clean transcripts and want disfluencies inserted without re-transcribing. Do not adopt it if you cannot run a Python 3.10+ service, if you need a permissively licensed model rather than permissively licensed code, or if a plain intended-mode transcript is all you ever want. Verify first: the model licence on the HuggingFace model card, whether your deployment can install the torch and transformers dependencies for the one-time CT2 conversion, and whether your audio domain resembles the read and conversational speech the project's benchmark covers.

Frequently asked questions

Official sources

1. Issues
2. nyrahealth/CrisperWhisper on GitHub
3. Project website
4. README
5. Releases