Six Whisper backends behind one --device flag, each in its own venv

The device router builds a fresh environment on first use

One flag decides the engine. `--device` picks the backend and, when you leave it out, the router auto-detects one. The cpu and cuda names land on openai-whisper as the reference path. `insane` and `insane-flash` route to insanely-fast-whisper on a batched Hugging Face pipeline with FlashAttention2. The Apple Silicon device name mlx picks lightning-whisper-mlx on Metal, xpu drives an Intel Arc card through torch+xpu, `whisperx` brings VAD plus wav2vec2 alignment plus diarization, and `sensevoice` exists for CJK audio.

No backend runs inside your own environment. Each one is built into its own isolated uv venv the first time you select it, which is why the package itself declares no torch dependency and why changing engines later costs a build rather than a flag change. Before any of that, yt-dlp and static-ffmpeg turn whatever you pointed at into 16-bit wav.

out.srt and out.vtt come free, but speaker.json needs a token

Every backend writes the same five files: `out.txt`, `out.srt`, `out.vtt`, `out.json`, and `speaker.json`, that last one partitioning the transcript by who was speaking and when. The project's own pitch calls the speaker file the thing no other front-end produces. The wiring tells a narrower story. The architecture diagram attaches speaker.json to the shared output node for every backend, yet the only diarization path actually described runs through WhisperX and is gated behind two flags plus a Hugging Face token:

transcribe-anything video.mp4 --device whisperx --diarize --hf_token <hf_xxxx>

On cpu, cuda, mlx or sensevoice you get four text formats and no speaker attribution unless you change backends. The README does not say what speaker.json holds on a run with no diarization at all, so treat that file as backend-dependent rather than guaranteed.

Installing from PyPI, plus the extras that skip the build

The package is distributed through its PyPI project page, and the README prints no install line to copy from, so take the command from there. pyproject.toml sets `requires-python = ">=3.10"`, which rules out anything older.

Two extras change what gets installed. The server group adds fastapi, uvicorn, python-multipart and websockets so that `transcribe-anything serve --no-iso-env` runs the daemon inside your own environment instead of paying for an isolated build on first launch. The stream group adds faster-whisper and silero-vad, and `--allow-stream` only does real work when that extras group is present; without it, the flag falls back to a canned protocol-validation path.

A first transcription takes a media path and a device name, for example:

transcribe-anything video.mp4 --device xpu

On that first run the XPU environment is assembled from wheels pulled at PyTorch's official XPU index, pinned by exact version and explicit index so nothing in the chain can resolve from an untrusted source. Budget for each new backend to spend its first run building an environment, then measure it again on the second.

insane-flash verifies the CUDA extension before it starts

`insane-flash` runs the same insanely-fast-whisper model path as `insane`, but inside a separate environment with pinned FlashAttention2 wheels for Windows x86_64, Linux x86_64 and Linux aarch64, on Python 3.11 with `torch==2.7.0+cu128` and `flash-attn==2.8.3`. Before a single segment is transcribed it verifies flash_attn and the compiled CUDA extension, and when no controlled wheel matches the platform it stops with diagnostics instead of quietly downgrading to something slower.

transcribe-anything video.mp4 --device insane-flash --batch-size 8

That early failure is deliberate, and unusual for a wrapper. The plain `insane` device stays available for machines where those wheels do not exist, and you give up the guaranteed attention kernel to get there. Arc carries a matching caveat: that path is labelled experimental, and the project asks for an issue naming your card model if it misbehaves.

The align pass borrows WhisperX and can fall back silently

Timestamp drift on long audio is what `--align` is for. Added to `insane` or `insane-flash`, it runs a WhisperX wav2vec2 forced-alignment post-pass over the finished transcript: every chunk grows a `words: [{word, start, end, score}]` array, and segment timestamps tighten to the first and last word. The subtitle files inherit those bounds without extra work.

Two costs follow. Reuse means the WhisperX environment gets built even when you asked for the fast backend, so alignment drags in a second heavy stack. Hedging means the pass is best-effort by design: an unsupported language, a failed environment build or a crashed runner falls back to the original output with a warning on stderr and transcription continues.

# Phoneme-precise timestamps on the fast path:
transcribe-anything video.mp4 --device insane --align
transcribe-anything video.mp4 --device insane-flash --align

# Force a specific wav2vec2 aligner for languages outside the 41 defaults:
transcribe-anything video.mp4 --device insane --align --align_model facebook/wav2vec2-large-960h-lv60-self

Outside the 41 default languages you name the aligner yourself through `--align_model`, passing a model id in the form shown above.

Reading a URL puts yt-dlp and static-ffmpeg on the path

Point the CLI at a YouTube, Rumble or BitChute link and a fetch stage runs before a model loads: yt-dlp downloads, static-ffmpeg normalises the result to 16-bit wav, and only then does the router see a local file. pyproject.toml pins yt-dlp at 2026.6.9 or newer, and the comment beside that line gives the reason, advisory GHSA-537c/69qj/vx4q/c6mh, covering --exec command injection, an aria2c manifest code execution path, a filename sanitization bypass and a cookie leak. The downloader is therefore the riskiest dependency in the chain, not the model.

The remaining direct dependencies stay small. appdirs locates cache directories, disklru and FileLock keep concurrent runs from colliding over them, and uv-iso-env at 1.0.44 or newer does the environment work, alongside webvtt-py pinned at 0.4.6, python-dotenv and httpx. For anyone auditing this setup, the first question is not which model runs but which network yt-dlp reaches on your behalf.

Docker merges the two insane backends into one environment

The image is built FROM nvidia/cuda:12.8.0-base-ubuntu22.04, installs the package with pip in editable mode, and prebuilds backend environments during the build even though no GPU is visible at that moment, which is why the runtime CUDA checks are skipped. The revealing line is an environment variable: TRANSCRIBE_ANYTHING_SHARED_INSANE_BACKEND=flash. On a laptop, insane and insane-flash are deliberately separate environments with different pinned wheels. Inside the image they collapse into one FlashAttention-capable environment to avoid duplicating the whole torch stack, and PREBUILD_BACKENDS picks how much gets built, with none giving the smallest image.

docker run --gpus all transcribe-anything <args>

A compose file maps host port 8092 to container port 80 and reserves GPU capabilities, so a served deployment is one file away, and the repository ships Caddyfile and nginx examples under examples/. Two gaps matter before planning that: the image is CUDA-based, so the Intel Arc and Apple Silicon device paths have no home in it, and the README never explains how to start the service it exposes.

What the files leave unresolved, and where the alternative bites

Two routes reach the same transcript. Calling openai-whisper, insanely-fast-whisper or WhisperX directly gives you one environment you control, one dependency set you can pin, and no first-run build; the price is that CUDA, torch and flash-attn versions become your problem to reconcile, which is exactly the friction this router absorbs. A hosted service removes the model work entirely and requires the audio to leave your machine, which cuts against the local-processing design here. Note the asymmetry in that claim: a local file never leaves, but a URL still leaves your network through yt-dlp.

Packaging drift sits underneath both routes. pyproject.toml carries a manual `version = "4.1.2"` with a comment telling the maintainer to update it by hand, one step past the newest two builds listed for the project, 4.1.1 and 4.1.0, even though the same file already requires setuptools-scm in its build backend. The licence lines disagree too: the GitHub project page shows MIT while pyproject.toml declares `license = { text = "BSD-3-Clause" }`. Two artefacts, two answers, no note explaining which governs.

The 4.0 notes also head a section with read-only installs, and no flag in the documented command set turns one on.

Editorial conclusion

Adopt it if you want one command across CPU, CUDA, Apple Silicon and Intel Arc, and can accept a first run that assembles an isolated environment for each backend. Leave it if you already settled on openai-whisper or WhisperX and need only that one path, or if your machines cannot reach the wheel index to build those environments. A missing pinned CUDA wheel fails loudly instead of falling back to a slow path, and that behaviour is worth keeping. Check the yt-dlp floor you inherit, and note that the last push was on 2026-07-11.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. zackees/transcribe-anything on GitHub