Whisper ASR Box: a self-hosted speech recognition API for Whisper, faster-whisper and WhisperX

What Whisper ASR Box actually replaces

Running Whisper locally usually means writing a script: load the model, decode the audio, loop over files, format the output. That is fine for one person on one machine. It stops being fine when a second service needs transcription, or when the audio arrives as a file upload from something else, or when you want subtitles rather than plain text.

Whisper ASR Box turns that script into a service. The README describes it as "a general-purpose speech recognition toolkit" and the pyproject.toml description calls it "a general-purpose speech recognition webservice". The audience is therefore infrastructure-minded: people who want an HTTP endpoint they can call from another program, not a library they import. The topics attached to the repository (asr, docker, speech-to-text) point the same way.

The scope is deliberately narrow. It does not do audio capture, it does not queue jobs, and it does not manage users. It accepts audio, runs one of three Whisper implementations, and returns text, JSON, VTT, SRT or TSV.

Three engines behind one FastAPI process

The design decision that matters most is that the project does not pick a Whisper implementation for you. Release v1.10.0 bundles openai/whisper at v20250625, SYSTRAN/faster-whisper at v1.2.1 and whisperX at v3.4.5. You select one at runtime with the ASR_ENGINE environment variable, whose documented values are openai_whisper, faster_whisper and whisperx.

The three are not interchangeable in behaviour. The README lists speaker diarization as a feature "(with WhisperX)", which means that if you choose openai_whisper or faster_whisper, that capability is simply not part of the response. Word-level timestamps and voice activity detection are listed as general features, but the engine you set still determines what the underlying implementation can produce.

Model selection is separate from engine selection. ASR_MODEL takes values such as tiny, base, small, medium and large-v3, and ASR_MODEL_PATH lets you point at a custom location for storing and loading weights. ASR_DEVICE chooses cuda or cpu. MODEL_IDLE_TIMEOUT controls unloading an idle model, which is the project's answer to the memory cost of holding weights between requests.

The container itself is assembled from three stages in the Dockerfile: an onerahmet/ffmpeg image supplies the FFmpeg binary, swaggerapi/swagger-ui supplies the Swagger UI assets, and python:3.12-bookworm is the runtime. That is why the README can claim broad audio and video format support without the project writing any decoding code of its own. It also means the licence surface is wider than the MIT text in LICENCE suggests, because FFmpeg is included under LGPLv2.1.

Installing Whisper ASR Box with docker run and a first transcription

The README's quick usage section is Docker-first. The CPU command starts the service detached on port 9000, with the base model and the OpenAI Whisper engine. The environment variables are the same ones described above, passed inline.

docker run -d -p 9000:9000 \
  -e ASR_MODEL=base \
  -e ASR_ENGINE=openai_whisper \
  onerahmet/openai-whisper-asr-webservice:latest

The GPU variant is the same command with --gpus all and the :latest-gpu tag instead of :latest. The README gives no partial-GPU or multi-GPU example, so a single visible device is the documented assumption.

docker run -d --gpus all -p 9000:9000 \
  -e ASR_MODEL=base \
  -e ASR_ENGINE=openai_whisper \
  onerahmet/openai-whisper-asr-webservice:latest-gpu

The README also documents a cache volume, which is the difference between a container that starts immediately and one that re-downloads weights every time. Mounting $PWD/cache at /root/.cache/ is the documented way to persist them.

docker run -d -p 9000:9000 \
  -v $PWD/cache:/root/.cache/ \
  onerahmet/openai-whisper-asr-webservice:latest

After the container is up, the README says to visit http://localhost:9000 or http://0.0.0.0:9000 in a browser. What you get there is the Swagger UI, not a web app: the endpoints are listed and can be tried from the page. That is the intended first use, and it is also the honest limit of the built-in interface.

The repository's docker-compose.yml is the alternative entry point for people who prefer a file to a long command. It builds from the local Dockerfile rather than pulling the published image, sets ASR_MODEL=base, maps 9000:9000, mounts ./app into /app/app for live code, and declares a named cache-whisper volume at /root/.cache. A docker-compose.gpu.yml file also exists at the top level, though the README does not describe it.

For development without Docker, the README gives a Poetry path. It requires poetry v2.X, installs with an extra of either cpu or cuda, and runs the console script that pyproject.toml declares as whisper-asr-webservice.

pip3 install poetry
poetry install --extras cpu
poetry run whisper-asr-webservice --host 0.0.0.0 --port 9000

Where Whisper ASR Box is the wrong tool

The first limitation is resource cost, and the project does not hide it but does not solve it either. Whisper weights are large, and MODEL_IDLE_TIMEOUT exists precisely because keeping a loaded model around is expensive. On a CPU-only host, the larger ASR_MODEL values are not a configuration detail, they are a capacity decision. If your machine cannot hold the model, no environment variable fixes that.

The second is that the service is synchronous in the way it presents itself. The README documents a REST API and Swagger documentation, and nothing about a job queue, batching, or asynchronous submission. If you need to transcribe a backlog of long files and poll for results, the documentation does not describe that workflow, and the README is silent on concurrency behaviour. Treat throughput as something you must establish yourself before depending on it.

The third is that diarization is engine-bound. Anyone who reads "speaker diarization" in the feature list and then sets ASR_ENGINE=faster_whisper will not get it. The parenthetical in the README is the whole specification.

Finally, this is a self-hosted component. There is no hosted endpoint, no account, and no managed upgrade path. Upgrading means pulling a new image or bumping the pinned engine versions yourself. The repository's last push was on 2026-08-09, and v1.10.0 was released the same day, so the project is current, but currency is not the same as a support contract.

Whisper ASR Box compared with calling the OpenAI API or using whisper.cpp

The obvious alternative is the hosted OpenAI transcription API. The difference is not accuracy, it is where the audio goes and who pays per minute. The hosted route removes the GPU problem entirely and gives you a managed endpoint, but every file leaves your network and the cost scales with volume. Whisper ASR Box keeps the audio local and the cost fixed at the machine you already own, in exchange for operating a container and choosing a model size that fits.

The other realistic alternative is a native CPU inference project such as whisper.cpp, which is not bundled here. That approach trades the Python and PyTorch stack for a compiled binary, which can be attractive on small hardware where installing torch is the hard part. The trade is ecosystem: Whisper ASR Box gives you FastAPI, Swagger UI, five output formats and three engine backends out of the box, while a native binary leaves the HTTP layer and the subtitle formatting to you.

A narrower comparison is worth making inside the project itself. If you are choosing between the bundled engines, faster-whisper is the CTranslate2 reimplementation and whisperX is the one the README associates with diarization. The project's value is that switching between them is an environment variable rather than a rewrite, which is a real architectural benefit and the main reason to prefer this over wiring one engine directly.

Licence and the cost of staying current

The project is MIT licensed, and pyproject.toml carries license = { text = "MIT" }. That is permissive and unsurprising for a wrapper of this kind.

The complication is in the credits section of the README: the software uses libraries from the FFmpeg project under LGPLv2.1. Because the Dockerfile copies the FFmpeg binary into the image from a separate build stage, anyone redistributing the container is distributing FFmpeg too. The README states the licence and stops there. Whether that matters depends on how you ship the image, and it is worth a look before you bake it into a product. None of this is legal advice; the LICENCE file and the FFmpeg licence text are the sources.

Upgrade cost is mostly the engine pins. v1.10.0 pins openai/whisper to v20250625, faster-whisper to v1.2.1 and whisperX to v3.4.5, and pyproject.toml pins torch and torchaudio to 2.7.1 with separate CPU and CUDA 12.6 sources. Moving any of those forward is a deliberate act, not something that happens quietly, which is good for reproducibility and mildly annoying for security patches in the dependency tree. The repository keeps a CHANGELOG.md at the top level, so the release history is traceable without reading commits.

Editorial conclusion

Adopt Whisper ASR Box if you want transcription as an internal HTTP endpoint and you can run Docker on a machine with enough RAM for the model you pick, and if one of the three bundled engines (openai_whisper, faster_whisper, whisperx) covers your audio. Do not adopt it if you need a hosted API with an SLA, if you cannot run Docker, or if you expect diarization from every engine, since the README ties speaker diarization to WhisperX. Before rolling it out, verify three things: that your chosen ASR_MODEL fits the host memory, that your audio decodes through the bundled FFmpeg, and that the ASR_ENGINE you set is the one whose output format you actually need. The project is MIT licensed, but the container also ships FFmpeg under LGPLv2.1, so check that combination against your own distribution rules.

Frequently asked questions

Official sources

1. ahmetoner/whisper-asr-webservice on GitHub
2. License: MIT
3. Project website
4. README
5. Releases