# Whisper-Flow turns OpenAI Whisper into a streaming WebSocket transcriber

> Whisper-Flow wraps OpenAI Whisper in a FastAPI service that accepts 16 kHz PCM audio over a WebSocket and emits partial and final transcripts as events. Segment boundaries come from speech pauses, so latency improves and accuracy can suffer.

**dimastatz/whisper-flow** — Whisper-Flow is a framework designed to enable real-time transcription of audio content using OpenAI’s Whisper model. Rather than processing entire files after upload (“batch mode”), Whisper-Flow accepts a continuous stream of audio chunks and produces incremental transcripts immediately.

- Repository: https://github.com/dimastatz/whisper-flow
- Website: https://github.com/dimastatz/whisper-flow
- Stars: 997 · Forks: 137
- Language: Python
- License: MIT
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/dimastatz-whisper-flow

## Batch Whisper answers after the file, Whisper-Flow answers while you speak

OpenAI Whisper is built for batch work. You hand it a complete audio or video file, it processes the file, and it returns the text afterwards, much like giving someone a recording and receiving the transcript later. That model does not fit live captions, live note taking or a dictation box that has to show words as they are spoken.

Whisper-Flow changes the shape of the interaction, not the model. The description calls it a framework for real-time transcription of audio content using OpenAI's Whisper model. Instead of uploading a file, a client delivers a continuous stream of audio chunks and the service returns an incremental transcript immediately.

That is the whole promise, and it comes with a stated price. Streaming beats batch on speed and on the ability to embed speech-to-text in an application, but the documentation is direct that this speed may come at the expense of accuracy in some cases. A team that treats transcripts as a final artifact has to weigh that against the benefit of seeing text early.

## Tumbling windows decide when a partial transcript becomes final

The interesting part of the implementation is how it knows when to stop revising a result. The approach follows the streaming windowing pattern described in the documentation: operations run inside temporal windows, and a tumbling window gathers events into segments until a condition is met.

Concretely, Whisper-Flow splits the audio stream into segments based on natural speech patterns such as speaker changes or pauses. Within a segment the service sends back a series of events, and each response contains more transcribed speech than the one before it. The event carries the transcript, an end time and a flag for whether it is partial, and the sequence in the documentation shows a line growing word by word before a final copy is emitted.

A segment becomes final after a pause in speech, when its text stops changing, at the window cap, or when the client sends a flush or stop control frame. Four different termination paths matter in practice: speech pauses cover normal conversation, the window cap bounds how long a segment can grow, and flush and stop let a caller close the current segment deliberately when it knows more audio is not coming.

## The WebSocket contract: 16 kHz PCM in, JSON events out

Clients connect to the /ws endpoint and stream audio as binary frames in 16 kHz, mono, int16 PCM format. Results come back as JSON documents shaped like this:

```json
{"is_partial": bool, "data": {"text": ...}, "time": ms}
```

The time field is in milliseconds, the data object carries the text, and is_partial tells the client whether to keep revising or to commit the line. A client can also send optional JSON text frames to control the session: start, which carries the model, the language and a vocabulary prompt; flush, which forces the current segment out; and stop, which ends the session. A separate GET /ready endpoint reports the protocol version and the available models, so a client can check compatibility before opening a stream.

The full contract, including close codes, endpointing settings and how to add models, lives in docs/protocol.md. That split is a sensible one for a streaming protocol, since the wire format changes less often than the tuning knobs, but it also means the README gives you the shape of the conversation and nothing about close code semantics or endpointing parameters.

## Latency is measured between partials, accuracy with word error rate

The project evaluates itself on two metrics. Word Error Rate measures transcription accuracy, and latency measures the time between two subsequent partial results, with the stated goal of sub-second latency. The README explains that it does not start from scratch on quality measurement, because benchmarks have already been run for different ASR engines, and it leans on a research article benchmarking open source and paid speech-to-text services for guidance.

For its own measurements the author uses LibriSpeech. The sample output in the documentation shows the partial flag, a latency figure in the 175 to 470 millisecond range, and a sentence that is revised repeatedly as it grows, with the final line marked as not partial.

Two cautions for a reader. The latency numbers come from the project's own documentation, so they describe the author's setup rather than yours, and they are quoted here as reported. And a sub-second partial cadence says nothing about word error rate, which is the number that decides whether the transcript is usable.

## Installing whisperflow and pointing a client at /ws

The README does not give a pip install line, so the packaging story comes from the repository files. setup.py names the distribution whisperflow, reads its version out of whisperflow/__init__.py, and takes its install requirements straight from requirements.txt. The runtime dependency list is short enough to read in full:

```text
fastapi==0.108.0
uvicorn[standard]==0.52.2
python-multipart==0.0.32
openai-whisper==20250625
torch
numpy
PyAudio==0.2.14
```

The service layer is FastAPI served by uvicorn, the recogniser is the openai-whisper package on top of torch, and PyAudio covers local capture. The package name on install is simply:

```bash
pip install whisperflow
```

For a container, the Dockerfile starts from python:3.12-slim-bookworm and installs the system libraries that PyAudio needs to build, which is the part a bare pip install often trips over:

```dockerfile
FROM python:3.12-slim-bookworm
RUN apt-get update && apt-get install -y \
    portaudio19-dev \
    python3-dev \
    build-essential \
    && rm -rf /var/lib/apt/lists/*
```

A first real use is a client that opens /ws, sends 16 kHz mono int16 PCM frames, and prints the text field of each event until the is_partial flag comes back false.

## Compared with batch Whisper, the difference is the API and the timing

Whisper-Flow is not a different recogniser. The same OpenAI Whisper model does the work in both cases, and the quality ceiling is the model's. What changes is the interface.

Batch usage hands over a file and receives one finished text. Streaming usage hands over chunks and receives a growing sequence of revisions, which means the caller has to decide how to display text that is still moving. That is a real cost in application code: a caption overlay, a chat input or a document editor each need their own handling of a partial line, and the is_partial flag is what makes that possible.

The trade the project names is accuracy for latency. A batch pass can use the full context of a file, while a streaming segment is decided by pauses, speaker changes and the window cap, and a segment that ends at the wrong place is a segment with a truncated sentence. For long-form material where you can wait, batch is the better tool.

## Model files ride along in the package, and the version lives in __init__.py

The distribution ships more than Python modules. setup.py sets package_data to models/*.pt for the whisperflow package, so the model files are part of the installed package rather than something fetched at runtime, and include_package_data is on so the data files ship with the distribution.

The version is not repeated in setup.py. It is read out of whisperflow/__init__.py with a regular expression, because in an isolated build the package is not yet on the import path, so importing it to ask for its version would fail. The version in the file is also what a release is tagged from, and the recent tags show a fast cadence: v1.3.0 on 2026-10-10, v1.2.1 on 2026-10-08, and a much older 1.0-RC from 2024-08-10.

Project metadata is handled separately. pyproject.toml configures pytest with asyncio_mode set to strict and the tests directory as the test path, and points coverage at the whisperflow package. The licence is MIT, and the repository is not archived, with the last push on 2026-10-11. Because the protocol surface is documented in a separate file, an upgrade can change endpointing behaviour without changing the wire format, which is worth knowing before you pin a version.

## Conclusion

Use it when an application needs a transcript while the audio is still being produced, for captions, live notes or voice input. Keep batch Whisper for files you can afford to process in one pass, since streaming trades accuracy for latency. Before building on it, read docs/protocol.md for the endpointing settings and the close codes, and confirm that the window cap fits the speech pattern of your audio, because that is what decides when a partial result is finalised.

## FAQ

### How do I install Whisper-Flow and what are its dependencies?

The distribution is named whisperflow and the install line is pip install whisperflow. Its runtime requirements come from requirements.txt and include fastapi, uvicorn, python-multipart, openai-whisper, torch, numpy and PyAudio, and setup.py declares support for Python 3.8 and newer.

### What audio format does Whisper-Flow expect on the WebSocket?

Clients send binary frames to /ws in 16 kHz, mono, int16 PCM format, and receive JSON results shaped like {"is_partial": bool, "data": {"text": ...}, "time": ms}. Optional JSON text frames start the session with a model, language and vocabulary prompt, or flush and stop it.

### How does Whisper-Flow decide that a transcript segment is final?

A segment becomes final after a pause in speech, when its text stops changing, at the window cap, or when the client sends a flush or stop frame. The endpointing settings that tune this behaviour are described in docs/protocol.md.

### What is the latest release of Whisper-Flow?

The newest tag is v1.3.0, published on 2026-10-10, and it arrived two days after v1.2.1 on 2026-10-08. The repository is not archived and the last push was on 2026-10-11.

## Sources

- [dimastatz/whisper-flow on GitHub](https://github.com/dimastatz/whisper-flow)
- [License: MIT](https://github.com/dimastatz/whisper-flow/blob/main/LICENSE)
- [Project website](https://github.com/dimastatz/whisper-flow)
- [README](https://github.com/dimastatz/whisper-flow/blob/main/README.md)
- [Releases](https://github.com/dimastatz/whisper-flow/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/dimastatz-whisper-flow