Unmute: Kyutai's Websocket Bridge Between Text LLMs and Real-Time Speech

What Unmute adds on top of a text-only LLM

A text LLM has no audio path. Unmute supplies one: speech-to-text transcribes what the user says, the LLM produces a text reply, and text-to-speech reads that reply back. The README describes the project as a system that lets text LLMs "listen and speak" by wrapping them in Kyutai's own STT and TTS models. The audience is anyone who already has an LLM endpoint and wants a spoken interface in front of it without writing their own streaming audio pipeline. The repository is Python, MIT licensed, and the models themselves live in a separate project, kyutai-labs/delayed-streams-modeling, so Unmute is best understood as the glue and the deployment recipe rather than the speech research. If you only want the STT or TTS model in isolation, the README points you elsewhere.

The websocket chain from browser to speaker

The architecture in the README is a fan-out from a single backend process. The user's browser loads a frontend, and clicking connect opens a websocket to the backend that carries audio and metadata in both directions. The backend in turn opens a websocket to the speech-to-text server, streaming the user's audio out and receiving transcription back in real time. When STT decides the user has stopped speaking, the backend queries an LLM server for a response. As that response is generated, the backend feeds it into the text-to-speech server and forwards the resulting audio to the browser. Every hop is streaming, which is why latency, not throughput, is the design pressure. The backend is the only component that talks to all three model services, so it is also the single place where turn detection, buffering and ordering have to be correct.

Swapping the LLM: OpenRouter in production, Gemma 3 1B by default

The README is explicit that any text LLM works. Kyutai serves GPT OSS 120B through OpenRouter for the hosted unmute.sh instance, and notes that you can host your own with VLLM. The default local configuration in docker-compose.yml uses google/gemma-3-1b-it. That default is a deliberate choice for footprint, not quality: a 1B model keeps the whole stack inside 16 GB of VRAM. There is a documentation inconsistency worth flagging. The section titled LLM access on Hugging Face Hub instructs you to accept conditions on the Mistral Small 3.2 24B model page, while the surrounding text and the compose default both reference Gemma 3 1B. If you follow the numbered steps literally and then run the default compose file, you may have gated the wrong repository. Check which model your configuration actually pulls before you spend time on token permissions.

Getting it running with Docker Compose

Docker Compose is the path the README recommends, on the grounds that Unmute is many services that must run simultaneously and Compose gives you one command and a reproducible environment. The hardware floor is a CUDA-capable GPU with at least 16 GB VRAM on x86_64. There is no aarch64 support planned, and the README states that native Windows and macOS are not supported, pointing at issues #84 and #74 respectively. Linux or Windows with WSL is the supported surface. You need Docker Compose and the NVIDIA Container Toolkit, and the README offers a check for the latter: sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi. You also need a Hugging Face token exported as HUGGING_FACE_HUB_TOKEN, verifiable with echo $HUGGING_FACE_HUB_TOKEN. Then docker compose up --build. The README warns against write-scoped tokens for public deployments, since a compromised server would hand an attacker write access to your Hub models and datasets. If you hit memory limits, the compose file carries NOTE comments marking the values to adjust.

Three GPUs is a latency decision, not a capacity one

On unmute.sh, Kyutai runs speech-to-text, text-to-speech and the VLLM server on separate GPUs. The README gives the reason and a number: TTS latency drops from roughly 750 ms on a single L40S to around 450 ms with the split. That is the only performance figure in the material, and it describes Kyutai's own deployment rather than a reproducible benchmark. The mechanism for achieving it is a deploy.resources.reservations.devices block with driver: nvidia, count: 1 and capabilities: [gpu], added to the stt, tts and llm services. The README frames this as an improvement over single-GPU operation, which means a one-GPU install is functional but slower on the speech output path. The Dockerless route scales further, listed as one to three GPUs across one to five machines.

Where Unmute is the wrong tool

The deployment table is the clearest limitation. Docker Swarm is documented but explicitly unsupported by Kyutai, included to show how unmute.sh is scaled, with the README stating that debugging multi-node applications is hard and that swarm deployments will not get help. So the multi-machine story exists as a template, not a supported product. The platform restrictions are hard rather than soft: no macOS, no native Windows, no aarch64. If your team develops on Apple silicon laptops, the local loop is unavailable and you are deploying to a Linux host to test anything. The Dockerless path is labelled easy in the table but the README concedes it is more difficult because of dependencies, requiring uv, cargo, pnpm and CUDA 12.1, with the Rust toolchain needed for the tts and stt processes. Anyone treating Unmute as a library to import into an existing Python service will find it is a set of long-running servers instead.

What Unmute is not: a speech model project

The nearest alternative is not another conversation framework but the models themselves. kyutai-labs/delayed-streams-modeling is where Kyutai's STT and TTS live, and the README directs you there if you want either in isolation. The difference in approach is scope: delayed-streams-modeling gives you a model and its inference interface, leaving audio transport, turn detection, LLM routing and browser delivery to you. Unmute gives you those four things wired together and takes a position on the runtime, which is why it insists on Docker Compose and a specific GPU class. A team already running its own websocket audio layer gains little from Unmute and inherits its platform constraints. A team starting from zero on a Linux GPU box gains a working loop. The pre-print at arxiv.org/pdf/2509.08753 covers the models, not the orchestration, so it will not answer questions about the backend's buffering behaviour.

Maintenance, upgrades and the MIT licence

The repository is not archived and the last push recorded is 2026-09-09, but no releases were retrieved, so there is no tagged version to pin. That matters for a project whose Dockerless path depends on a Rust toolchain and a specific CUDA 12.1, because you cannot point at a release and say the dependency set is frozen there. Your upgrade unit is the main branch plus the compose file. The MIT licence covers the code in this repository; it does not cover the model weights, which are downloaded from the Hugging Face Hub under their own terms and, in the Gemma case, a gated agreement you must accept. The README's warning about token scope is a security instruction rather than a licence term, and it is worth following literally: fine-grained read access to gated repos is sufficient for the default setup. None of this is legal advice; read the model cards for the terms that actually bind your deployment.

Editorial conclusion

Adopt Unmute if you have a single x86_64 Linux or WSL machine with a CUDA GPU of at least 16 GB VRAM and you want a working browser voice loop without building the audio transport yourself. Do not adopt it if you need macOS or native Windows, aarch64, or a supported multi-node deployment, because the README rules those out. Before committing, verify that the Hugging Face token you export as HUGGING_FACE_HUB_TOKEN has read-only scope and that the gated model conditions for the LLM you intend to run have actually been accepted on the Hub, since the default docker compose up path depends on both.

Official sources

1. Issues
2. kyutai-labs/unmute on GitHub
3. License: MIT
4. README