Chatterbox TTS Server: self-hosted OpenAI-compatible speech synthesis with voice cloning
Self-host the powerful Chatterbox TTS model. This server offers a user-friendly Web UI, flexible API endpoints (incl. OpenAI compatible), predefined voices, voice cloning, and large audiobook-scale text processing. Runs accelerated on NVIDIA (CUDA), AMD (ROCm), and CPU.
At a glance
- What is it?
- A FastAPI wrapper around Resemble AI's Chatterbox model family, with a Web UI, predefined voices and Docker images for CUDA, ROCm and CPU. The last push was on 2026-05-26, so judge the maintenance cadence yourself.
- Who is it for?
- Adopt it if you want Chatterbox behind an HTTP API without writing the FastAPI layer yourself, and if you can pin Python 3.10 on the host or accept the Docker images. Do not adopt it if you need a single-maintainer project with a published support window, or if you cannot run the model locally at all.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 113 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 16, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What problem the Chatterbox TTS Server actually removes
Resemble AI's Chatterbox is a model, not a service. Anyone who wants to call it from an application has to load weights, manage a reference-audio cache, chunk long text, expose an HTTP route and keep a process alive. This repository is that missing layer. It wraps the Chatterbox family (Original, Multilingual and Turbo) in a FastAPI application with a browser UI and several HTTP endpoints, one of which is shaped like OpenAI's speech API.
The audience is narrow but real. It is for people who already have a GPU and want speech output on their own hardware: audiobook and narration pipelines, game or agent voice lines, accessibility readouts, and internal tools where sending text to a hosted TTS vendor is not an option. The README frames the use case around voice cloning, large text processing via chunking, and reproducible voices through a generation seed. The Colab notebook exists so you can see the UI before installing anything, which is a reasonable concession to how annoying local TTS setup is.
It is not a general-purpose speech platform. There is no diarization, no speech-to-text, no streaming ASR. It converts text to WAV, and the interesting engineering is all in how it does that reliably across three GPU vendors and a CPU fallback.
The Chatterbox family, the engine split and the chunker
The repository separates concerns in a way worth understanding before you deploy it. `server.py` is the FastAPI application. `engine.py` holds the model-facing logic. `config.py` and `config.yaml` carry runtime settings. `models.py` and `utils.py` hold the supporting pieces. The `ui/` and `static/` directories hold the Web UI, and `voices/` and `reference_audio/` hold the predefined voice assets and user-supplied reference clips.
The model itself is not vendored. The requirements file states plainly that `chatterbox-tts` is not included and must be installed separately with `--no-deps`, because the dependency resolver otherwise pulls ONNX source builds and mismatched torch versions. The Dockerfile installs `git+https://github.com/devnen/chatterbox-v2.git@master` with `--no-deps` alongside `s3tokenizer==0.3.0` and `onnx==1.16.0`, then pins `protobuf>=4.25.0` to resolve a conflict the comments describe between onnx and descript-audiotools. That is a fragile install chain, and the repository acknowledges it rather than hiding it.
Two mechanisms are worth calling out. First, voice conditioning caching: repeated requests against the same reference voice skip re-encoding, which matters for batch workloads and anything hitting the OpenAI-compatible endpoint in a loop. Second, the chunker, which splits large text into processable pieces. The v2.0.0 notes mention a fix where stray dashes in narrative prose were treated as bullet items and swallowed the rest of a paragraph, which tells you the chunker is heuristic rather than a full parser. Long documents are the intended use case, so any chunking bug surfaces as missing audio, not as an error.
Installing the server with Docker and generating a first clip
The Docker route is the least ambiguous. The compose file builds from `Dockerfile` with a `RUNTIME` build argument that defaults to `nvidia`, maps port 8004, and mounts `config.yaml`, `voices`, `reference_audio`, `outputs` and `logs` from the host. It also declares a named volume for the Hugging Face cache so models survive container rebuilds. Set `HF_TOKEN` in the environment before starting if you are pulling gated weights. The README does not spell out a single compose command, so start the stack the way you normally start a Compose project in that directory.
The service listens on port 8004 by default; the compose file reads `${PORT:-8004}` so you can override it. Open the Web UI in a browser, pick a predefined voice from the `voices` directory, and submit a short sentence. The output lands in the mounted `outputs` folder.
For a manual install the requirements file gives the order explicitly, and the order matters because of the `--no-deps` step.
pip install -r requirements.txt
pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master s3tokenizer==0.3.0 onnx==1.16.0Python 3.10 is the only fully supported version, per the README badge and the Portable Mode notes. On Windows, `start.bat` offers Portable Mode, which bundles an embedded Python 3.10 runtime so the folder can be copied to another machine and launched without installing Python. `--portable` skips the prompt; `--no-portable` creates a standard virtual environment. Portable Mode does not exist on Linux or macOS, so those hosts need Python 3.10 present.
Once the server is up, the OpenAI-compatible endpoint is the shortest path from existing code. Point a client at the local base URL and use the speech route as you would with a hosted provider. Voice cloning works by supplying a reference clip through the voice parameters; the v2.0.0 notes state that path traversal attempts on those parameters now return HTTP 400, which is the fix for CWE-22.
Where the Chatterbox TTS Server is the wrong tool
The install chain is the first honest limitation. Because `chatterbox-tts` is installed with `--no-deps`, pip is not resolving the model's own dependency graph for you. The requirements comments attribute this to ONNX source builds and torch version mismatches. The practical consequence is that a torch upgrade on your host can break the engine in ways pip will not warn you about, and the fix is usually to reinstall the pinned trio by hand.
Hardware coverage is broad on paper and uneven in practice. The repository ships separate compose files for CUDA 12.1 and 12.8, CUDA 13.0 for DGX Spark (sm_121), ROCm, RDNA4, Strix Halo (with `HSA_OVERRIDE_GFX_VERSION=11.0.0`), and CPU. Each carries its own requirements file. That is a lot of surface area for a project whose last push was on 2026-05-26, and it means a bug on one stack may sit unfixed while the others move. If your card is not one of the named targets, expect to debug the image yourself.
Latency is the second boundary. BF16 inference is opt-in via `TTS_BF16=on` or `=auto`, and the release notes describe it as roughly 40% throughput on capable GPUs, with `off` as the default to preserve upgrade behavior. If you need that speedup, you are testing a non-default path. Streaming is likewise opt-in: `stream: true` on the `/tts` endpoint returns a `StreamingResponse` that flushes WAV bytes per chunk with 20 ms crossfades, while default behavior is unchanged. Neither is a default you can assume.
Finally, this is a single-maintainer project under the devnen account, with sibling projects for Dia and Kitten TTS. There is no documented support policy, no LTS branch and no stated deprecation process in the repository files. Treat it as software you operate rather than software someone else operates for you.
Chatterbox TTS Server compared with running the model directly
The obvious alternative is Resemble AI's upstream `chatterbox` repository. The difference is scope. Upstream gives you the model and inference code; you write the HTTP layer, the chunker, the reference-voice cache, the UI and the GPU handling. This project is that work already done, at the cost of tracking a fork: the Dockerfile and requirements both install `git+https://github.com/devnen/chatterbox-v2.git@master`, not the upstream package. If upstream changes its API, this server follows on the maintainer's schedule, not yours.
A second alternative is a hosted speech API. The trade is the reverse: no GPU, no CUDA or ROCm image to pick, no Python 3.10 constraint, but your text leaves your machine and per-character costs scale with volume. The OpenAI-compatible endpoint here exists precisely so that switching between the two is a base-URL change rather than a rewrite, which is a sensible design decision regardless of which side you land on.
Within the same author's set, Dia-TTS-Server and Kitten-TTS-Server share the architecture and UI but use different engines. If you are choosing between them, the deciding factor is the model, not the server, since the surrounding application is largely the same.
Licence, upgrade cost and what to check before you deploy
The server code is MIT. That covers the FastAPI application, the UI and the launcher scripts in this repository. It does not automatically cover the model weights or the upstream `chatterbox-tts` package, which come from Resemble AI under their own terms, and it does not cover any reference audio you clone from. Voice cloning in particular carries consent and publicity-rights obligations that no software licence settles; if you clone a real person's voice, that is a separate question from the MIT grant on this code. This is not legal advice, and the repository does not attempt to resolve it.
Upgrade cost is driven by the pinned stack. `torch==2.5.1`, `transformers==4.46.3`, `fastapi>=0.100.0,<0.116.0` and `descript-audio-codec==1.0.0` are pinned for reasons the comments explain, and the GPU variants add their own constraints on top. Moving any one of them means re-testing the `--no-deps` install, because that is the step pip cannot verify for you. The `config.yaml` is mounted from the host in Docker, so settings survive image rebuilds, but a new release can add keys that your existing file does not have. Diff it against the repository copy after each upgrade.
Before deploying, confirm that the model weights are reachable from your network (the compose file expects `HF_TOKEN` and sets `HF_HUB_ENABLE_HF_TRANSFER=1`), that your GPU matches a shipped compose file rather than something adjacent, and that you have a rollback plan for the pinned dependency set, since the README does not document one.
Editorial conclusion
Adopt it if you want Chatterbox behind an HTTP API without writing the FastAPI layer yourself, and if you can pin Python 3.10 on the host or accept the Docker images. Do not adopt it if you need a single-maintainer project with a published support window, or if you cannot run the model locally at all. Verify three things before committing: that the `chatterbox-tts` package installs cleanly against your torch build, that your GPU stack matches one of the shipped compose files, and that the voice cloning consent requirements in the upstream model licence are acceptable for your use case.
Frequently asked questions
What is Chatterbox TTS?
It is Resemble AI's open-source text-to-speech model family, which this server wraps. The repository covers three variants: the original high-quality model, a multilingual model supporting 23 languages, and Chatterbox-Turbo, a 350M-parameter model with paralinguistic tags such as [laugh] and [cough].
What is a chatterbox used for?
In this project the model is used to turn text into speech audio, including voice cloning from a reference clip and audiobook-scale text processed through the chunker. The README describes the server as suitable for narration and voice agents, with reproducible voices via a generation seed.
How does chatterbox work?
It runs the Chatterbox engine behind a FastAPI application, exposing a Web UI plus HTTP endpoints including an OpenAI-compatible speech route. Large text is split by a chunker before synthesis, and repeated requests against the same reference voice reuse a conditioning cache instead of re-encoding.
What is the chatterbox?
Chatterbox is the text-to-speech model family from Resemble AI that this repository self-hosts. The server adds a Web UI, predefined voices, voice cloning and an OpenAI-compatible API on top of the original, multilingual and Turbo model variants.
Community notes