Kokoro-FastAPI: a Dockerized OpenAI-compatible TTS server for Kokoro-82M

What Kokoro-FastAPI actually solves

Kokoro-82M is a small text-to-speech model. Running it directly means writing Python that loads weights, handles phonemization, picks a voice tensor and writes a WAV file. That is fine for a notebook and awkward for anything with more than one caller. Kokoro-FastAPI turns the model into a network service: a FastAPI application that exposes an OpenAI-compatible speech endpoint, so an existing client that already posts to an OpenAI-style audio route can point at your host instead. The README frames the project as a "Dockerized FastAPI wrapper for Kokoro-82M text-to-speech model".

The intended audience is engineers building voice features who do not want to depend on a hosted TTS vendor. The README lists community projects that use it as a backend, and they cluster around a few shapes: home automation (the wyoming_openai and openai_tts integrations for Home Assistant), readers and audiobook pipelines (openreader, epub_to_audiobook), and chat front ends (SillyTavern, OpenWebUI). Those are all cases where someone wants speech generated on hardware they control, behind an interface their existing tool already understands.

The language list is a real constraint rather than a footnote. The README names English (US and GB), Spanish, French, Hindi, Italian, Japanese, Brazilian Portuguese and Mandarin Chinese. If your product needs a language outside that set, this project is the wrong layer to fix it in.

How the wrapper is put together

The repository separates the service from the runtime. The api/ directory holds the FastAPI application, web/ holds the optional read-along UI, and docker/ holds per-platform compose files for cpu, gpu and rocm. The pyproject.toml pins the model stack: kokoro==0.9.4 and misaki[en,ja,ko,zh]==0.9.4, with torch supplied through optional extras rather than as a base dependency. That split is why the same application code runs on CPU, on CUDA, and on ROCm without a branch inside the request handler.

The extras encode hardware policy. The gpu extra installs torch 2.8.0+cu126 on x86_64 and torch 2.8.0+cu129 on aarch64. A separate gpu-cu128 extra exists for Blackwell and RTX 50-series cards, and the comment in pyproject.toml explains why it is not the default: the cu128 wheels carry sm_120 kernels but drop Maxwell and Pascal support, so making them the default would break older cards. That is a deliberate trade-off, and the comment references issue 443 for the reasoning.

On top of plain synthesis, the README describes features that go beyond a thin HTTP shim: inline multi-speaker generation, voice mixing with aliasing for weighted combinations, SSML support, phoneme endpoints in both directions (text to phonemes, phonemes to audio), and timestamped captions at per-word or per-chunk granularity. Custom voicepacks come from a separate project, inno-kokoro, which appears in the dependency list as inno-kokoro==0.2.0.

Installing Kokoro-FastAPI and generating your first clip

The fastest path is a prebuilt image with models baked in. The README warns that :latest exists but recommends pinning to a release tag for stable usage, so substitute a tag such as v0.9.0 if you are deploying anything you care about. This command runs the CPU image and publishes port 8880 on the host.

# how to install kokoro-fastapi on a machine without a GPU
docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest

For an NVIDIA card the README gives a GPU variant that passes through the GPU device. The RTX 50-series and Blackwell cards need the cu128 tag instead, because the default GPU tag ships cu126.

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest

If you prefer to build from source, the repository ships compose files per platform. Clone the repository, change into the matching docker directory, and bring it up. The README notes that models auto-download, and that Apple Silicon users should use docker/cpu because the GPU image is CUDA-only.

git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI
cd docker/gpu   # or docker/cpu, or docker/rocm for AMD
docker compose up --build

There is also a path that skips Docker entirely, which people search for as running kokoro-fastapi without Docker. The README documents a direct run via uv, and start-gpu_mac.sh for native MPS acceleration on Apple Silicon. The repository root contains start-cpu.sh, start-cpu.ps1, start-gpu.sh, start-gpu.ps1 and start-gpu_mac.sh for those entry points.

Once the service is listening on 8880, the useful first check is the voice list, because every request names a voice and a typo produces a failure that looks like a model problem. The README points at docs/configuration.md for environment variables and volume mounts; it does not document the voice-list route in the excerpt available here, so confirm the exact path against the running service before scripting against it.

The examples directory is the practical reference for client code. It contains captioned_speech_example.py for timestamped output, openai_streaming_audio.py and simul_openai_streaming_audio.py for streaming, phoneme_examples/ for the phoneme routes, and voice_samples/ with audio you can compare against.

Where Kokoro-FastAPI is the wrong tool

The clearest limitation is platform coverage on the AMD side. The README labels the ROCm image experimental and linux/amd64 only, so anyone on an ARM server with an AMD accelerator has no supported image. The CPU image works there, but you are then paying CPU inference cost for the whole workload.

Apple Silicon has a similar shape. There is no GPU image for it, because the Docker GPU image is CUDA-only. You either run the CPU container or step outside Docker and run natively via uv with start-gpu_mac.sh. That is a real fork in your deployment process, not a configuration flag.

Second, the project is a wrapper, not a model. Anything the underlying Kokoro-82M checkpoint cannot say, this service cannot say. If you need voice cloning from a reference clip, the README points at a separate project for custom voicepack generation rather than describing an in-service cloning endpoint. If you need languages outside the eight listed, no amount of API polish helps.

Third, the README does not document a rollback procedure for model or image upgrades, and it does not describe a health or readiness endpoint in the excerpt available here. If you are putting this behind a load balancer, verify those details against the running service rather than assuming them. Pinning to a release tag is the mitigation the README itself suggests.

Finally, this is not a hosted service. There is no uptime commitment, no capacity guarantee, and no support contract. You are operating a model server. If your team has no one who wants to own a GPU box, a hosted TTS API is the more honest choice.

Kokoro-FastAPI compared with calling Kokoro directly

The obvious alternative is not another server, it is the kokoro Python package itself. The README's own dependency list includes kokoro==0.9.4, which means the wrapper sits on top of the same library you would use in a script. The difference is what each one gives you for free.

A direct script gives you a function call and a tensor of audio. You own phonemization choices, batching, concurrency, streaming, and the HTTP layer if you need one. Nothing is hidden, and there is no container to build.

Kokoro-FastAPI gives you the HTTP layer already shaped like the OpenAI audio API, plus the features that are tedious to write yourself: multi-speaker mixing with weighted aliases, SSML parsing, phoneme conversion in both directions, and timestamped captions at per-word or per-chunk granularity. It also gives you a deployment story, with prebuilt multiplatform images for CPU, CUDA and ROCm and a Helm chart under charts/ for Kubernetes. That last point matters if you are the person who has to get it running on a cluster.

The trade-off is control. When something sounds wrong, you are debugging through a service boundary. When you need a preprocessing step that the API does not expose, you are either patching the wrapper or running a second service in front of it. For a single-user script that generates an audiobook once a week, the direct library is simpler. For anything with multiple callers, the wrapper earns its place.

Licence, upgrade cost and maintenance signals

The repository is Apache-2.0. That is a permissive licence, and the practical implication is that you can ship the service inside a commercial product without a copyleft obligation on your own code. It does not settle the model question: the Kokoro-82M weights live on Hugging Face under their own terms, and the README links to a specific model commit. Check the model card separately before you ship. This is not legal advice.

The dependency surface is the real upgrade cost. The pinned set includes torch, transformers, spacy with an en_core_web_sm wheel fetched from a GitHub release URL, misaki with four language extras, phonemizer-fork, and espeakng-loader. A torch major bump ripples through the CUDA and ROCm images, which is exactly why the project carries separate cu126 and cu128 extras with a comment explaining the kernel coverage difference. Upgrading the model stack is not a one-line change.

On maintenance, the last push to the default branch was on 2026-09-10, and the most recent release, v0.9.0, was published the same day, following v0.8.2 on 2026-09-05 and v0.8.1 on 2026-08-24. The repository is not archived. The version in pyproject.toml reads 0.9.1-rc1, so a release candidate is in flight. Release cadence has been frequent, but the project is maintained by a single account with community integrations rather than a foundation, and the README does not describe a support policy or a long-term release branch. Pin your image tag and read CHANGELOG.md before moving it.

Editorial conclusion

Adopt Kokoro-FastAPI when you already have a client that speaks the OpenAI audio API and you want the model to run on your own hardware, particularly if you need per-word caption timestamps or multi-speaker mixing that a plain Kokoro script does not give you. Skip it if you need a hosted service with an uptime commitment, if your target language is outside the eight the README lists, or if you cannot accept that the AMD ROCm path is labelled experimental and limited to linux/amd64. Before you commit, run the CPU image on a representative paragraph, confirm the voice names your client sends actually exist in the served list, and check whether the caption format your front end expects matches what the timestamped endpoint returns.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. remsky/Kokoro-FastAPI on GitHub