IndexTTS-vLLM puts the GPT stage of IndexTTS on vLLM and leaves the rest alone

One stage is replaced, the rest of the pipeline is untouched

The project description is short: vLLM support added to IndexTTS for faster inference. The README body says more precisely that it reimplements the inference of the gpt model with the vllm library, on top of the index-tts project, and that this accelerates the inference process of index-tts.

That scope is the most important fact about the project. IndexTTS is a text to speech system with several stages, and this fork does not replace it. It replaces one stage, the autoregressive GPT model that produces the speech conditioning, with an implementation on vLLM. Everything else, the reference audio handling, the s2mel stage that turns the conditioning into a spectrogram, the vocoder, stays as it was in the upstream project.

That is why the change is attractive in practice. The gpt stage is the one that benefits from continuous batching, paged attention and a scheduler written for throughput, because it is a transformer that runs many similar requests against the same weights. The other stages are not transformer decoders, and vLLM has nothing to offer them.

The repository layout matches the structure. The indextts/ directory holds the package, the api_server.py and api_server_v2.py files are the two FastAPI servers for the two model generations, webui.py and webui_v2.py are the two Gradio interfaces, and patch_vllm.py sits alongside them because the vLLM integration is not entirely stock. There is also a Dockerfile, a docker-compose.yaml, an entrypoint.sh and a set of example audio files under examples/.

The speedup is measured on one card and two model versions

The README reports the improvement for Index-TTS v1 and v1.5 on a single RTX 4090, and the numbers are given in three forms.

For a single request, the Real-Time Factor goes from about 0.3 to about 0.1. For a single request, the decode speed of the gpt model goes from about 90 tokens per second to about 280 tokens per second. And for concurrency, with gpu_memory_utilization set to 0.25, which is about 5GB of VRAM, roughly 16 concurrent requests are handled without pressure, with the measurement script referenced as simple_test.py.

These are the project's own measurements, on one consumer GPU, for two specific model versions. The context that matters is that a real time factor of 0.1 means the synthesis is ten times faster than real time, which is the number that matters if you are serving a queue of requests rather than making one file.

The two other numbers are the ones to keep in mind when planning capacity. The memory fraction of 0.25 leaving 16 concurrent requests is a useful default rather than a limit, and the decode speed of 280 tokens per second is what determines how quickly a single long utterance is produced, since a long utterance is mostly decode.

The concurrency figure also has a caveat attached by the project's own TODO list, which is discussed below.

The bottleneck moved to s2mel and its 25 step DiT loop

The TODO list at the top of the README is short and it is the most useful part of the document for anyone with throughput problems.

The first item is concurrency optimisation of the v2 API. The README states that currently only the gpt2 model inference is parallel, that the other modules are serial, and that the s2mel inference overhead is large, since it needs 25 DiT iterations, which greatly affects concurrency performance. The second item is simply accelerating s2mel inference.

So the acceleration story has two halves with different maturity. The gpt stage is parallel, batched and measured. Everything downstream of it is serial, and the specific cost is a diffusion transformer running 25 denoising steps per request.

This is a coherent picture rather than a contradiction. Moving the transformer stage onto vLLM removes the bottleneck that limited a single request. Once you run many requests concurrently, the serial stages become the limit, and a 25 step iterative stage is a poor fit for concurrency because each request occupies the GPU for the whole diffusion trajectory.

The practical consequence is that the 16 concurrent figure quoted for the v1 and v1.5 models should not be assumed to hold for the v2 path, where the README says the acceleration of the gpt stage was initially ineffective and only fixed in a later change, on 2025-09-29. Anyone deploying this for v2 has to measure the concurrency themselves.

Installation needs a pinned vLLM and a dependency override

The install steps are explicit, and the third one carries a warning that is worth reading before you start.

The first step is the clone.

git clone https://github.com/Ksuriuri/index-tts-vllm.git
cd index-tts-vllm

The second is a conda environment on Python 3.12.

conda create -n index-tts-vllm python=3.12
conda activate index-tts-vllm

The third is the dependency install, and the README explains that it uses a forced overwrite to work around a protobuf version conflict between vllm 0.16.0 and descript-audiotools 0.7.2.

pip install uv
uv pip install -r requirements.txt -c overrides.txt

The requirements file pins vllm at exactly 0.16.0 and pulls in the pieces the pipeline needs, including librosa, gradio for the web UI, modelscope for the weight download, sentencepiece, omegaconf, munch, loguru and ninja. The platform conditionals are handled in the same file, with WeTextProcessing on everything except Darwin and wetext on Darwin, which is the kind of detail that would otherwise cost an afternoon.

The protobuf conflict is the thing to expect to go wrong. It is why the override file exists, and why the instructions say to install with uv and the constraints file rather than a plain pip install of the requirements.

Weights come from ModelScope or from Hugging Face

The weights are published in converted form, so you do not normally have to convert anything yourself. The README marks the automatic download as the recommended path, and gives commands for the three model generations.

# Index-TTS
modelscope download --model kusuriuri/Index-TTS-vLLM --local_dir ./checkpoints/Index-TTS-vLLM

# IndexTTS-1.5
modelscope download --model kusuriuri/IndexTTS-1.5-vLLM --local_dir ./checkpoints/IndexTTS-1.5-vLLM

# IndexTTS2
modelscope download --model kusuriuri/IndexTTS-2-vLLM --local_dir ./checkpoints/IndexTTS-2-vLLM

The Hugging Face route is given for the v2 model.

# IndexTTS-2
huggingface-cli download ksuriuri/IndexTTS-2-vLLM --local-dir ./checkpoints/IndexTTS-2-vLLM

The ModelScope route is described as the recommended one for users in China, which is a small but useful signal about where the project expects most of its users to be.

There is an optional path for converting the original upstream weights yourself, and the README marks it as not recommended.

bash convert_hf_format.sh /path/to/your/model_dir

The container configuration automates the same steps. The .env.example file sets MODEL to IndexTeam/IndexTTS-1.5, turns on VLLM_USE_MODELSCOPE, DOWNLOAD_MODEL and CONVERT_MODEL, points MODEL_DIR at checkpoints, sets PORT to 8001 and GPU_MEMORY_UTILIZATION to 0.25, and the compose file passes that file as the environment for the service.

A web UI per version, and a FastAPI server that speaks OpenAI

There are two front ends, one per model generation, and the choice is made by which file you run.

# Index-TTS 1.0
python webui.py

# IndexTTS-1.5
python webui.py --version 1.5

# IndexTTS2
python webui_v2.py

The README warns that the first start can take a while, because bigvgan has to be compiled for CUDA. That is a build step at first launch rather than at install time, and it is the kind of thing that surprises people on a machine without a compiler or with a mismatched toolchain.

The API is the same for both generations, wrapped with FastAPI.

# Index-TTS-1.0/1.5
python api_server.py

# IndexTTS2
python api_server_v2.py

The startup parameters are documented: --model_dir is required and is the path to the weights, --host defaults to 0.0.0.0, --port defaults to 6006, and --gpu_memory_utilization defaults to 0.25, the same fraction used in the concurrency measurement. The example requests live in api_example.py for v1 and v1.5 and api_example_v2.py for v2.

There is also an OpenAI compatible surface. The README lists an added /audio/speech API path that is compatible with the OpenAI interface, and an added /audio/voices path that returns the list of voices or characters, with a link to the OpenAI createSpeech documentation. For a team that already has code calling a speech endpoint, that compatibility is the cheapest possible migration path.

Quality is measured with WER, and the multi-role feature is experimental

A speedup only matters if the output is still usable, so the README includes a Word Error Rate table on the seed-test evaluation set from BytedanceSpeech, with Chinese and English columns.

The numbers are close together. Human speech scores 1.254 for Chinese and 2.143 for English. The original index-tts with num_beams=3 scores 1.005 and 1.943, the same model with num_beams=1 scores 1.107 and 2.032, and index-tts-vllm scores 1.12 and 1.987. The README's own summary is that it basically maintains the performance of the original project.

The interesting comparison is between the two num_beams settings of the original, because the vLLM version lands close to the beam search configuration rather than to the faster one. Anyone who wants maximum speed can already reduce num_beams, and the table suggests that path has a smaller quality cost than the vLLM change alone would suggest.

The new feature section covers multi-role audio mixing for v1 and v1.5. You can pass several reference audio files, and the output character voice is a mix of those references. The README is candid about the catch: passing multiple reference audios causes the output character voice to be unstable, and you can keep drawing until you get a voice you like and then use that as the reference audio.

For the licensing and maintenance picture, the project is Apache 2.0, the repository also carries an INDEX_MODEL_LICENSE file and a DISCLAIMER, and the last push was on 2026-04-13, the same day as the IndexTTS-vLLM 0.10.2 release. The project is not archived.

Editorial conclusion

IndexTTS-vLLM fits a team that already runs IndexTTS and wants higher throughput from the same GPU, especially for the v1 and v1.5 models where the acceleration is measured. It is the wrong choice for the IndexTTS2 path if throughput is your goal, since the project states that only the gpt2 stage is parallel there and that the s2mel stage dominates the cost. Verify first which model version you are running, read the WER table before you trade quality for speed, and expect to install with the override file because of the protobuf conflict between vllm 0.16.0 and descript-audiotools 0.7.2.

Frequently asked questions

Official sources

1. Issues
2. Ksuriuri/index-tts-vllm on GitHub
3. License: Apache-2.0
4. README
5. Releases