EmotiVoice: Prompt-Controlled TTS With Emotion Tags, and What the Pipeline Actually Requires
EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine
At a glance
- What is it?
- EmotiVoice is an Apache-2.0 PyTorch text-to-speech engine from NetEase Youdao that takes a speaker ID plus a style or emotion prompt and synthesizes English or Chinese speech. It is a two-stage acoustic-model-plus-vocoder pipeline with a Docker quickstart, and its real cost is in the model files and the phoneme preprocessing, not in the install.
- Who is it for?
- Adopt EmotiVoice if you need offline English or Chinese synthesis where the emotion or style is a per-utterance parameter and you can host the models yourself; it is a poor fit if you need Japanese or Korean today, or if you cannot preprocess text into its phoneme format. Before committing, verify two things: that the pretrained checkpoints download completely from the ModelScope or Google Drive links, and that frontend.py emits phonemes your target text does not choke on.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 13 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The Problem EmotiVoice Solves: Emotion as a Per-Utterance Parameter
Most open text-to-speech stacks give you a voice and a speed knob. If you want the same sentence delivered angry in one call and sad in the next, you either train a separate model per style or you accept whatever prosody the model defaults to. EmotiVoice's stated differentiator is emotional synthesis: the README lists happy, excited, sad and angry among the supported emotions, and the inference format makes the emotion a field in the input line rather than a property of the model you loaded. The project also claims over 2000 voices, with a wiki page dedicated to listing them, and it speaks both English and Chinese. The audience is therefore not the generic text-to-speech user but the developer building something where delivery matters: game dialogue, dubbing, audio for a reading app, or a voice interface that should not sound flat. The README positions it as free and open source under Apache-2.0, with an easy-to-use web interface and a scripting interface for batch generation, so the intended workflow covers both interactive auditioning and offline batch runs.
Speaker ID Plus Emotion Prompt: The Inference Line Format
The input is a pipe-delimited line, not free text. The README gives the format as `<speaker>|<style_prompt/emotion_prompt/content>|<phoneme>|<content>`, and the worked example is a speaker number, the word Happy, a phoneme sequence in the project's own bracket notation, and the plain sentence. That means the emotion is selected by a literal token in the second field, and the third field is not the raw sentence but a phoneme string the model consumes directly. The README shows phonemes such as [IH0], [M], [AA1] with an engsp token used as a word separator, and the trailing content field carries the readable text. This design is worth being explicit about: the engine is not doing grapheme-to-phoneme conversion for you at inference time in the batch path. It expects you to have produced that phoneme field beforehand. The prompt field is also described as accepting style, emotion, or content, which suggests the same slot can carry different kinds of conditioning, though the README does not enumerate which values beyond the emotion words it names.
The Phoneme Preprocessing Step You Cannot Skip
The command that turns text into the required third field is `python frontend.py data/my_text.txt > data/my_text_for_tts.txt`. It is a single line in the README and it is the step most likely to break a first integration, because it runs outside the synthesis call and its output has to be spliced into the pipe-delimited line by whatever script drives the engine. The dependency list gives a clue about what is happening inside: the install pulls in `g2p_en`, `jieba`, `pypinyin` and `pypinyin_dict`, plus an NLTK downloader call for `averaged_perceptron_tagger_eng`. So English grapheme-to-phoneme goes through g2p_en and its POS tagger, while Chinese goes through jieba segmentation and pypinyin. That is a reasonable split, but it means the frontend inherits the failure modes of those libraries on unusual text: names, acronyms, mixed-script strings, code, and anything the tagger misreads. The README does not document a fallback or an override mechanism for phonemes the frontend gets wrong, so plan on inspecting the generated file rather than trusting it.
Running It: Docker Ports, the Joint Inference Script, and the API
The fastest path is the published image. With an NVidia GPU and the container toolkit configured, the README's command is `docker run -dp 127.0.0.1:8501:8501 syq163/emoti-voice:latest`, and the updated form also publishes port 8000 for the OpenAI-compatible TTS API: `docker run -dp 127.0.0.1:8501:8501 -p 127.0.0.1:8000:8000 syq163/emoti-voice:latest`. The Streamlit interface is then at http://localhost:8501 and the API at http://localhost:8000. From source, the README uses a conda environment named EmotiVoice on Python 3.8, then `pip install torch torchaudio` followed by numpy, numba, scipy, transformers, soundfile, yacs, g2p_en, jieba, pypinyin and pypinyin_dict, and the NLTK tagger download. The batch path is `python inference_am_vocoder_joint.py --logdir prompt_tts_open_source_joint --config_folder config/joint --checkpoint g_00140000 --test_file $TEXT`, with output landing in `outputs/prompt_tts_open_source_joint/test_audio`. For interactive use there is `streamlit run demo_page.py` after `pip install streamlit`. The API path installs fastapi, pydub, uvicorn[standard] and pyrubberband, then `uvicorn openaiapi:app --reload`. Note that the script name, inference_am_vocoder_joint, tells you the architecture in one word: a separate acoustic model and vocoder run jointly at inference.
Model Files Are the Real Setup Cost
Nothing in the README suggests the repository ships usable weights. You need the SimBERT Chinese text encoder, fetched either with `git lfs install` and `git lfs clone https://huggingface.co/WangZeJun/simbert-base-chinese WangZeJun/simbert-base-chinese`, or from ModelScope with `git clone https://www.modelscope.cn/syq163/WangZeJun.git`. Separately, the pretrained synthesis models come from a Google Drive folder or from `git clone https://www.modelscope.cn/syq163/outputs.git`. The README itself points to a wiki page, How to download the pretrained model files, for people who hit trouble, which is an admission that this step is where users get stuck. Two different hosts, two different clone commands, and a Google Drive folder for the main checkpoints is a fragile combination for automated deployment: Drive links are not a package registry, and nothing in the material describes checksums or versioning for those weights. The `--checkpoint g_00140000` argument in the inference command is a bare filename, so the checkpoint has to sit where the script expects it under the logdir.
Where EmotiVoice Is the Wrong Tool
Language coverage is the clearest boundary. The README says EmotiVoice speaks English and Chinese, and the only entry under features under development is support for more languages such as Japanese and Korean, listed as an unchecked item. If your product needs either of those today, this project does not have you covered, and the roadmap item carries no date. The second boundary is the phoneme requirement. Because the batch interface takes phonemes rather than raw text, any workflow that expects to hand over an arbitrary string and get audio back has to add the frontend step and own its errors. The third is hardware: the Docker quickstart explicitly requires a machine with an NVidia GPU, and while the source install does not restate that, nothing in the README describes a CPU-only path or gives timing expectations. Anyone planning to run this on a laptop without a discrete GPU should treat that as unverified rather than assumed. Finally, the release record is thin: v0.2 in November 2023 and v0.3 in December 2023, with the repository last pushed in 2026 per the metadata but no newer tagged release listed. The Mac app shipped alongside v0.3 in December 2023. That is a long gap between the last tagged release and the last push, and the README does not explain what changed in between.
How It Differs From a Single-Stage Neural TTS Model
The obvious comparison is a modern end-to-end TTS model that maps text straight to a waveform and takes a reference audio clip as the style cue. EmotiVoice splits the job: an acoustic model produces intermediate features and a vocoder turns them into audio, run together by inference_am_vocoder_joint.py. Style and emotion arrive as a discrete token in the input line rather than as an audio prompt. That difference matters in practice. A discrete label is cheap and reproducible: you can put Happy in a config and get the same conditioning every run, with no reference clip to source or license. An audio-prompt approach is more flexible in the styles it can express but requires you to supply and manage reference audio per style. EmotiVoice also offers voice cloning with personal data, documented on the wiki with DataBaker and LJSpeech recipes under data/, so the reference-audio path exists on the training side even though the inference interface is label-driven. The trade-off is that you are limited to the emotion vocabulary the model was conditioned on, and the README names only a handful of emotions plus a generic style or content slot.
Licence, Maintenance, and What to Verify First
The repository is Apache-2.0, which permits commercial use and modification subject to the licence terms, but this article is not legal advice and the licence file is the authority. Two things sit outside that licence and deserve separate checking. First, the pretrained weights are hosted on Hugging Face, ModelScope and Google Drive, and their terms are not stated in the README. Second, the README notes that the HTTP API provides over 13000 free calls and points to additional voices from Zhiyun at ai.youdao.com, which is a hosted commercial service rather than part of the open repository; using it means accepting separate terms. On maintenance, the model download instructions already route users to a wiki page for troubleshooting, and the language roadmap item is open, so the practical upgrade cost is re-downloading checkpoints and re-running the frontend against your text corpus to see whether the phoneme output shifted. Verify the checkpoint download completes and that frontend.py handles your actual text before you build anything on top of it.
Editorial conclusion
Adopt EmotiVoice if you need offline English or Chinese synthesis where the emotion or style is a per-utterance parameter and you can host the models yourself; it is a poor fit if you need Japanese or Korean today, or if you cannot preprocess text into its phoneme format. Before committing, verify two things: that the pretrained checkpoints download completely from the ModelScope or Google Drive links, and that frontend.py emits phonemes your target text does not choke on.
Community notes