SonicVale: multi-character, multi-emotion AI dubbing for text and video

What the platform does: from a script to a cast of voices

SonicVale is an open source platform for AI dubbing with multiple characters and multiple emotions, aimed at novels, scripts and video, with automatic dubbing and export. The software is presented under a Chinese name as well, and its identity page gives the positioning as an AI speech synthesis and dubbing service for content such as novels, scripts and video.

The feature list is what makes the scope clear. It imports text from a novel or a script. It manages a library of characters across a project. It selects emotional voices and binds them to characters. It splits the dialogue automatically and generates the dubbing. It manages batch tasks and exports the result. It lets you choose and call a custom LLM endpoint, and it provides a multi-emotion TTS service based on Index-TTS-2.0. After generation it offers precise audio editing, where a clip can be deleted or a silent segment inserted. And it supports custom prompts so that the splitting can be adapted to a particular kind of text.

That last point, the custom prompts for splitting, is the feature that tells you what the LLM is actually for. The text is not simply chopped on punctuation; an LLM reads the text and decides who speaks which line, which is the step that makes multi-character dubbing possible at all for prose without speaker labels.

The screenshot list in the README follows the workflow exactly: an LLM configuration screen, a TTS configuration screen, a voice management screen, then project creation, chapter creation, chapter content import, automatic line splitting, role binding where the same character's voice is shared across chapters, line editing, dubbing generation, and finally the editable audio. That sequence is the product.

The repository is small in structure, with a SonicVale/ directory holding the backend application and a sonicvale-front/ directory for the Electron front end, plus image assets and committed log files for both the back end and the front end.

The stack: Electron and Vue in front, FastAPI and an LLM behind

The technology stack is listed in four lines, and each line is a choice you inherit.

The front end is Electron with Vue and Element Plus, which is the shape of a desktop application that is also a web application. The back end is FastAPI in Python. The AI interface is a large language model that speaks the OpenAI API protocol, so any provider or local model that exposes that protocol can be plugged in. And the TTS service is IndexTTS-2, either deployed through the Cloud Native Build platform, which the README says offers free H20 GPU support, or through a local deployment bundle.

The choice of an OpenAI-compatible endpoint for the LLM is the flexible one. It means the splitting model can be a hosted API, a local inference server, or anything in between, without changing the platform, and it means the platform is not tied to one vendor's pricing.

The back end layout is described in the README, and it reads like a layered FastAPI service. The core/ directory holds the global configuration, the TTS engine, the LLM engine, the ffmpeg wrapper, subtitle generation, the websocket layer and an async queue. Then there are db/ for the database connection and base, models/ for the ORM models, dto/ for data transfer objects used to validate requests and responses, entity/ for entity classes that combine the ORM with the business layer, repositories/ for the database access wrappers, services/ for the core business logic, routers/ for the FastAPI routes, and main.py as the entry point.

Two of those deserve a note. The async queue is what makes batch generation of a long chapter practical rather than blocking a request thread, and the subtitle generation in core is why the export can carry more than audio. The ffmpeg wrapper is what lets the tool cut and pad the generated audio without shelling out ad hoc.

The editing loop: what you can change after generation

The line editing screen is described in detail, and it is where the product stops being a demo.

In the line editing area the user can manually modify the line text, the character attribution, the emotion type and the emotion intensity, and the changes are saved and updated automatically. So the automatic splitting is a starting point rather than a decision, and a human can reassign a line to the wrong speaker, rewrite a line that read badly, or dial the emotion up or down without regenerating the whole chapter.

The audio editing after generation is described as precise, with support for deleting an audio segment or adding a silent segment. That is a small feature that changes how the tool is used: a generated take with one bad line does not have to be regenerated, since the clip can be removed and replaced, and a pause can be inserted where the original text had a paragraph break that the TTS did not honour.

The role binding model is also worth understanding. The screenshot list describes role binding with the same character's voice shared across chapters, so a character defined once in a project keeps its voice in every chapter of that project. For a serialised work, that consistency is the difference between a dubbing that sounds like one production and a set of chapters that sound like separate recordings.

And because the dubbing is produced by a TTS service that can be pointed elsewhere, the emotion dimension is a property of the engine rather than of the platform: the platform supplies the per-line emotion binding and the UI for it, and IndexTTS-2 is what turns an emotion tag into an audio contour.

Running it locally: ffmpeg, uvicorn and the Electron front end

The quick start has three steps, and the first one has a prerequisite that is easy to miss.

Clone the repository:

git clone https://github.com/xcLee001/SonicVale.git
cd SonicVale

For the back end, the README says you first need to download ffmpeg.exe into app/core/ffmpeg/ffmpeg.exe, taking it from the official ffmpeg builds page or from a mirror, and then copy it into the app/core/ffmpeg/ directory. That is a Windows binary placed by hand, which tells you the intended deployment target for the back end is a Windows machine. The dependencies are then installed and the service started:

cd SonicVale
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8200

The port is 8200, and the --reload flag is the normal development mode for a FastAPI service.

The front end is a separate Node project in its own directory:

cd sonicvale-front
npm install
npm run start

The comments in those lines note that npm install installs the dependencies and that npm run start starts the front end including Electron, so the Electron shell and the Vue application are launched by one command.

The overall shape is therefore a two-process development setup: an API server on port 8200 and an Electron client that talks to it, which is also the shape of the production application.

Licence: AGPL-3.0 plus conditions the project adds on top

The project is released under the GNU Affero General Public License version 3. The README states the standard terms, including that the original licence and copyright notice must be included in distributed copies, that if you modify and publish the software, or provide it over a network such as a SaaS or a web application, you must publish the modified source, and that you may not attach any restriction that conflicts with the AGPL terms.

On top of that, the second development section sets out two additional conditions for anyone building on the project. The first is an attribution requirement: a derivative work must clearly state in its user interface and in its code documentation that it is based on the open source project, and must include a link to the original repository. The second is a commercial use restriction: without written commercial authorisation, no derivative work may be used commercially or offered as a commercial service.

There is a tension between those two clauses and the AGPL text quoted a few lines later, since the AGPL terms forbid adding restrictions that conflict with them, and a prohibition on commercial use is not something the AGPL grants a licensor to withhold. Anyone planning a commercial deployment should resolve that with the project owner rather than assume a reading, and should also weigh the network copyleft clause on its own, which applies whether or not the additional terms are accepted.

The README also carries a disclaimer, in both Chinese and English, stating that the project is for research and educational purposes, that users are prohibited from cloning or imitating voices without authorisation or from infringing on the rights of others, including voice rights, portrait rights, copyright and reputation, and that the developer is not liable for the consequences of use. That last point is the one to keep in mind when the tool is pointed at content you do not own.

Maintenance and where SonicVale is the wrong choice

The releases move in small steps: v1.1.4 on 2026-03-31, v1.1.5 on 2026-04-01 and v1.1.6 on 2026-08-16, which is also the date of the last push to the repository. The project is not archived, and the repository even ships the log files from both the back end and the front end, which suggests a development workflow where the author works from the same checkout rather than a curated release.

There is a documentation dependency worth noticing. The detailed usage guide is not in the repository but on a Feishu wiki, linked from the README, and the repository's own README links the release badge to an older version number than the latest tag. In other words, the user documentation lives outside the source tree, which is convenient for readers but means it can drift from the code without a version check catching it.

The first wrong case is a commercial product. The licence conditions, and the AGPL network clause underneath them, make anything beyond a private or research deployment a licensing decision rather than a technical one.

The second wrong case is a project that needs the TTS to be its own property. The platform's voice capability comes from IndexTTS-2 running elsewhere, so the part that produces the audio is not part of this repository, and if that service is unavailable or changes its licence, the platform does not stand on its own.

The third wrong case is a Linux-only or server-only workflow. The documented back end setup expects a Windows ffmpeg binary placed by hand in the application directory, and the front end launches Electron, so the documented path is a desktop on Windows rather than a headless service.

The realistic alternative is to compose the same pipeline yourself: an LLM with an OpenAI-compatible endpoint for the splitting, a TTS engine for the voices, ffmpeg for the audio edits, and a text editor for the script. The difference in approach is that SonicVale supplies the layer between them, including the role library, the per-line emotion binding, the batch queue and the audio editing UI, which is the part that takes real work to rebuild and the part that a small studio would otherwise assemble by hand.

Editorial conclusion

SonicVale earns its place for someone turning their own writing or licensed material into an audio drama, because the line splitting, the role library, the emotion binding and the post-generation audio editing are all in one tool rather than spread across a script and a voice console. It is the wrong choice for a commercial product built on it, because the licence requires attribution, forbids commercial use without written authorization and requires source disclosure when you offer the software over a network. Verify first that the TTS engine you point it at is one you are licensed to use, that your content does not involve voices you have no right to imitate, and that the licence conditions match the way you intend to distribute anything you build with it.

Frequently asked questions

Official sources

1. Issues
2. License: AGPL-3.0
3. README
4. Releases
5. xcLee001/SonicVale on GitHub