OpenVoice

An audio foundation model from MIT and MyShell that copies tone color from a reference clip, works across languages the training set never saw, and went MIT licensed for commercial use in 2024.

V2 of the model arrived with a different training strategy that delivers better audio quality, natively covering English, Spanish, French, Chinese, Japanese, and Korean. In April 2024 both V2 and V1 were released under the MIT license, free for commercial use. That licensing move is as much a part of the story as the modeling itself.

The original release handled accurate tone color cloning and could generate speech in multiple languages and accents. Control was granular: emotion, accent, rhythm, pauses, and intonation could each be adjusted rather than inherited wholesale from the reference clip.

The headline trick is zero shot cross lingual voice cloning. Neither the language being generated nor the language of the reference speech has to appear anywhere in the multilingual training dataset. In practice that means one reference clip in one language can produce speech in another, with the training set never having met either.

The main contributors span MIT, Tsinghua University, and MyShell. The implementation builds on several existing projects, TTS, VITS, and VITS2 among them, and a common issues section links to a QA list for the usual setup troubles. The repo is Python under MIT with its project page at research.myshell.ai and around 37,100 stars.

OpenVoice has powered the instant voice cloning capability of myshell.ai since May 2023, and the model has been used tens of millions of times by users worldwide. The numbers come from the project itself, but they still say something: this is not a demo that works twice and falls over.

• Source 1 · Project repository
• Source 2 · README