OpenVoice

Last reviewed Aug 26, 2026 · 730 words

Give OpenVoice a reference clip of a few seconds and it will speak any text in that voice, in a language the speaker never used, with the emotion and accent set separately from the timbre. The catch is that this is research code, not an app: there is no Docker image, no web UI, no server process, and the "install" is a conda environment plus a checkpoint download. With 37,297 stars and an MIT licence it is the most-copied of the open cloning models, and it is worth running yourself, but only if you accept that you are going to write the 30 lines of Python that turn it into a tool.

Two models, one trick

OpenVoice splits the job. A base speaker model produces speech with the style you asked for (emotion, accent, rhythm, pauses), and a separate tone colour converter then reshapes that audio to match the reference speaker. That decoupling is why it clones from short clips: the converter only needs to learn timbre, and timbre is cheap to extract. It is also why you control style explicitly instead of hoping the reference clip happened to sound cheerful. Version 2 swapped the base model for MeloTTS and added English, Spanish, French, Chinese, Japanese and Korean as base languages; the clone can target any of them regardless of what language the reference clip was in.

The setup is Python, and Python is the hard part

git clone https://github.com/myshell-ai/OpenVoice
cd OpenVoice
conda create -n openvoice python=3.9 -y && conda activate openvoice
pip install -e .
pip install git+https://github.com/myshell-ai/MeloTTS.git
python -m unidic download

Then download the V2 checkpoints the README links and unzip them into checkpoints_v2/. The Medium difficulty rating is fair: none of these steps is hard, but each is a place for a version conflict, and the project is not actively smoothing them. Budget 4 GB of RAM as a floor. It runs on CPU, which is fine for batch jobs and painful for anything interactive; a modest NVIDIA card with 4 to 6 GB of VRAM gets a sentence down to a few seconds.

What the output is actually like

Quality is good for a 2024 model and audibly synthetic against current commercial services. Timbre match is strong; prosody is where you hear the base model rather than the person. Expect artefacts on reference clips under 5 seconds and on noisy recordings, so give it 10 to 20 seconds of clean speech. Cross-lingual cloning works but with an accent that drifts toward the base speaker for that language. If you need the most natural English voice for a home assistant, a purpose-built TTS such as Coqui TTS may beat it; OpenVoice's edge is cloning specifically.

Turning it into something a stack can call

Because there is no server, the practical pattern is a thin FastAPI wrapper that loads the models once and exposes an endpoint per voice. Pre-compute the tone colour embedding for each voice you care about and store it; extracting it is the slow step and it only needs doing once per speaker. Pair it with faster-whisper for the listening half and you have a fully local speech loop. For chat, Open WebUI can speak replies through an OpenAI-compatible TTS endpoint, and it takes about 40 lines to make your wrapper answer /v1/audio/speech in that shape.

The part nobody puts in the README

Cloning a voice from a short clip is easy enough that the consent question is yours to answer, not the software's. Clone your own voice, voices you have permission for, or synthetic ones. The model card asks for the same and I would not treat that as boilerplate.

What I'd do

Set it up in a dedicated conda env on a machine with a small GPU, record 15 seconds of your own clean speech, and generate one paragraph in English and one in a second language to hear the trade-off for yourself. If cloning is the requirement, keep it and wrap it in FastAPI. If you only need a good default voice, skip the conda dance and run a TTS server built for the job; the AI category lists the ones that ship Docker images.