YouDub WebUI: self-hosted AI video dubbing from YouTube and Bilibili

What YouDub WebUI actually localizes, and for whom

YouDub WebUI turns a single video into a target-language version. You import a YouTube link, a Bilibili link, or a local file; the tool transcribes the speech, translates the subtitles, and then renders one of three outputs: a hard-subtitled video that keeps the original audio, a dubbed video with no hard subtitles, or a video with both hard subtitles and dubbing. In dubbing mode it also separates vocals from background audio, generates the dubbed speech, and mixes the two back together. The finished file plays in the web interface and can be downloaded.

The audience is narrow and clearly stated. The core mature path is English YouTube to Chinese dubbing. Bilibili Chinese to English dubbing is supported. Local video Japanese to Chinese dubbing exists, but the README says that direction has only passed automated parameter-chain and regression tests and has not been validated with real Japanese media. If your source language is Japanese, treat the output as unverified.

The project is not a demo. The README says the author's Bilibili channel, 黑纹白斑马, uses YouDub WebUI to translate and dub its entire catalogue across tech, gaming, science, animals and history. That is a production claim from the maintainer, not an independent benchmark. What it does tell you is that the pipeline runs end to end on real long-form video rather than only on clips.

The pipeline: yt-dlp, FunASR, an LLM translator, Demucs and VoxCPM

The data flow follows the task stages. Media acquisition uses yt-dlp, which is why the README requires a proxy and a Netscape-format YouTube cookie for YouTube sources. Speech recognition is handled by FunASR, with the default model listed as iic/SenseVoiceSmall and a VAD model of fsmn-vad. Translation goes through an OpenAI-compatible Chat Completions endpoint: OPENAI_BASE_URL, OPENAI_API_KEY and OPENAI_MODEL, with gpt-4o-mini as the example default. If OPENAI_API_KEY is empty, the configuration offers Atlas Cloud aliases instead.

Dubbing splits into two tracks. Demucs, pulled in as a git submodule rather than a plain dependency, separates vocals from background music and effects. Speech synthesis uses VoxCPM, with VOXCPM_MODEL defaulting to OpenBMB/VoxCPM2 and a VOXCPM_MIN_REFERENCE_MS floor of 1200 for the reference audio used in voice cloning. Audio is stretched and mixed with audiostretchy, pydub, librosa and soundfile, then rendered through FFmpeg.

The service layer is FastAPI on the backend and a Next.js app under apps/web. The root package.json defines the run scripts: dev:api starts uvicorn on port 8000, dev:web starts the frontend, and test:backend runs pytest against backend/tests. That split matters for debugging, because you can exercise the API without the browser.

Installing YouDub WebUI and running a first dubbing job

The README targets Windows 10/11 with PowerShell 5.1+ first, then Linux, WSL2 and macOS. You need Python 3.12, Node.js 20+, FFmpeg and ffprobe on PATH, a proxy for YouTube, a Netscape-format YouTube cookie, and an OpenAI-compatible endpoint. Clone with the submodule, because Demucs is vendored that way:

git clone https://github.com/liuzhao1225/YouDub-webui.git
cd YouDub-webui
git submodule update --init --recursive

Create the virtual environment and install the Python and frontend dependencies. The README uses the Aliyun PyPI mirror and the npmmirror registry:

python3.12 -m venv .venv
.venv/bin/python -m pip install -U pip
.venv/bin/pip install -i https://mirrors.aliyun.com/pypi/simple/ -r requirements.txt
(cd apps/web && npm ci --registry=https://registry.npmmirror.com)

If you have an NVIDIA GPU, install the CUDA PyTorch wheels before requirements.txt, using requirements-pytorch-cu128.txt. Without them, set DEVICE=cpu in .env. Copy the example environment file and fill it in:

cp env.txt.example .env

The backend enforces authentication and refuses to start when YOUDUB_AUTH_PASSWORD_HASH is missing. Generate the Argon2id hash interactively so the plaintext password never reaches your shell history:

.venv/bin/python -c "from getpass import getpass; from pwdlib import PasswordHash; print(PasswordHash.recommended().hash(getpass('YouDub password: ')))"

Paste the printed hash into .env. On Windows, point FFMPEG_PATH and FFPROBE_PATH at a shared build whose directory contains the av*.dll files. Start the API and the web app from the repository root:

npm run dev:api
npm run dev:web

Then open the frontend, log in with the password you hashed, paste a YouTube URL, pick the target language and output mode, and submit. Expect the first run to download ASR, TTS and audio models, which the README says takes both disk space and network time.

Where YouDub WebUI breaks: FFmpeg builds, CUDA, and Japanese

The most common failure is the FFmpeg build on Windows. YouDub uses TorchCodec, and the README is explicit that a static build containing only ffmpeg.exe, ffplay.exe and ffprobe.exe will not work. The directory must contain avcodec-*.dll, avformat-*.dll and avutil-*.dll. The startup code reads FFMPEG_PATH, checks for av*.dll beside it and calls os.add_dll_directory(), because Python 3.8+ does not pick those libraries up from PATH alone. The README notes that misconfiguration is reported at startup rather than mid-job, which is the right place for it.

Hardware is the second constraint. DEVICE=cpu runs part of the pipeline, but the README states that full transcription, separation and TTS become very slow. DEVICE=mps forces Whisper back to CPU on purpose, to avoid MPS float64 limits, so Apple hardware does not get accelerated Whisper. On Windows CUDA, VoxCPM's torch.compile path needs Triton; without it you see a warning and TTS falls back to the uncompiled path, which the README links to Issue #127 describing it as roughly twice as slow. triton-windows is not a default dependency and you must match it to your PyTorch version yourself.

The third limit is language coverage. Japanese to Chinese has not been validated with real Japanese media, so the regression suite passing is not evidence that the output sounds right. And every translation depends on the external Chat Completions endpoint you configure. If that endpoint returns malformed or over-long subtitles, the problem is upstream of YouDub, and the repository does not document a rollback path for a half-finished job.

How YouDub WebUI differs from KrillinAI and Linly Dubbing

KrillinAI and Linly Dubbing appear in the searches people run around this project, and both sit in the same video localization space, but the architecture differs in ways that affect operations. YouDub WebUI is a two-process application: a FastAPI backend and a Next.js frontend, started separately with npm run dev:api and npm run dev:web, with authentication enforced by an Argon2id password hash in .env. That is a self-hosted service you log into, not a command you fire and forget.

The voice stack is the sharper difference. YouDub uses VoxCPM for synthesis and Demucs for stem separation, so background music and effects survive the dub, and VOXCPM_MIN_REFERENCE_MS controls how much reference audio the clone needs. Tools that do not separate stems before synthesis tend to flatten the mix. The trade-off is dependency weight: Demucs arrives as a submodule you must initialize, and the requirements list pulls in openunmix, spacy, librosa and openai-whisper alongside FunASR.

YouDub also pins its scope. It does not claim a broad language matrix; it claims English to Chinese as the mature path, Chinese to English as supported, and Japanese to Chinese as untested. If you need many source languages with equal confidence, that honesty is a warning rather than a feature.

Maintenance, licence and upgrade cost

The last push to the default branch was on 2026-09-08, which is recent enough that the repository is not stale. It is not archived. The only release listed is demo-assets from 2026-04-22, which the README describes as downloadable sample output rather than a versioned software release. There is no tagged version to pin, so an upgrade means pulling main and re-running pip install -r requirements.txt plus npm ci in apps/web.

The Apache-2.0 licence applies to the repository and is declared in both LICENSE and package.json. That covers the code YouDub ships. It does not automatically cover the models the pipeline downloads: FunASR's iic/SenseVoiceSmall, OpenBMB/VoxCPM2 and the Demucs weights come from their own projects under their own terms, and the README does not restate those terms. If you plan to publish dubbed video commercially, check each model's licence separately. This is not legal advice.

Operational cost is mostly local. Models are cached under MODEL_CACHE_DIR, defaulting to ./data/modelscope, and per-task media and intermediates land under WORKFOLDER. Both grow with use, and the README warns not to commit API keys, cookies, downloaded videos or generated artifacts. Because translation is metered by your OpenAI-compatible provider, the recurring cost that scales with volume is the LLM calls, not the local compute.

Editorial conclusion

Adopt YouDub WebUI if you publish English YouTube content and want Chinese dubbing without uploading media to a third party, and you are willing to run Python 3.12, Node 20, FFmpeg with libass, and a CUDA GPU. Do not adopt it if you need a managed service, if your source language is Japanese and you expect validated quality, or if CPU-only hardware is your only option. Verify three things before committing: that your FFmpeg build is shared/full-shared with av*.dll present, that YOUDUB_AUTH_PASSWORD_HASH is set or the backend refuses to start, and that your OpenAI-compatible endpoint returns usable translated subtitles for your target language.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. liuzhao1225/YouDub-webui on GitHub
4. README
5. Releases