xiaohu-video-translate: a local Whisper pipeline for Chinese subtitles on foreign-language video

What xiaohu-video-translate actually is: skills for an AI coding CLI, not an app

The repository is not a standalone program you launch. It is three skills, each a folder under `skills/` holding a `SKILL.md` instruction file plus scripts, and the README describes the whole thing as "scripts + a skill instruction file". The AI coding tool reads the instruction file and decides which script to run. That means the user-facing interface is a sentence in natural language, not a command line with flags. The README's own examples are phrased that way: "translate this link into a Chinese-subtitled video" plus a URL, or "give this local video Chinese subtitles" plus a path. The three skills split responsibilities. xiaohu-video-md is the orchestrator: download, audio extraction, Whisper transcription, calling the polish step, burning subtitles, emitting Markdown. xiaohu-subtitle-polish handles translation and subtitle cleanup: error correction, translation, line breaking, punctuation removal, timestamp alignment, bilingual ASS. xiaohu-video-download is a plain downloader for video, audio, playlists, or burning subtitles onto a local file. They can be used together or separately. The intended audience is someone who already has an AI coding CLI installed and is comfortable with Homebrew and pip. Anyone who wants a desktop application with buttons will not find it here.

The five-stage pipeline and where the word-level timestamps come from

The README diagrams the flow as download, then audio extraction plus Whisper transcription with word-level timestamps, then translation, then polish, then burn-in, producing a subtitled video and a Markdown transcript. The interesting design decision is the timestamp handling. Subtitle timing is derived from word-level timestamps and then cut on sentence boundaries and pauses, rather than from segment-level output. That is what the README claims prevents subtitles from running ahead of the speaker or splitting a sentence across two cues. The polish stage is where the pipeline stops looking like machine translation. According to the README, it corrects proper nouns that Whisper mishears (the examples given are Claude heard as cloud and MCP heard as NCP), breaks lines semantically, and leaves technical terms in English. Bilingual output uses ASS rather than SRT, because the README states SRT cannot express the size contrast between a large Chinese line and a small English one. Burn-in and watermark happen in a single encode pass, which the README says avoids quality loss from re-encoding. Translation itself is not a separate paid API: it reuses whatever AI the coding CLI already talks to. Whisper auto-detects the source language, and the README says any language Whisper can hear can be turned into Chinese subtitles; Chinese source video skips translation and only produces transcription and transcript.

Installing xiaohu-video-translate on macOS and running a first translation

The README documents macOS as the smoothest path, especially Apple Silicon. The commands below are copied from it. Homebrew is assumed present; the README points to brew.sh otherwise. This block installs yt-dlp and ffmpeg, then the MLX Whisper engine, then clones the repository and runs install.sh, which the README says copies the three skills into `~/.claude/skills/`.

brew install yt-dlp ffmpeg
pip3 install --break-system-packages mlx-whisper
git clone https://github.com/xiaohuailabs/xiaohu-video-translate.git
cd xiaohu-video-translate
bash install.sh

On a Mac without Apple Silicon, the README gives the fallback engine instead of mlx-whisper:

pip3 install --break-system-packages faster-whisper

After install, edit `~/.claude/skills/xiaohu-video-md/config.json` and set `output_dir` to an absolute path. The MLX model downloads automatically from HuggingFace on first run, roughly 1.5GB, and the README says you do not fetch it manually. whisper-cpp is described as an optional backup engine and is not needed by default. Then restart the AI coding tool and phrase the request in plain language. The README's table maps sentences to actions, so a request naming a video URL and asking for a Chinese-subtitled video should trigger the full chain and leave a subtitled video plus a Markdown transcript in your configured output directory.

Windows and Linux: three concrete differences from the Mac instructions

The README is direct that the tool was tuned on a Mac and that Windows has three real differences. First, the transcription engine: MLX only supports Apple silicon, so Windows uses faster-whisper, and the scripts reportedly fall back to it automatically when MLX is absent. The README's native Windows block installs the dependencies like this:

pip install yt-dlp faster-whisper
winget install Gyan.FFmpeg

Second, `install.sh` is a bash script, so native Windows needs Git Bash, or you copy the three folders under `skills/` into your tool's skill directory by hand and rename each `config.example.json` to `config.json`. Third, and most likely to bite, the burned-in subtitle font. The scripts default to PingFang SC, which does not exist on Windows or Linux, and Chinese text renders as boxes. The README says to replace `FontName=PingFang SC` with `FontName=Microsoft YaHei` on Windows and with a Noto Sans CJK family on Linux, and to point the watermark font path at a system font such as `C:/Windows/Fonts/msyh.ttc`. The README also states that automatic cross-platform font selection is still being worked on and invites issues. That is an honest limitation, and it means a Linux or Windows user should expect to edit the burn-in command rather than run it untouched. The WSL route is presented as the least painful Windows option, and the README's WSL commands install ffmpeg via apt and faster-whisper via pip rather than using Homebrew.

Where it breaks: Douyin login, YouTube 403s, and the Chinese-only output assumption

Three failure modes are documented rather than hidden. Douyin requires a one-time interactive login: run the login script, scan the QR code in the browser window, and the session stays on your machine. If Douyin later reports a login error, the README says to rerun that script.

python3 ~/.claude/skills/xiaohu-video-md/scripts/douyin_login.py

YouTube downloads can fail with 403, SABR or PO Token errors; the scripts retry by reading cookies from the browser, Chrome by default, and the README suggests adding a proxy flag if that fails. Subtitle timing that looks wrong on individual clips is attributed to loud background music confusing Whisper, with re-transcription as the remedy. The larger constraint is directional. The project translates into Chinese. The README's own framing is "any foreign language to Chinese", and it says Chinese source video is transcribed but not translated. So the four search questions people actually ask, about translating a Chinese video to English or getting English subtitles for Chinese video, describe the opposite direction from what this pipeline is built for. You could in principle feed the Markdown transcript to your own model, but nothing in the README describes an English or Japanese subtitle output path, ASS styling for other targets, or burn-in of non-Chinese subtitles. Treat it as a Chinese-subtitle tool.

How it compares to running Whisper and ffmpeg yourself

The obvious alternative is the raw toolchain: yt-dlp to fetch, Whisper to transcribe, a translation step of your choosing, then ffmpeg to burn in. That gives you complete control over models, fonts and cue formatting, and it is what this project wraps. The difference is what sits between the stages. Here, the orchestration is an instruction file interpreted by an AI coding CLI, which means the glue logic is not a fixed program but a set of directions the model follows, and the polish step is a model call that fixes ASR mishears and re-breaks lines. A hand-rolled ffmpeg pipeline gives you deterministic output; this one gives you a pipeline that adapts when you say "bilingual" or "no watermark" or "fast mode". The cost is that behaviour depends on which CLI drives it and how well it follows the SKILL.md. The README states the skills are not tied to one tool and lists Claude Code, OpenClaw, Gemini CLI and Codex, with Gemini CLI loading the bundled `gemini-extension.json` and others being fed the `SKILL.md` as rules. If you want reproducibility and already know ffmpeg, the wrapper adds a layer you may not want. If you do not want to write the glue, the wrapper is the point.

Licence, maintenance and what upgrading costs you

The repository is MIT licensed, so the subtitle styling and watermark defaults are yours to change; the README says as much. The one licence-adjacent warning it gives is operational rather than legal: do not commit your own `config.json` or the Douyin login state to a public repository, and `.gitignore` is said to exclude them by default. On maintenance, the last push to the default branch was on 2026-06-08, and the repository is not archived. There are no retrieved releases, so there is no versioned upgrade path to reason about: you track `main`. That matters because the skills live in your CLI's skill directory after `install.sh` copies them there, so pulling new commits means re-running the install or copying the folders again, and any local edits to the scripts (the font name being the likely one) can be overwritten. Keep your changes in `config.json` where possible. The MLX model download is a separate moving part: it comes from HuggingFace on first run, so a fresh machine needs network access once even though transcription afterwards is local.

Editorial conclusion

Adopt it if you already drive Claude Code or a comparable AI coding CLI on Apple Silicon and mostly handle English, Japanese or Korean talks; the install is a git clone plus bash install.sh, and Whisper transcription stays on your machine with no API fee. Skip it if you need a GUI, a Windows-first experience, or translation into anything other than Chinese: the pipeline is built around Chinese output, and the README itself flags the font and engine gaps on Windows and Linux. Before committing, verify two things on your own machine: that mlx-whisper installs and the roughly 1.5GB model downloads from HuggingFace, and that your target language pair survives the polish step, since the README documents correction of ASR mishears but not any evaluation of translation quality.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. xiaohuailabs/xiaohu-video-translate on GitHub