VideoCaptioner: an LLM subtitle pipeline you drive from the CLI or a desktop GUI

The gap VideoCaptioner is trying to close

Most subtitle work is not one task. It is four: get words out of the audio with timestamps, cut those words into readable lines, translate them if the audience differs from the speaker, and put them back onto the video. Each step usually means a different tool, a different file format and a different failure mode. VideoCaptioner's README frames the project as covering exactly that chain: speech recognition, subtitle optimisation, translation and video composition in one place. The target user is someone who already knows what an SRT file is and does not want to stitch four utilities together for every video. The CLI is the primary interface for that user; the GUI desktop app exists for the same workflow without a terminal. The README also ships a Claude Code Skill, which suggests a second intended audience: people who want an AI coding assistant to invoke the pipeline on their behalf rather than typing the commands themselves.

The pipeline: ASR, segmentation, LLM pass, translation, muxing

The README states the flow plainly: audio and video in, then speech recognition, then subtitle sentence splitting, then an LLM optimisation pass, then translation, then video composition. Two details in that description matter more than the rest. The first is that the recogniser produces word-level timestamps and uses VAD, voice activity detection, so the segmentation step has something finer than whole utterances to work with. The second is that the sentence splitting is done by an LLM reading the transcript semantically rather than by a fixed character or duration rule. That is the design bet of the whole project: line breaks in subtitles are a comprehension problem, not a timing problem, and a language model is better at guessing where a thought ends. Translation is described as context-aware with a reflection step, meaning the model is given surrounding lines rather than each line in isolation. The README also mentions batch concurrent processing, which is what makes the LLM passes practical on a long video instead of a serial crawl through hundreds of segments.

Free paths versus LLM paths, and what each costs you

VideoCaptioner splits its features into two tiers, and the split is the single most useful thing to understand before installing. The free tier needs no API key at all: the README names 必剪 (Bijian) recognition and Bing or Google translation as working out of the box. The LLM tier covers subtitle optimisation and large-model translation, and requires an OpenAI-compatible endpoint. The transcription command lists five engines: faster-whisper, whisper-api, bijian, jianying and whisper-cpp. That mix is deliberate. faster-whisper and whisper-cpp run locally, bijian and jianying are free hosted services, and whisper-api points at a remote Whisper endpoint. So the recognition stage can be kept entirely on your own machine, while the optimisation and translation stages cannot, unless you skip them. The trade-off is explicit: the feature that gives the project its name, LLM-driven segmentation, is exactly the feature that requires sending your transcript to a third party. The README also links to a VideoCaptioner relay station at api.videocaptioner.cn alongside SiliconCloud and DeepSeek as example providers, which tells you the maintainers expect a hosted endpoint rather than a local model.

Getting it running: install, config keys and precedence

Installation is a single pip command that brings in both interfaces: pip install videocaptioner. The README states that the free features need no configuration and work immediately after install. For the LLM features you set three keys, and the exact names matter: videocaptioner config set llm.api_key, videocaptioner config set llm.api_base, and videocaptioner config set llm.model. The example base URL is https://api.openai.com/v1 and the example model is gpt-4o-mini. Configuration resolves in a documented order: command-line arguments beat environment variables prefixed VIDEOCAPTIONER_, which beat the config file, which beats defaults. That precedence is worth internalising, because it means a stale shell variable can silently override the file you just edited. videocaptioner config show prints the effective values, and videocaptioner config path tells you where the file lives. The everyday commands are short: videocaptioner transcribe video.mp4 --asr bijian for recognition, videocaptioner subtitle input.srt --translator bing --target-language en for translation, videocaptioner process video.mp4 --target-language ja for the full chain, videocaptioner synthesize video.mp4 -s subtitle.srt to burn subtitles in, and videocaptioner download for pulling source video from YouTube or Bilibili. There is also a dub command that produces a dubbed audio track or video from subtitles. Running videocaptioner with no arguments, or videocaptioner-gui, opens the desktop app.

Where the design is thin, and where it will bite

The README is a feature list, not a specification, and several things a careful adopter would want are simply not there. There is no statement about what happens when the LLM returns malformed output, no description of retry or fallback behaviour, and no error taxonomy for the recognition stage. For a pipeline that depends on a remote model to decide where sentences end, the absence of any documented handling for a bad response is the most obvious gap. The second gap is cost and latency: the README says batch concurrent processing is used, but gives no sense of how many requests a one-hour video generates or what a run costs against a metered endpoint. The third is operational. A tool that both downloads video and calls third-party APIs inherits the network conditions of wherever it runs; the free bijian and jianying engines are hosted services, so 'free' here means 'no key', not 'no network'. And the LLM optimisation step is not optional within the process command, so if you want the full chain you are accepting an external dependency by definition. Anyone who needs a fully offline pipeline should plan to use transcribe with a local engine and do the segmentation themselves.

How it compares to running Whisper and an SRT editor yourself

The realistic alternative is not another all-in-one tool. It is assembling the parts: run faster-whisper or whisper.cpp locally for word-level timestamps, then edit the resulting SRT by hand or with a script, then translate with a separate tool if needed, then mux with ffmpeg. The difference in approach is where the intelligence sits. In that assembly, sentence boundaries are yours to decide and the whole job can be done with no network access and no per-request cost, at the price of your own time on every video. VideoCaptioner moves the boundary decision to a language model and charges you in API calls and in the requirement that transcript text leaves your machine. There is a middle position the project itself supports: use videocaptioner transcribe with faster-whisper or whisper-cpp to stay local, then run videocaptioner subtitle with the bing translator, which the README lists as free. That gets you recognition and translation without an LLM key, but you give up the LLM segmentation that the project is built around. Which side of that line you sit on depends on whether your bottleneck is editing time or data residency.

Licence, dependencies and the cost of staying current

VideoCaptioner is GPL-3.0. That is a copyleft licence, and it governs the whole distributed program, not just the parts you modify. If you are wiring the CLI into an internal pipeline for your own use, the practical effect is limited. If you intend to ship a product that bundles or links this code, GPL-3.0 imposes source disclosure obligations on the combined work, and that is a decision for your legal team rather than something to infer from a README. Note also that the project distributes an FFmpeg binary as a separate release artefact, tagged ffmpeg-bin with a note not to delete it. That bundling is a reminder that the video muxing and burn-in steps depend on FFmpeg being present, so the effective dependency surface is larger than the pip package alone. On maintenance, the release history shows v1.4.2 in May 2026, v1.4.1 in March 2026 and a binary refresh in June 2026, with the repository last pushed in July 2026. The cadence is regular but the version numbers move slowly, which fits a tool whose core loop is stable. Because the LLM-facing parts depend on external endpoints and model names, the code that ages fastest is the configuration surface, not the pipeline. Development instructions use uv: uv sync, uv run videocaptioner, uv run pyright for type checking and uv run pytest tests/test_cli/ for the test suite, so a contributor can verify changes against the CLI tests specifically.

Editorial conclusion

Adopt VideoCaptioner if you want one Python entry point that goes from an audio file to a translated, optionally burned-in subtitle track, and you are comfortable sending subtitle text to an OpenAI-compatible endpoint. Do not adopt it if your content cannot leave your network, or if you need a permissive licence for a closed product. Verify first: run videocaptioner config show to confirm where your API key is being read from, and run videocaptioner transcribe on one short clip with --asr bijian to check the free path before you commit a long video to it.

Official sources

1. License: GPL-3.0
2. Project website
3. README
4. Releases
5. WEIFENG2333/VideoCaptioner on GitHub