Video-subtitle-extractor: local hardsub OCR without a cloud API
视频硬字幕提取，生成srt文件。无需申请第三方API，本地实现文本识别。基于深度学习的视频字幕提取框架，包含字幕区域检测、字幕内容提取。A GUI tool for extracting hard-coded subtitle (hardsub) from videos and generating srt files.
At a glance
- What is it?
- VSE is a Python GUI and CLI that detects burned-in subtitles in a video and writes an srt file using local OCR models. The trade-off is a heavy PaddlePaddle and CUDA install, and a hard rule that your file paths contain no Chinese characters or spaces.
- Who is it for?
- Adopt VSE if you have a fixed-resolution batch of videos with burned-in subtitles and you can give it a GPU, a clean ASCII path and a PaddlePaddle environment. Do not adopt it if your subtitles move between frames, if your files sit under a path with Chinese characters or spaces, or if you need the newest NVIDIA 50-series cards on CUDA, since the README states Paddle 3.3.1 does not yet support CUDA 12.8 and points those users at the DirectML build.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 160 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem VSE targets: subtitles baked into the pixels
Hard-coded subtitles are part of the video frame. There is no text stream to demux, no track to select, nothing a player can toggle. If you want a translatable or searchable subtitle file, someone has to look at the picture and type what it says. VSE automates that pass: it samples frames, finds where text sits in the frame, reads the characters with a local OCR model, and emits an srt file. The README describes the pipeline in four steps: extract key frames, detect text positions, recognise text content, and filter out text that is not a subtitle.
The intended user is someone with a local video collection and no interest in wiring up Baidu or Alibaba OCR. The project's stated feature is that recognition runs locally, so no third-party API key is required. That matters for volume work: a two-hour film is a lot of frames, and per-call cloud OCR pricing turns that into a budget question. It also matters when the material should not leave the machine. VSE also lists batch extraction, which the README says requires every video in the batch to share the same resolution and subtitle region. That constraint is the honest one: the region you draw in the GUI is reused across the batch, so mixed-resolution folders will not work as a single job.
Frame sampling, region detection and the three recognition modes
The README names VideoSubFinder as the subtitle detection engine on Windows, Linux and macOS, and lists a VSE engine used only in precise mode. The mode table is the most useful part of the documentation because it tells you what you are trading. Fast mode uses a mini OCR model with VideoSubFinder detection. Auto mode picks the model for you: mini on CPU, large on GPU. Precise mode uses the large model with the VSE detection engine and, per the README, detects frame by frame on GPU.
The README is explicit about the cost. Fast mode may drop a small number of subtitles and produce a small number of wrong characters. Auto mode on GPU is slower and may still drop a few lines but almost never misreads characters. Precise mode does not drop lines and almost never misreads, but is described as very slow. The README's own guidance is to start with fast or auto and only move to precise if the first two drop too many lines. That is a sensible default and it is stated plainly rather than buried.
Two post-processing steps sit after recognition. Duplicate subtitle lines are removed before the srt is written. Separately, a typo map at backend/configs/typoMap.json handles two jobs at once: fixing recurring OCR mistakes and deleting text you never want in the output. The README's example maps "l'm" to "I'm" and "威筋" to "威胁", and maps a watermark string to an empty value so it is removed entirely. The same mechanism is what the README points to for stripping watermarks, station logos and existing hard subtitles, with a companion project, video-subtitle-remover, suggested for the removal side.
Install paths: CUDA, DirectML, ONNX and plain CPU
VSE requires Python 3.12 or newer. The README's source install starts with a virtual environment:
python -m venv videoEnv
On Windows you activate it with videoEnv\Scripts\activate; on macOS or Linux, source videoEnv/bin/activate. Then you cd into the source directory and pick one of four runtimes.
For NVIDIA cards, the recommended combination is CUDA 11.8 with cuDNN 8.6.0, followed by:
pip install paddlepaddle-gpu==3.3.1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/ pip install -r requirements.txt
For AMD, Intel or APU acceleration on Windows, you install the CPU build of PaddlePaddle and then the DirectML requirements on top:
pip install paddlepaddle==3.3.1 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/ pip install -r requirements.txt pip install -r requirements_directml.txt
The README notes the DirectML route is also the suggested fallback for NVIDIA 50-series cards, because CUDA 12.8 or newer is required by those GPUs and Paddle 3.3.1 does not yet support it. The ONNX route targets macOS, AMD ROCm and Apple Silicon. The README warns that this path is untested and asks that issues not be filed against it, and it tells you to adapt requirements_directml.txt to your own execution provider, giving requirements_coreml.txt with paddle2onnx and onnxruntime-coreml as an example. CPU-only installs just use the CPU PaddlePaddle wheel plus requirements.txt.
Once the environment is in place, python gui.py starts the graphical version and python ./backend/main.py starts the command line version. The GUI workflow is open a file, adjust the subtitle region, press run; selecting multiple files at open time switches to batch extraction.
Path restrictions, GPU support gaps and the precise mode trap
The README puts a hard constraint in bold: video and program paths must not contain Chinese characters or spaces, and it gives two failing examples, a path under a Chinese-named folder and a video filename with a space in it. This is not a stylistic preference. It is the kind of constraint that surfaces as an unexplained failure, and it will bite anyone whose media library lives under a localised folder name. Plan the working directory before you start.
The second limitation is hardware coverage. CUDA support in the shipped Paddle version stops short of the newest NVIDIA generation, and the README's answer is to use DirectML instead. That is a real functional difference, not a footnote: DirectML is the Windows-oriented path, so a Linux user on a 50-series card has no documented route in the README. The ONNX path exists but is explicitly untested and unsupported for issue reports.
The third is precise mode. It is the only mode that promises no dropped lines, and the README calls it very slow and does not recommend it. So the mode that solves the accuracy problem is the one you are told not to use by default. If your source has fast-cut subtitles or subtitles that appear and disappear between the sampled frames, the fast and auto modes may miss lines, and the only documented remedy is the slow path. There is no documented setting for adjusting the sampling interval in the supplied material.
A fourth point the README does not resolve: the detection engine is VideoSubFinder on all three operating systems, which means the project depends on that component's behaviour for where text is found. The README does not describe how the region you draw in the GUI interacts with that detector, or what happens when a subtitle moves vertically mid-video. Treat that as unverified.
How VSE differs from a subtitle editor with OCR, and from cloud OCR
The obvious alternative is a subtitle editor with a built-in OCR import, such as Subtitle Edit. The difference is where the work happens. An editor's OCR import typically asks you to define the text region, then runs recognition and hands you a list of characters to correct by hand, with the matching done interactively. VSE instead treats the whole thing as a batch pipeline: detection, recognition, deduplication and srt writing run as one job, and corrections are expressed as rules in typoMap.json rather than as manual edits. That is better for a folder of episodes and worse for a one-off clip where you would rather see and fix each line.
The other alternative is a cloud OCR service. VSE's stated advantage is that nothing is sent anywhere and no API application is needed. The cost is that you own the runtime: CUDA, cuDNN, PaddlePaddle and a virtual environment, plus the path restrictions above. A cloud service moves that burden to the provider and adds per-call cost and a data-handling question. For a single video, the cloud route is less setup. For a large local archive, the local route is the one that scales without a bill.
Within the project's own orbit there is video-subtitle-remover, which the README suggests pairing with VSE. The division is clean: VSE reads the burned-in text and writes an srt, VSR removes the burned-in text from the picture. They are separate installs and separate models, so pairing them means maintaining two environments.
Upgrade cost, licensing and what the release history suggests
The project is Apache-2.0 licensed. That is a permissive licence, and it is worth noting that it covers the code in this repository, not the model weights or the VideoSubFinder component, whose terms the README does not discuss. If you plan to redistribute a build, check those separately rather than assuming the repository licence settles it. Nothing here is legal advice.
On maintenance, the release history shows 2.0.0 in October 2023, 2.0.3 in April 2025 and 2.2.0 in April 2026, with the last push to main in April 2026. The gap between 2.0.0 and 2.0.3 is roughly eighteen months, so this is not a project that ships constantly. The practical upgrade cost is the pinned PaddlePaddle version. The README pins paddlepaddle 3.3.1 for every runtime, and the CUDA 11.8 plus cuDNN 8.6.0 combination is the documented target. Moving to a newer CUDA means waiting for Paddle support, which the README already flags as the blocker for 50-series cards. Upgrading the tool therefore means re-testing the whole OCR stack, not just pulling a new srt writer.
The README also points users to a set of QQ groups for support and directs improvement suggestions to Issues and Discussions. There is no homepage listed for the project, so the repository and its release page are the documentation. For a tool whose failure modes include silent line drops, that thinness is the main risk: you will be measuring output quality yourself rather than reading it off a spec sheet.
Editorial conclusion
Adopt VSE if you have a fixed-resolution batch of videos with burned-in subtitles and you can give it a GPU, a clean ASCII path and a PaddlePaddle environment. Do not adopt it if your subtitles move between frames, if your files sit under a path with Chinese characters or spaces, or if you need the newest NVIDIA 50-series cards on CUDA, since the README states Paddle 3.3.1 does not yet support CUDA 12.8 and points those users at the DirectML build. Verify three things before committing: that the release archive runs on your machine, that your chosen mode does not drop subtitle lines on a sample clip, and that the typoMap.json replacement list covers the watermarks in your source.
Community notes