claude-real-video: scene-change keyframes and a transcript in one folder for your LLM
Let Claude (or any LLM) actually watch a video — scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.
At a glance
- What is it?
- crv is a local Python CLI that turns a video URL or file into deduplicated scene-change frames, a transcript, and a manifest an LLM can read. It solves a real sampling problem, but it is a preprocessing step, not a video understanding model.
- Who is it for?
- Adopt crv if you already run ffmpeg and want a local, inspectable folder of frames and transcript to paste into an LLM, especially for screen recordings, lectures, and long calls where you can name a time window. Do not adopt it if you expect the tool itself to interpret the video, if your source has no captions and you refuse to install the whisper extra, or if you need a hosted pipeline that runs without ffmpeg on the machine.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 5 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap crv targets: fixed-interval sampling and transcript-only tools
The README frames the problem in terms of what existing tools do with video. Pasting a YouTube link into ChatGPT, it says, means the model reads the transcript rather than the picture. Claude, per the README, will not accept a video file at all. Gemini can read video natively, but the README states it samples frames at a fixed interval, 1 fps by default, so fast cuts slip past. That last point is the technical core of the pitch. A fixed rate spends its budget uniformly across time, which is wrong for any clip whose information density varies. A 90-minute call with one 10-minute screen share is the example the README uses. At a fixed rate you pay for the other 80 minutes and may still miss the moment the shared screen changed. crv instead detects scene changes and drops near-duplicate frames, so the frame count tracks how much the picture actually changed. The README gives one concrete comparison: a 58-second clip sampled at 1 fps yields 58 frames, while crv keeps 26 that differ, packed into 3 contact sheets with --grid. Treat that as an illustration of the selection rule, not a benchmark. The audience is anyone who wants an LLM to look at a video without uploading the video: developers using Claude Code or another coding agent, and people doing screen recordings, lectures, or interviews who are willing to run a command locally.
What the pipeline produces and where the processing happens
The output is a folder, not an answer. Running the CLI on a URL produces crv-out/frames/*.jpg, a frames.json holding per-frame timestamps, a transcript.txt and transcript.json, and a MANIFEST.txt. The README's instruction is to drop the frames plus MANIFEST.txt into Claude, ChatGPT, or Gemini and ask your question there. That division of labour matters when you evaluate the tool: crv does selection and transcription, and the model you paste into does interpretation. The README also states that processing runs locally and that what leaves the machine is only the frames or text you choose to paste afterwards. Two details in the recent release notes show the URL path is still moving. v0.10.2 is titled "URL runs use the source's own captions", which suggests that when a source ships captions, crv prefers them over generating its own transcript. v0.10.3 is a caption header fix, small but a reminder that the caption path is recent. The transcription side depends on Whisper, installed through the whisper extra. The README also notes a non-LLM use: as a general-purpose keyframe extractor, scene detection plus dedup with no ML models to download.
Installing crv and the flags that change what gets sampled
The README gives the install as `pip install "claude-real-video[whisper]"`, with Python 3.10+ per the badge. For agent hosts there is `npx skills add HUANGCHIHHUNGLeo/claude-real-video`, described as installing the skill into Claude Code, Cursor, Codex, Copilot, Gemini CLI and other agentskills.io-compatible hosts. Claude Code users can instead add the plugin marketplace and install claude-real-video@claude-real-video. A manual path exists: clone the repository and copy skills/claude-real-video into ~/.claude/skills/. CLI-only use is just `crv "<url>"`. The flags worth knowing are the ones that change the sampling decision rather than the output format. `--from 28:00 --to 43:00` restricts the run to a window; the README states ffmpeg seeks rather than decoding the whole file, Whisper only hears the window, and the frame budget is spent inside it, while reported timestamps remain source timecodes. `--adaptive` picks frames against a rolling neighbourhood instead of a fixed threshold, aimed at slow pans and gradual morphs. `--text-anchors` forces extra frames at subtitle-cue timestamps, needs a sidecar .srt/.vtt or an embedded subtitle track, allows at most one forced frame per second, and leaves scene detection untouched. `--speakers` adds speaker labels to each transcript line and needs the speakers extra plus a one-time 45 MB diarization model download. `--frame-width 1600` raises resolution for text-heavy footage such as a terminal or spreadsheet. `--viewer` writes a local viewer.html with video, keyframe grid, and transcript. `crv-web` opens a local page with Traditional Chinese, Simplified Chinese, and English UI.
Where crv breaks down or is the wrong tool
The most concrete limitation is stated in the README itself: captions burned into the pixels cannot be detected by --text-anchors, which needs a sidecar or embedded subtitle track. If your lecture recordings have hardcoded subtitles and no separate track, that flag does nothing for you. The same flag caps forced frames at one per second, so a fast-talking segment with several visual changes per second gets one anchor, not several. The deduplication rule is also a selection rule, and selection can be wrong. A frame that differs from its neighbours by a small margin may be dropped even when the small difference is the point, which is why --adaptive exists as a separate flag rather than being the default. Resolution is the other failure mode the README calls out directly: at 640px on a 1920-wide screen recording, the right moment is found and then the detail that made it worth finding is thrown away. That is a real cost, since raising --frame-width raises the token cost of every frame you paste. Diarization is an optional extra with its own model download, so --speakers is unavailable on a bare install. And crv is a preprocessing tool: it does not answer questions about the video. The README is explicit that the paid crv Pro add-on is what adds shooting style and a timestamped timeline of gestures, expressions, and sound events. If your question depends on tone or delivery rather than visible content, the free tool's frames and transcript are the wrong input.
How this differs from native video input and from ffmpeg one-liners
The nearest comparison is Gemini's native video handling, which the README describes as uploading the video to Google and sampling at a fixed interval. The difference is twofold. First, the sampling rule: fixed interval versus scene-change detection with deduplication, which changes how the frame budget is spent on a clip with uneven visual activity. Second, where the bytes go: crv processes locally and sends nothing until you paste the output, whereas native video input sends the video to the provider. The second comparison is the ffmpeg scene-detection one-liner, which is the honest alternative for anyone who just wants frames. A plain ffmpeg scene filter can emit frames at scene changes, and it needs no Python package. What it does not give you is the rest of the bundle: deduplication, frames.json with per-frame timestamps, a transcript, a MANIFEST.txt written for a model to read, contact sheets via --grid, the local viewer, and the window and adaptive selection logic. If you only need JPEGs, ffmpeg is fewer moving parts. If you need a folder that a model can consume without you writing a script around it, that assembly is what crv is selling. A third option is a hosted video-understanding API, which removes the ffmpeg dependency but moves the video off your machine, the opposite of what crv is built for.
Licence, extras, and what maintenance you are taking on
The package is MIT-licensed, which permits commercial use and modification; the repository ships a LICENSE file, and the README links it. I am not giving legal advice, and the MIT grant covers the code, not any model weights you download separately. Those downloads are worth separating out when you plan a deployment. Whisper arrives through the whisper extra, and diarization through the speakers extra with a 45 MB model fetched once. The README states the diarization model needs no account or token, but it is still a network fetch on first use, so an air-gapped machine needs that handled in advance. Upgrade cost is the more interesting number. The release cadence visible in the material is tight: v0.10.1, v0.10.2, and v0.10.3 all landed within about four days in late August 2026, with v0.10.2 changing how URL runs obtain captions. Behaviour on the URL path is therefore still being adjusted, and anyone pinning a version for reproducibility should expect to re-verify output after a jump. There is also a commercial split to keep straight: the free package is claude-real-video on PyPI, and the paid add-on is crv Pro, sold under the listing name llm-real-video Pro. The README does not describe how the paid add-on installs or what it does to the free package's output, so I cannot say whether it is a separate binary, a plugin, or a hosted service.
Who should install it and what to check before trusting the output
The fit is narrow and clear. You have ffmpeg on the machine, you are comfortable with a CLI or with an agent host, and you want a local folder of frames plus a transcript that you will paste into a model yourself. Screen recordings, lecture slides, interviews, and long calls with a known window are the cases the README argues for, and the flags map onto them: --from/--to for the window, --frame-width for small text, --text-anchors when a subtitle track exists, --adaptive for slow changes, --speakers for multi-speaker audio. The tool is a poor fit if you want an answer rather than an input, if your footage has burned-in captions and you were counting on --text-anchors, or if you need a pipeline that runs without ffmpeg on the host. Before trusting a run, check the manifest. Open crv-out/MANIFEST.txt and frames.json and ask whether that is what you would have chosen to show a model. Then check the timestamps against the source, since the README promises source timecodes even for windowed runs, and a wrong offset makes every quoted moment useless. Then run the same clip twice, once default and once with --adaptive, and compare the frame counts and the moments kept. That comparison costs one command and tells you more about whether the selection rule matches your footage than any description of the algorithm will.
Editorial conclusion
Adopt crv if you already run ffmpeg and want a local, inspectable folder of frames and transcript to paste into an LLM, especially for screen recordings, lectures, and long calls where you can name a time window. Do not adopt it if you expect the tool itself to interpret the video, if your source has no captions and you refuse to install the whisper extra, or if you need a hosted pipeline that runs without ffmpeg on the machine. Verify three things first: that ffmpeg is on PATH, that `pip install "claude-real-video[whisper]"` resolves on your Python version (3.10+), and that a single `crv "<url>"` run on one representative clip produces the frames.json timestamps and MANIFEST.txt you would actually paste. If the manifest is not what you would hand to a model, the tool has not solved your problem, and no flag will fix that.
Community notes