video-use: Letting Claude Code Cut Video Without Ever Watching It

What video-use actually does

video-use is a skill you install into a coding agent like Claude Code or Codex. It gives that agent a set of commands for editing video: cutting out filler words, arranging takes, adding captions, applying color grades, and rendering a final file. The README describes a workflow where you drop raw footage into a folder, tell the agent something like "edit these into a launch video," and it returns edit/final.mp4. The intended user is someone who already lives in a terminal and prefers chatting with an agent over opening a traditional video editor. The project is 100% open source under the MIT license, and it borrows the browser-use idea of giving an LLM a structured representation instead of raw pixels.

The two-layer reading mechanism

The central design choice is that the LLM never watches the video. Instead, video-use builds two representations. Layer one is an audio transcript produced by ElevenLabs Scribe, with word-level timestamps, speaker diarization, and audio events like laughter or applause. All takes are packed into a single takes_packed.md file, roughly 12KB, which becomes the agent's primary reading view. Layer two is a visual composite called timeline_view, generated on demand. It produces a filmstrip, a waveform, and word labels as a PNG for any time range. The agent calls this only at decision points, such as ambiguous pauses or retake comparisons. The README contrasts this with a naive approach of dumping 30,000 frames as tokens, which would cost 45 million tokens of noise. video-use reduces that to 12KB of text plus a handful of PNGs. That is the same philosophy as browser-use giving an LLM a structured DOM instead of a screenshot, applied to video.

The pipeline: transcribe, pack, reason, render, self-evaluate

The pipeline is explicit in the README: Transcribe, Pack, LLM Reasons, EDL, Render, Self-Eval. After transcription, the agent reasons over the packed transcript and proposes an editing strategy. It waits for your approval before touching anything. Then it produces an EDL, which is an edit decision list, and renders the output. The self-eval step runs timeline_view on the rendered output at every cut boundary. It checks for visual jumps, audio pops, and hidden subtitles. If it finds an issue, it fixes and re-renders, up to a maximum of three attempts. Only after passing self-evaluation do you see a preview. This loop is a concrete mechanism, not a vague promise. The README also states that all outputs live in a <videos_dir>/edit/ folder, so the skill directory stays clean.

Getting it running: commands and config

The manual install path is straightforward. You clone the repository into ~/Developer/video-use and symlink it into your agent's skills directory. For Claude Code that is ~/.claude/skills/video-use, for Codex it is ~/.codex/skills/video-use. Then you install dependencies with uv sync or pip install -e ., and you need ffmpeg installed via Homebrew. yt-dlp is optional for downloading online sources. You must add an ElevenLabs API key by copying .env.example to .env and editing it. The README suggests pasting a setup prompt into Claude Code, Codex, Hermes, Openclaw, or any agent with shell access. That prompt tells the agent to read install.md, wire up ffmpeg, register the skill, and ask for the API key. After installation, the agent should not transcribe anything on its own; it waits for you to drop footage into a folder. This is a skill, not a standalone application, so the runtime is whatever agent you already use.

Limitations and wrong-tool cases

The most obvious limitation is the hard dependency on ElevenLabs for transcription. The README says one ElevenLabs Scribe call per source, and the setup prompt asks for an API key. If you cannot use that service, or you do not want to send your footage to a third party, video-use has no fallback path described. Another limitation is that the LLM never watches the video, so it cannot see visual content unless it explicitly calls timeline_view. That means edits that depend on visual continuity, like matching action between two shots, may be missed if the agent does not request a visual check. The README's own design principle says audio is primary and visuals follow. For content where the visual track carries the meaning, like a silent montage or a product demo with no narration, this approach could be the wrong tool. Also, the self-eval loop is capped at three re-renders. If a cut problem persists beyond that, the README does not say what happens; presumably you see the flawed output or the agent gives up. That is a real boundary.

Alternatives and the difference in approach

The obvious alternative is a traditional non-linear editor like DaVinci Resolve or Premiere Pro, where you see frames and waveforms directly and cut with your own eyes. The difference is fundamental: those tools give you full visual control but no LLM reasoning over the transcript. Another alternative is an AI video editor like Descript, which also transcribes audio and lets you edit by editing text. Descript's approach is similar in that it uses the transcript as the primary editing surface, but it is a closed product, not a skill for an agent, and it does not hand the edit decision to a coding agent. video-use is distinct because it makes the agent the editor, and it keeps the visual layer as an on-demand aid rather than the main interface. If you want a human-in-the-loop editor that understands speech, Descript is more mature. If you want an agent that can run unattended in a terminal, video-use is the one that fits.

Maintenance, upgrade cost, and license

The README does not mention version numbers or a release cadence, and the repository metadata shows no recent releases. That means the upgrade path is unclear; you are likely cloning the main branch and following the install.md instructions each time. The project is not archived, but last push is unknown, so you should check the repository for activity before relying on it. The license is MIT, which allows commercial use and modification with attribution, but that is not legal advice. The dependency on ElevenLabs has its own pricing and terms, which you must accept separately. The skill persists session memory in project.md, which is a maintenance feature: next week's session picks up where you left off, but it also means you have a state file that can grow stale or conflict across projects. There is no mention of automated tests or CI in the README, so reliability is unverified.

Where the project needs more evidence

The README makes strong claims: 30ms audio fades, word-boundary precision, self-evaluation that catches visual jumps and audio pops. None of these are backed by benchmark numbers or user reports in the material I have. The self-eval loop is described as running timeline_view on the rendered output, but there is no detail on how it detects an audio pop or a hidden subtitle programmatically. It might be that the agent visually inspects the PNG, which would be slow and unreliable, or it might use audio analysis, which is not specified. This is a gap. The README also says the LLM proposes a strategy and waits for your OK, but it does not describe what happens if you reject the strategy or want to change it mid-edit. The project is clearly at an early stage, and the marketing tone suggests a demo rather than a hardened tool. Before adopting it for professional work, you should read SKILL.md and helpers/ to see the actual rules and scripts, because the README is light on implementation details.

Editorial conclusion

Adopt video-use if you already work with Claude Code, Codex, or a similar agent and want to turn raw footage into a finished edit by typing a short instruction. It suits talking heads, tutorials, and interviews where the transcript carries the meaning. Do not adopt it if you need frame-accurate visual editing without an LLM in the loop, or if you cannot accept the ElevenLabs dependency for transcription. Before trusting it on a paid project, verify that the 12 hard rules in SKILL.md match your editorial standards and test the self-eval loop on a short clip to see whether it catches the cut errors you care about. The project is young, so check the repository for recent commits and open issues before building your workflow around it.

Official sources

1. Official README
2. Project repository