claude-motion gives a coding agent motion rules, sound and a frame review loop

Skills teach an agent the API, this teaches it taste

The framing in the README is a comparison. Remotion skills teach an agent the API, and claude-motion teaches it taste, makes it check its own frames, and gives it a sound engine. The intended user is therefore not a video editor but a developer who works with coding agents, because the repository ships agent instructions rather than a user interface. The skills live in .claude/skills/, so the toolkit works with Claude Code and Cursor out of the box, and with Codex, OpenCode or anything else that reads AGENTS.md. The motion rules are numeric rather than vague. Instead of asking for something smooth, the skill specifies a bezier curve for entrances, exits at half the duration of the entrance, and a fixed gap between words, alongside spring presets, stagger, reading holds, a type scale, a single accent colour, texture and motivated transitions. There is also a reference piece in the repository with reusable components such as masked text reveals, rolling numbers, film grain, a hand-drawn line boil and a background with a drifting grid and glow, which gives the agent something concrete to take apart.

One JSON timeline drives picture, sound and review

The architecture is a pipeline, and the key decision is that every beat is a named timestamp in a JSON file. Scenes, sound and the review step all read that same timeline, so retiming the video retimes the sound and the checks with it. The flow runs from an idea to a storyboard, then to timeline.json, and from there into two branches. The picture branch goes to scenes, to a draft render, to a contact sheet, and back to the agent for another pass. The sound branch goes to a cue sheet, then to the Python sound module, which produces a waveform and a loudness reading, and finally to a render and mux that produces the finished MP4. The npm scripts in package.json map directly onto those stages, with studio for a live preview of every composition, draft for a half-scale quick render at a higher compression setting, render for the silent picture, sfx for the cue sheet, mux for combining picture and audio, build for the whole sequence, sheet and wave for the diagnostics, and cover and gif for stills and a preview loop. The three scripts that matter most for an agent are build, sheet and wave, since they are the render, the check and the measurement.

Contact sheets are how an agent looks at its own work

The review loop is the most interesting idea in the project, and the reason for it is stated plainly: agents cannot watch video, but they can look at frames. The agent renders a draft, receives a contact sheet with one timestamped frame per beat, checks it against a list that includes collisions, clipping, holds, safe area and phone readability, and fixes what it finds before saying it is done. The tool that produces that sheet is npm run sheet with a video path, and it accepts a from, to and every interval so a transition can be scrubbed frame by frame. The output is a grid of tiles, one per beat, each labelled with its timestamp, which is exactly the kind of image an agent can read. The audio side has the same treatment with npm run wave, which takes a WAV or an MP4 and renders a waveform with beat markers plus LUFS and true peak readings, so the agent can check the mix numerically rather than guessing. Together they replace the part of the workflow that requires a human to watch the output, and they do not replace the part that requires taste.

A sound engine in pure Python with no samples

The sound engine is the piece most likely to surprise you, because it produces audio without any sample library. It is pure Python standard library, with no samples and no licences to clear. It ships generators for typing, clicks, pops, whooshes, risers, chimes, impacts, textures and pads, a mixer that places sounds on the same timeline as the picture, and a mastering stage that reaches a target loudness of minus 14 LUFS on its own. The unit of work is a cue sheet, which is one Python function, and the agent writes one cue sheet per video and adds generators when it needs a new sound. A cue sheet looks like this:

from sfx.synth import pop, whoosh

def score(mix, tl):
    s1 = tl["s1"]
    mix.put(s1["title"], whoosh(0.4, 500, 2400, peak=0.45), 0.12)
    for i, t in enumerate(s1["items"]):
        mix.put(t, pop(900 + i * 150, 400), 0.3, pan=-0.4 + i * 0.3)

The timeline is passed in, so a cue is placed at a named timestamp rather than at a hard-coded time, and the pan and peak arguments are plain numbers the agent can vary. The sound module is invoked as python3 -m sfx, with a --list flag to see which generators exist and what each is for, and the build script runs the cue sheet for the example video to produce a WAV that the mux step then maps onto the silent render.

Web sims captured frame by frame into 60 fps video

The second pipeline exists because some pieces are simulations rather than motion graphics, and those are easier to write as a web page than as React components. Each piece is one HTML page with three.js and fonts inlined, which works offline, and opening it plays it live. The design constraint that makes capture possible is that render(t) is a pure function of time, with agents and physics simulated once at load, so any frame can be drawn on demand. The capture step uses headless Chromium to step the page frame by frame into ffmpeg, producing 60 fps video at 1440 by 1800 or any size you ask for, no matter how slow an individual frame is. Cuts land on the beat because a helper script named beats.py finds the tempo, bars and drop of a track and hands them to the simulation. Three examples are included to take apart, crawlers with 256 spider agents crawling a wall of websites, a dossier that turns one username into a diagram of connected threads, and a rocket engine explorer with a fly-through and an exploded view whose sound is generated in Python. The capture step needs a headless browser, which is why the setup asks for Playwright and Chromium:

npm install && npx playwright install chromium

The capture command takes the piece name, a size and an output path, and the sims build step produces a standalone HTML file per piece.

Setting up with an agent, or by hand

The README offers two routes and the first one is the unusual one. You paste a prompt into Claude Code, Cursor, Codex or OpenCode asking it to clone the repository, check that Node 20, Python 3 and ffmpeg are installed and install whatever is missing, run the install and build steps, show the resulting example video, then read AGENTS.md and the skills in .claude/skills/ and ask you what to make next. For simulations and 3D the same prompt adds the Playwright install and points at the sims documentation. The manual route is three commands:

git clone https://github.com/whaleyxbt/claude-motion && cd claude-motion
npm install
npm run studio       # live preview

Once it is running, the request to the agent is a short one, something like asking for a 15-second 1080 by 1080 motion graphic and following AGENTS.md, storyboard first, then the timeline, review loop and sound. The dependencies explain the shape of the project. Remotion 4.0.529 and its CLI sit at the centre with React 19, so the composition is a React component, and the fonts are shipped as packages, Inter, Instrument Serif and JetBrains Mono. Development adds Playwright for the capture step, three.js for the simulations, esbuild and TypeScript. The repository also ships an .mcp.json.example, a timeline.json at the root, a sfx directory, a scripts directory and an examples directory, and the package itself is marked private with the name effort-video, which is worth knowing if you expected a published library.

What this will not do for you

The limits are concrete rather than philosophical. The whole pipeline assumes Node 20 or newer, Python 3 and ffmpeg on the machine, plus a headless Chromium for the review and capture steps, so an environment without those is a non-starter, and a locked-down CI image is the likely failure. The agent instructions live in .claude/skills/, so an agent that only reads AGENTS.md is working from the summary rather than the full skill set, which is a real difference in behaviour between a Claude Code session and, say, a Codex session. There is also a question of taste that the tooling can only partly answer. The numeric rules, the review checklist and the loudness target make the output consistent, but the choice of what to animate and why still comes from the prompt, and the review loop checks collisions, clipping, holds, safe area and phone readability rather than whether the idea is worth watching. Finally, the packaging is informal. The repository has no GitHub releases, the last push was on 2026-10-09, and the package is private, so there is no version to pin, no changelog to read and no upgrade path beyond tracking the main branch. The licence is MIT, so forking and freezing a copy is straightforward if you need reproducibility.

Editorial conclusion

claude-motion fits developers who want an agent to produce a finished motion piece with sound rather than a rough mockup, and who are willing to review the contact sheet the agent produces. It does not fit a team without Node 20, Python 3, ffmpeg and a headless Chromium in the loop, since the review and capture steps depend on all of them. Before your first render, run the reference example end to end with npm run build to confirm the toolchain works on your machine, and note the last push was on 2026-10-09 with no tagged releases, so you are tracking the main branch.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. whaleyxbt/claude-motion on GitHub