story-to-handdrawn-video: turn Chinese story copy into a silent hand-drawn picture track
Agent skill: convert Chinese story copy or ordered images into a hand-drawn diary-comic animation (silent MP4 picture track).
At a glance
- What is it?
- A Remotion project plus a distributable Agent Skill that storyboards Chinese text or ordered images into 3:4 hand-drawn animation, with 20 selectable styles and a silent H.264 output meant for later voice-over.
- Who is it for?
- Adopt it if you already work in a Skill-capable agent runtime, have Node 20, Python 3.10 and FFmpeg on the machine, and want a silent 1080x1440 picture track you will dub yourself. Skip it if you need a finished video with audio, a hosted editor, or a photoreal sketch filter over live footage.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 42 days ago.
- What is it written in?
- Mainly JavaScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 18, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What story-to-handdrawn-video actually produces
The output is a picture track, not a finished video. The README describes the default render as H.264 with no voice-over and no music, at 1080x1440 in a 3:4 vertical frame, so the intended workflow is to dub and score the MP4 afterwards. That single decision shapes everything else: there is no audio mixing, no subtitle burn-in beyond the hand-written caption layer, and no delivery format for horizontal screens.
The audience is narrow and specific. You need Chinese source copy, because the splitting and storyboard logic works on Chinese sentences, and you need to be operating inside an agent that can load a Skill. The repository ships two halves: a Remotion renderer at the root, and skill-package/, a distributable Skill that lets Codex, Claude Code or Kimi Code drive that renderer from natural language. If you are comfortable running npm scripts by hand, the renderer half is usable on its own.
How the storyboard pipeline is wired
The project keeps a JSON storyboard as the contract between text and pixels. Two files sit at the root, storyboard.json and storyboard.uploaded.json, and the check:storyboard script validates both with node scripts/validate-storyboard.mjs. That means the generated plan is inspectable before a single frame is rendered, which is the most useful property here: a bad split shows up as bad JSON, not as a wasted render.
On the text path, a story is split into sentences, one beat per complete sentence by default, and each beat becomes a scene. For uploaded images, import-uploaded-pages.mjs keeps the supplied order and composition. The visual reveal runs left to right through three states: text, then a black-and-white line layer generated locally and aligned to the color illustration, then the color illustration. An optional page-curl transition is available for the bottom-right corner, and the README notes the paper back keeps a faded texture of the original page.
Style selection is explicit rather than automatic. Twenty styles ship with fixed example images generated from the same characters, actions and composition, so the contact sheet is a comparison of materials and line quality, not of subject matter. The README is careful about this: the examples are style evidence only, and characters and scenes are still controlled by the source text and character locking during generation.
Installing it and rendering a first preview
The README lists Node.js 20 or higher, Python 3.10 or higher, FFmpeg with ffmpeg and ffprobe callable from the terminal, npm, Google Chrome or a Remotion-managed compatible browser, and an agent runtime that supports Skills. Clone the renderer and run the offline check first; it does a TypeScript pass and a storyboard structure validation without touching the network.
git clone https://github.com/gnipbao/story-to-handdrawn-video.git
cd story-to-handdrawn-video
npm ci
npm run checkIf that passes, install the Skill into your agent's skills directory. The README gives three targets, and the path differs per runtime, so copy the one that matches yours.
cp -R skill-package/story-to-handdrawn-video ~/.codex/skills/
cp -R skill-package/story-to-handdrawn-video ~/.claude/skills/
cp -R skill-package/story-to-handdrawn-video ~/.agents/skills/Point the Skill at the renderer if you are not running the agent from inside the project directory.
export STORY_VIDEO_PROJECT=/absolute/path/to/story-to-handdrawn-videoThen drive it in natural language. The README's default prompt asks for a preview first, which renders at 720x960 so you can judge pacing before paying for the full 1080x1440 pass.
使用 $story-to-handdrawn-video 先给这个故事生成一个预览版。You can also skip the agent and render directly. The package scripts expose the same two modes, with the preview variant using --scale=0.6666666666666666 and a higher CRF.
npm run render:preview
npm run renderExpect out/picture_silent-preview.mp4 from the first and out/picture_silent.mp4 from the second. Both are muted by design.
Where the pipeline pushes back
The silence is the first constraint, and it is not a setting you can flip. There is no audio track to enable, so any project that needs a finished deliverable with narration has to add a second tool. The 3:4 frame is the second: 1080x1440 is a vertical format, and nothing in the README suggests a landscape or square variant.
The generation backend is the sharper edge. Images default to the Codex Image2 workflow, and the OpenAI API path is described as explicitly selected and requiring OPENAI_API_KEY. Those are different routes with different availability, so a team standardized on one provider may find the default path unusable and the fallback gated behind a key and an explicit request. The README also flags its own weak spots: time jumps, unclear referents, medical scenes and age-sensitive characters are cases where it recommends asking the Agent for a visual plan first, a JSON keyed by scene number, and confirming before generating. That is an admission that the automatic split is not reliable on ambiguous prose.
Finally, the repository is private in package.json while the licence is MIT, and there are no releases. Installation is from the default branch, so the version you get is whatever main holds at clone time.
How it differs from sketch filters and manual animation
The obvious comparison is a sketch or cartoon filter applied to existing footage, the kind of tool people search for when they want to convert a video to sketch style. That approach preserves real motion and timing and needs no storyboard; this project does the opposite. It has no footage input at all. It builds scenes from sentences or from still images, generates the illustrations, and animates the reveal between layers. If your source is a video, this is the wrong tool.
The other comparison is hand animation in a timeline editor. There you control every keyframe and pay for it in hours per minute. Here the unit of control is the storyboard JSON and the style id, and the render is a batch command. You trade frame-level authorship for a repeatable pipeline. The page-curl transition and the three-state reveal are the two places where the project has a distinct visual idea rather than a generic template, and both are optional.
Against a plain Remotion setup, the difference is the Skill wrapper and the twenty-style library with fixed comparison images. Remotion gives you the rendering primitive; this repository gives you the storyboard schema, the validation script and the style vocabulary on top of it.
Maintenance, licence and upgrade cost
The last push to the default branch was on 2026-08-08, about six weeks before this writing, and the repository is not archived. package.json declares version 1.1.0 and pins @remotion/cli at 4.0.487 with React 19.2.7. There are no tagged releases, so upgrade cost is the cost of tracking main: you re-run npm ci, then npm run check, and you re-read the storyboard validator's expectations if the schema moved.
The licence is MIT, which permits commercial use and modification, and the LICENSE file is at the root. Two things to note without giving legal advice: the repository is marked private in package.json, which affects publishing rather than your rights under MIT, and the example images under references/style-examples are part of the repository and carry whatever the MIT grant covers. If you redistribute the style library, read the LICENSE text yourself rather than relying on the badge.
The real recurring cost is not the code. It is the image generation step, which runs per scene, and the manual review loop the README recommends for ambiguous stories. A twenty-scene story means twenty generations plus a storyboard pass before you have a preview.
Editorial conclusion
Adopt it if you already work in a Skill-capable agent runtime, have Node 20, Python 3.10 and FFmpeg on the machine, and want a silent 1080x1440 picture track you will dub yourself. Skip it if you need a finished video with audio, a hosted editor, or a photoreal sketch filter over live footage. Before committing, run npm run check on a clean clone and confirm which image backend you are allowed to use, because the default Codex Image2 path and the explicit OpenAI API path are not interchangeable.
Frequently asked questions
What is a hand-drawn animation called?
The README does not define a single industry term for the category. It describes its own output as a hand-drawn story animation and names the default style as a colored-pencil diary comic, which is the vocabulary the project uses.
What is it called when you draw over a video?
story-to-handdrawn-video does not cover drawing over existing video. It takes Chinese story text or an ordered set of still images as input; no video input path is documented.
How can I create hand-drawn animation with story-to-handdrawn-video?
Install the Skill into your agent's skills directory, optionally set STORY_VIDEO_PROJECT to the renderer path, then prompt the agent in natural language with the story text or a list of image paths. The README recommends generating a preview first, and the renderer can also be run directly through npm run render:preview and npm run render.
Is there a free way to convert videos to sketch style online?
That is not what story-to-handdrawn-video does. It has no online service and no video input; it renders locally through Remotion from story text or ordered images, and the output is a silent H.264 picture track.
Community notes