handdraw-story-video: Turn Hand-Drawn Illustrations into Line-Reveal Vertical Videos

The Two-Phase Animation Design

The project solves a specific format constraint: short vertical story videos (35 to 45 seconds, 720x960) where hand-drawn illustrations animate in a consistent style across every frame. The animation plays in two phases. First, the black-and-white line art draws itself from left to right. Second, colour fills in along the same left-to-right direction. The README describes this as the defining characteristic of the format: line and colour move together as a unit, not as separate layers.

The input is 7 to 9 colour source illustrations (referred to as mother images). The pipeline extracts the line art locally from each colour illustration, so both the colour and the line version are aligned pixel-for-pixel. This alignment means the colour-fill phase can follow the exact contours established by the line-reveal phase without any repositioning.

The README specifies that each of the 8 typical scenes runs about 5 seconds, and each scene must be a different composition. Cropping or scaling one illustration to create multiple scenes is not permitted by the format rules: the README states that padding time that way is not allowed.

Installing the Dependency Stack

The project requires four runtime environments. Python 3.10 or later, Node.js 18 or later, FFmpeg, HyperFrames (from npm), and GSAP 3 (from npm). The README shows the clone and install sequence:

git clone https://github.com/xiejunjie524/handdraw-story-video.git
cd handdraw-story-video
python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt
npm install gsap

The Python requirements are minimal: numpy 1.24 or later, opencv-python 4.8 or later, and requests 2.31 or later. opencv-python handles the line art extraction from colour source images.

Next, prepare the HyperFrames vendor directory:

mkdir -p hyperframes/assets/vendor
cp node_modules/gsap/dist/gsap.min.js hyperframes/assets/vendor/gsap.min.js

GSAP is subject to its own licence terms, which are separate from the project's MIT code licence. The README instructs users to verify GSAP licence compliance before use.

Configuring a Story and Generating Line Art

Each story is a JSON file that lists scenes. Copy the provided template to start:

cp templates/story-template.json story.json

For each scene, generate the aligned line art from the colour source image:

python scripts/make_lineart.py assets/images/scene-01-color.png assets/images/scene-01-line.png

The minimal scene structure the README shows is:

{
  "id": "scene-1",
  "duration": 5,
  "caption_lines": ["凌晨四点，", "他总会经过那家早餐店。"],
  "line_image": "assets/images/scene-01-line.png",
  "color_image": "assets/images/scene-01-color.png",
  "crop": {"scale": 1, "x": 0, "y": 0}
}

The id, duration, line_image, and color_image fields are required. Caption lines and crop adjustments are optional. Full field documentation is in docs/story-spec.md. A working example lives at examples/soy-milk-at-4am/story.json.

Building and Rendering the Final Video

Once all images are in place and the story.json is complete, three commands run the full build:

python scripts/build_story.py story.json hyperframes/index.html --check-assets
npx hyperframes check hyperframes/index.html --json
npx hyperframes render hyperframes/index.html --output renders/story-v1.mp4 --workers 1

The first command generates the HyperFrames HTML page that encodes the animation. The --check-assets flag verifies that all referenced images and audio files exist before the build proceeds. The second command checks the generated page for errors. The third renders it to an mp4.

The README instructs that old renders should not be overwritten: each run uses a new output filename. The default output dimensions are 720x960 at 30 fps for a 40-second total duration. The README states these settings are suitable for vertical platforms.

The build_story.py script also runs automatic validation: it checks the total duration, looks for repeated assets, and verifies that caption line count and character sizes are within the documented limits.

Drawing Constraints That Affect the Source Illustrations

The project imposes specific composition rules on the source illustrations. The README documents these as guardrails for the pipeline to work correctly, not suggestions. The subject should be in the lower 45 to 55 percent of the frame. Large white paper-white areas should be preserved in the upper portion. Each frame should contain at most 2 to 3 people and two background anchor points. Dense crowds, packed buildings, and large dark-filled areas are explicitly excluded.

Captions are limited to 3 lines, with each line capped at approximately 18 Chinese characters. The README does not fix a background music track into the template, leaving music selection per story. The docs/prompts.md file documents fixed style prompts for image generation models that produce illustrations compatible with the pipeline.

The automatic validation in build_story.py enforces the duration and caption rules programmatically. Layout constraints (subject position, composition density) are the illustrator's responsibility before the images enter the pipeline.

Limitations, Licence, and When to Use a Different Tool

The project has no GitHub releases. The last push was on 2026-07-23, and the repository is not archived. The code is MIT licensed. Generated images, fonts, GSAP, BGM, and other third-party assets each carry their own licence terms; the MIT licence covers only the Python and build scripts.

The pipeline is designed for exactly the line-reveal-then-colour-fill format at 720x960. It does not support other animation styles, horizontal formats, or resolutions outside of custom configuration work. If your story uses illustration styles with dense black areas or complex backgrounds, the line art extraction may produce results that do not look correct in the animation phase.

HyperFrames is an external dependency with its own npm package maintenance cycle. The project pins no HyperFrames version, so a breaking change in a future HyperFrames release could interrupt the pipeline. The README does not document a process for locking or pinning that dependency.

For creators who want a no-code path to similar animated story formats, hosted platforms like CapCut or Adobe Express offer template-based animation without a local install stack. The trade-off is that those platforms control the output format, apply their own branding options, and do not support the specific line-reveal-then-colour-fill mechanic that this project centres on.

Editorial conclusion

handdraw-story-video is the right tool for creators who already have hand-drawn story illustrations and want a repeatable pipeline for 720x960 vertical videos without locking into a proprietary service. The dependency chain (Python 3.10+, Node.js 18+, FFmpeg, HyperFrames, GSAP 3) is non-trivial to configure. Creators who cannot or will not manage that stack should use a hosted video tool instead. The working example at examples/soy-milk-at-4am/story.json gives the fastest path to understanding the story.json format before building a new story.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. xiejunjie524/handdraw-story-video on GitHub