AIComicBuilder: a self-hosted pipeline from script to animated video

From script to animated video: the 9-stage AI pipeline

AIComicBuilder breaks animated comic production into 9 sequential stages: script input, script parsing, character extraction, character four-view generation, smart storyboard, start/end frame generation (or scene reference frame mode), video prompt generation, video generation per shot, and final video synthesis with subtitle burn-in. Each stage can be triggered individually or in batch, giving the operator full control over pacing.

The storyboard page provides 3 views for working with shots: an editing drawer, an inline character panel, and a kanban board that groups shots by generation progress automatically. Multiple storyboard versions can exist for the same project, allowing comparison between different directorial approaches or AI parameter settings.

The project's data model captures the pipeline state across 5 entity types: Project (script and status), Character (name, description, reference images), Shot (index, prompts, duration, start and end frames, video), Dialogue (character, text, audio), and Task (background task queue). Everything persists in a local SQLite database.

Character four-view generation and how it maintains cross-shot consistency

For every character in a project, AIComicBuilder generates a four-view reference set: front, three-quarter, side, and back views. An AI image model produces these 4 images, which then feed into subsequent shot generation as visual constraints, keeping character appearance consistent across camera angles.

Style detection runs from the script before generation begins. The system identifies whether the script calls for an anime or realistic style and applies the appropriate parameters to both the character four-view and the start/end frame generation steps. Characters can be shared across episodes within a project or parsed independently per episode, and the UI separates main and supporting roles in the character list.

The four-view step is where the pipeline departs from a simple prompt-to-image workflow. Skipping it leaves each shot's AI call without a character reference, producing inconsistent character appearance across shots.

Running locally with pnpm, or via Docker on port 3000

Local setup requires Node.js 18 or newer, pnpm, and FFmpeg (which handles video synthesis). With those in place, 3 commands start the application:

pnpm install
pnpm drizzle-kit push
pnpm dev

The second command pushes the Drizzle ORM schema to the SQLite database before first run. After `pnpm dev`, the application is accessible at http://localhost:3000.

Docker is the alternative. The pre-built image is available at twwch/aicomicbuilder:latest:

docker run -d \
  --name ai-comic-builder \
  -p 3000:3000 \
  -v ./data:/app/data \
  -v ./uploads:/app/uploads \
  --platform linux/amd64 \
  twwch/aicomicbuilder:latest

The `--platform linux/amd64` flag is written into the README's Docker command. The 2 volume mounts keep data between container restarts: `./data` holds the SQLite database and `./uploads` holds uploaded files and generated assets.

SQLite for project data, FFmpeg for video, and the two storage env vars

SQLite stores all project data through Drizzle ORM. For local development, the .env.example file gives the default storage paths:

DATABASE_URL=file:./data/aicomic.db
UPLOAD_DIR=./uploads

The Docker image sets slightly different absolute paths for those same variables. Both point to directories that must persist; losing the uploads directory means losing generated images and video segments.

FFmpeg is required for video synthesis. The Docker image installs ffmpeg with libass support for subtitle burn-in, and also adds the font-noto-cjk package for CJK character rendering in subtitles. For a local non-Docker install, FFmpeg with libass support must exist on the host machine separately. The fluent-ffmpeg npm package calls the system FFmpeg binary, so a version mismatch between the expected and installed FFmpeg can break video synthesis without clear error output.

Configuring AI providers: text, image and video models set per project

Provider configuration happens in a settings page inside the application after startup. Text generation uses OpenAI or Gemini through the AI SDK. Image generation can use OpenAI DALL-E, Gemini Imagen, or Kling. Video generation can use Seedance, Kling, or Veo. Each category can draw from a different provider, and configuration can differ between projects.

Using AI providers directly, without AIComicBuilder, means managing prompts and file transfers between each step manually. AIComicBuilder's contribution is the pipeline layer: it stores shot metadata in SQLite, generates video prompts automatically from storyboard descriptions, and sequences the calls. For a single one-off video with no character reuse, direct provider access involves less setup. For multi-episode work where characters must stay consistent, the managed pipeline justifies the local infrastructure.

The AI provider logic lives under src/lib/ai/ in the project structure, alongside the prompt files, so prompt customization is a matter of editing those files before running `pnpm build`.

No API keys means no output: credit requirements and what each stage costs

Every AI operation in the pipeline makes an outbound call to an external service. Nothing generates locally. The video generation step calls Seedance, Kling, or Veo on every shot; a project with many shots multiplies the credit cost proportionally. No cost estimate or minimum credit recommendation appears in the repository or the README.

Text and image generation calls occur earlier in the pipeline during script parsing, character extraction, four-view generation, storyboard creation, and frame generation. Each of those steps also consumes API credits from the configured text and image provider.

For someone evaluating the tool before committing to API spend, the four-view and storyboard stages are the cheapest to test because they involve fewer calls than video generation. The video generation step is where costs scale with project scope.

NEXT_TELEMETRY_DISABLED set in Docker but absent from the local .env.example

A reading of the Dockerfile shows `ENV NEXT_TELEMETRY_DISABLED=1` in the production stage, which disables Next.js data collection for any container started from the image. The .env.example file used for local development does not include this variable. A local installation using only the defaults has Next.js telemetry active unless the developer manually adds `NEXT_TELEMETRY_DISABLED=1` to their .env file.

A second detail sits in the package.json: it declares `"version": "0.1.0"` while the repository's most recent tags are v0.2.4, v0.2.5, and v0.2.6 from April 2026. The package.json version was set at project creation and not updated as tags advanced, so it does not reflect the actual state of the software.

The last push was on 2026-04-27, and source code is under Apache-2.0.

Editorial conclusion

AIComicBuilder suits anyone who needs a managed pipeline for generating animated comic segments from a script, where character consistency across shots matters and manual prompt and file management would slow the work. It is the wrong tool for a one-off video with no reusable characters, because the episode and character management overhead does not pay off for a single clip. Before starting, verify that API subscriptions to at least one text provider, one image provider, and one video provider are in place with enough credits for the shot count, since every generation step calls live external APIs and no cost estimate appears in the repository.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. LingyiChen-AI/AIComicBuilder on GitHub
3. Project website
4. README
5. Releases