myccarl/ai-shortVideo-pipeline: a self-hosted AI short-video pipeline with failover and quality gating
End-to-end AI short-video production pipeline. FastAPI orchestration + Spring Boot gateway with multi-model failover, circuit breaker, metering, and full-stack observability. AI quality gating: prompt anchoring, CLIP consistency, AV sync auto-rescue.
At a glance
- What is it?
- The repository combines a FastAPI orchestrator with a Spring Boot gateway to run a seven-layer Chinese-language short-video pipeline, and adds circuit breaking, metering and CLIP-based consistency checks. It is Docker-first, model-agnostic, and heavier than a single-script generator.
- Who is it for?
- Adopt it if you already run Docker Compose, have DeepSeek and Kling credentials, and want a governed pipeline rather than a single prompt-to-video script. Skip it if you need a hosted service, generate English-first content, or cannot run Postgres, Redis and MinIO alongside it.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 16 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 18, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What myccarl/ai-shortVideo-pipeline actually automates
The README describes myAiVideos as an end-to-end automated short-video production pipeline with the tagline "One command in, a publish-ready video out." The scope it names is Chinese-language short video, covering topic discovery, creative generation, visuals, audio, post-production and distribution. It lists five suitable formats: hot-news commentary, knowledge explainers, emotional storytelling, curiosity and strange facts, and social insight clips.
The intended user is not someone who wants a video from a text box. It is a team that already produces this kind of content and wants the steps chained, instrumented and recoverable. That distinction matters because the repository ships a governance layer (auth, routing, circuit breaker, metering) alongside the generation code. A solo creator with one provider key will find most of that layer idle. An operator running several model vendors, or one who has been burned by a provider timing out mid-job, is the audience the design assumes.
The seven layers and where the Java gateway sits
The architecture is split into two runtimes. Python (FastAPI, asyncio, Pydantic, SQLAlchemy with asyncpg, Alembic, ARQ) holds the pipeline. Java (Spring Boot 3.5, WebClient, Resilience4j, Caffeine, Prometheus) holds the gateway. The README's diagram puts the client at the top, the gateway beneath it, then the FastAPI orchestrator, which talks both to an ARQ worker backed by Redis and to the seven-layer pipeline, which in turn calls DeepSeek, Kling and TTS providers.
The seven layers are numbered L1 through L7 under the layers/ directory: L1_trending for topic discovery and ranking, L2_creative for script generation and prompt anchoring, L3_visual for text-to-image, image-to-video and CLIP consistency, L4_audio for TTS, mixing and duration planning, L5_postprod for AV sync, rescue, captions and cover, L6_distribution for delivery back to the client, and L7_optimization for quality scoring and data feedback. The README states the layers are decoupled and independently replaceable, which is the claim worth testing against your own content type.
The gateway is the part that differs from most repositories in this space. It handles JWT, trace ID propagation, routing, circuit breaking and metering. Model failover across DeepSeek, Qwen and GLM is implemented with Resilience4j, and the README says failed providers are automatically rotated. Metering uses an AOP aspect that captures token and cost figures without code changes and aggregates them by tenant. That is platform work, and it is why the project runs two languages instead of one.
Installing myccarl/ai-shortVideo-pipeline with Docker Compose
The README lists Docker Desktop 24 or newer as the prerequisite, with Node 20 only for frontend development and Python 3.11 only for running outside Docker or running tests on the host. The clone step in the README uses a placeholder remote, so substitute your own fork URL.
git clone <your-fork>.git
cd myAiVideos
cp .env.example .envAfter copying the env file, the README says to edit .env and fill in at least the required AI provider API keys. The configuration table names DEEPSEEK_API_KEY for the text LLM, GLM_API_KEY for multimodal moderation, and KLING_ACCESS_KEY plus KLING_SECRET_KEY for text-to-image and image-to-video. Speech synthesis uses either the VOLCENGINE_TTS_* or MINIMAX_TTS_* variables. GATEWAY_AUTH_JWT_SECRET signs gateway tokens and the README requires it to be at least 32 bytes. Langfuse keys are marked optional.
Start the backend containers, then run migrations, then bring up the gateway:
docker compose up -d postgres redis orchestrator worker
docker compose exec -T orchestrator alembic upgrade head
docker compose up -d gatewayThe frontend is a separate Vue 3 application in sl-vue:
cd sl-vue && npm install && npm run devThe README gives three health endpoints to check: the Python API at http://localhost:8000/health, the Java gateway at http://localhost:8080/actuator/health, and the frontend at http://localhost:5173. If those respond, the stack is up. Note that docker-compose.yaml sets LLM_USE_GATEWAY to "1" and GATEWAY_INTERNAL_BASE to http://gateway:8080, so text calls are expected to route through the Java gateway rather than hitting DeepSeek directly. Running the orchestrator without the gateway container is therefore not the documented configuration.
The quality gating is the part with no obvious equivalent
Three mechanisms in the README address a problem that shows up in any multi-shot generated video: the subject drifts. Prompt anchoring maintains visual identity of subjects across segments. CLIP text-image consistency gating rejects off-prompt keyframes early, before they are assembled. Audio-video sync auto-rescue applies a four-tier strategy: audio tempo adjustment, video padding, and narration rewrite. The README lists three tiers explicitly and describes the strategy as four-tier, which is a small inconsistency in the documentation.
The ordering here is deliberate. Rejecting a bad keyframe at L3 costs one image generation. Discovering the same problem after assembly costs a re-render. The AV rescue ladder follows the same logic: stretch the audio first, pad the video second, rewrite the narration last, because rewriting narration is the most expensive option and the one that changes the creative output. Whether the CLIP threshold is tunable is not documented in the README, and that is the first thing to check in the configuration if you adopt this.
Resource limits and the cases where this is the wrong tool
The docker-compose.yaml sets mem_limit to 2g and cpus to 1.5 on the shared app base. That is a real constraint, not a default to ignore. The Dockerfile installs torch from the Tsinghua PyPI mirror, and a comment in the file states the mirror bundles CUDA, which adds one to two gigabytes to the image; the comment also notes the CPU-only index URL as the alternative if disk is tight. faster-whisper and transformers both pull model weights, and the compose file points HF_ENDPOINT at hf-mirror.com with HF_HOME under /app/data/hf_cache so the download survives restarts. There is also a WHISPER_MODEL_PATH option for a manually placed model, which avoids the HuggingFace download entirely.
This is a self-hosted stack that wants Postgres, Redis, MinIO and two runtimes. If your output is English-language video, the pipeline's model choices and prompt templates are oriented toward Chinese content, and the documentation is partly in Chinese (docs/操作使用手册.md, docs/启动-停止脚本.md). If you need a managed service with an SLA, nothing here provides one. If your workflow is a single prompt producing a single clip, the gateway, the metering aspect and the seven layers are overhead you will pay for and not use. The project also has no retrieved releases, so there is no versioned artifact to pin; you track the main branch.
How this differs from a single-model generator script
A typical open source short-video generator is a Python script that calls one video API and writes a file. The difference here is the gateway. Resilience4j circuit breaking means a provider that starts failing is taken out of rotation rather than retried until the job dies, and the README states failed providers are automatically rotated across DeepSeek, Qwen and GLM. That only matters if you configure more than one text provider, which the README does not present as mandatory.
The second difference is cost visibility. The metering aspect captures token and cost figures through AOP without code changes and aggregates them by tenant, which a single script does not give you. The third is tracing: trace_id is propagated across the Java and Python processes, and Langfuse provides call-tree visibility, with the compose file passing LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_HOST into the container and defaulting LANGFUSE_ENABLED to false. If you leave tracing off, you keep the queue and failover but lose the per-call view. The trade-off is real: you are running a Java service to get these properties.
Upgrading, licence and what to verify before you commit
The project is MIT licensed, which permits commercial use and modification; the LICENSE file is at the repository root. Note that the licence covers the code, not the model outputs. Provider terms for DeepSeek, Zhipu GLM, Kling, Volcengine, MiniMax and the optional Atlas Cloud and MuAPI backends are separate agreements, and the README does not address what those terms permit you to publish. That is a question for whoever owns your distribution channel, not for the repository.
Upgrade cost is mostly image rebuild time. The README's own development section gives the sequence after Python changes: build the orchestrator image, then recreate the orchestrator and worker containers with --force-recreate --no-build. Database changes go through Alembic, and the quick start runs alembic upgrade head inside the orchestrator container, so schema migrations are versioned. The README does not document a rollback procedure for a failed migration, and it does not document downgrade steps for a bad image. Given the absence of tagged releases, plan on rebuilding from a known commit rather than pinning a version.
The last push to the repository was on 2026-09-02. Before adopting, verify three things: that the layer directories under layers/ correspond to the stages your content actually needs, that GATEWAY_AUTH_JWT_SECRET is set to at least 32 bytes in your .env, and that you have keys for every provider the layers call, since a missing speech key will not fail until L4.
Editorial conclusion
Adopt it if you already run Docker Compose, have DeepSeek and Kling credentials, and want a governed pipeline rather than a single prompt-to-video script. Skip it if you need a hosted service, generate English-first content, or cannot run Postgres, Redis and MinIO alongside it. Before committing, verify that the seven layers under layers/ match your content format, that the gateway's JWT secret is at least 32 bytes, and that your provider keys cover text, image, video and speech.
Frequently asked questions
What is an AI video production pipeline and what are its stages?
In this project the stages are the seven layers listed in the README: topic discovery, creative generation, visuals, audio, post-production, distribution and optimization. Each layer lives in its own directory under layers/ and the README states they are decoupled and independently replaceable.
What is the AI pipeline in myccarl/ai-shortVideo-pipeline?
It is a FastAPI orchestration core plus a Spring Boot gateway that coordinates multiple AI models for Chinese-language short-video generation. The gateway handles auth, routing, circuit breaking and metering, while the Python side runs the seven-layer pipeline and an ARQ worker backed by Redis.
Can AI videos be monetized on YouTube?
The repository does not address platform monetization rules. It is MIT licensed, which covers the code, but the README does not state what the model providers' terms allow you to publish, so that question has to go to the provider agreements and the platform itself.
Community notes