StreamerHelper web-server: the queue-and-FFmpeg backend behind a Chinese streaming recorder

The service half of a product, not a standalone tool

The repository description is Core backend services for StreamerHelper, and the README's opening line is in Chinese: the StreamerHelper backend service, responsible for live status polling, recording and media processing, task scheduling, object storage, Bilibili submission and external platform authentication.

That first line matters for an adopter, because this repository is not self-contained in a useful sense. The README says the complete product description is in the StreamerHelper repository, and that containerised deployment is managed by the infra repository. So the code here, the front end elsewhere, and the deployment tooling in a third place.

The processing chain is where the shape of the product becomes clear:

直播状态轮询
  → 创建录制任务
  → FFmpeg 录制和分段
  → 字幕 / ASR / 转码
  → 对象存储
  → B 站投稿

In English: poll the live status, create a recording task, record and segment with FFmpeg, produce subtitles, speech recognition and transcodes, push to object storage, and submit to Bilibili. Every arrow is a piece of this repository, and the chain runs on a queue rather than in a request handler.

The module table spells out the boundaries. Platform adaptation reads live status and stream information from Bilibili, Huya, Douyu and Douyin. Recording starts and stops FFmpeg, handles segmentation, state recovery and abnormal termination. Media processing covers danmaku collection, ASS subtitles, transcoding and audio extraction. The task system uses BullMQ for polling, recording, transcoding, ASR and submission jobs. The Bilibili integration covers login authorisation, submission management, upload and status synchronisation. Storage manages PostgreSQL business data and S3-compatible object storage. Observability is Bull Board, structured logging, health checks and unified notification.

The primary language is TypeScript, the licence is MIT, the default branch is main, and the last push was on 2026-09-21 with no GitHub releases published. The version in package.json is 2.0.0, and the package is scoped as @streamerhelper/web-server, which is the clearest signal that this is an internal monorepo component rather than a general-purpose npm package.

The stack, and why Midway.js rather than Nest or Express

The technology table is short and the choices are legible.

HTTP is Midway.js 3 with Koa, the language is TypeScript, the database is PostgreSQL with TypeORM, the queue is Redis with BullMQ, object storage is the AWS SDK against an S3-compatible interface, media is FFmpeg, and browser automation is Puppeteer.

The Midway.js choice is the interesting one. Midway is a serverless-style, decorator-driven framework for Node, and it is used here for the full stack of concerns: bootstrap, Koa adapter, TypeORM integration, BullMQ integration, validation and logging, all as separate @midwayjs packages on version 3.12. The package.json lists them individually, @midwayjs/bootstrap, @midwayjs/bull-board, @midwayjs/bullmq, @midwayjs/core, @midwayjs/koa, @midwayjs/logger, @midwayjs/typeorm and @midwayjs/validate, which tells you the framework is not a thin layer but the thing that owns the application lifecycle.

The BullMQ choice follows from the shape of the work. A recording is a long-running job that must survive a restart, a transcoding is a job that should not block a request, and the platform polling runs on a timer. BullMQ is a Redis-backed queue with retries, concurrency control and a dashboard, and @midwayjs/bull-board is the dashboard integration that makes those queues visible at a URL. The Bull Board is served at /ui in the local development setup, which is the sort of thing that makes a queue-based design legible to a developer who has just started the stack.

The Puppeteer dependency is puppeteer-core, not puppeteer, which is the deliberate variant that does not download a bundled Chromium. The browser is pointed at something else, and since the platform adapters have to read live status pages, a headless browser is the pragmatic way to deal with sites that render their state client-side.

FFmpeg comes in through fluent-ffmpeg, with @types/fluent-ffmpeg for the typings, and dayjs for the date handling that scheduling needs.

Running it locally, and the six services that come up

The local development section lists the requirements first: Node.js 16 or newer, Docker Engine with Docker Compose v2, and FFmpeg, which must be installed when you need to run local recording or media tasks.

The startup sequence is four commands:

npm ci
docker compose -f docker-compose.dev.yml up -d
npm run migration:run
npm run dev

The first brings up the dependencies in Docker, the second runs the TypeORM migrations against the development database, and the third starts the service in watch mode. The dev script is `cross-env NODE_ENV=local mwtsc --watch --run @midwayjs/mock/app.js`, so the development server runs against the mock application and the compiler watches for changes.

The default entry points are the useful part to know, because they tell you what the stack looks like:

| 服务 | 地址 | | --- | --- | | API | http://localhost:7001 | | Bull Board | http://localhost:7001/ui | | PostgreSQL | localhost:5432 | | Redis | localhost:6379 | | MinIO API | http://localhost:9000 | | MinIO Console | http://localhost:9001 | | pgAdmin | http://localhost:5050 |

MinIO is the S3-compatible object storage in development, with the AWS SDK used against it, which is why the S3 client is in the dependency list and why the infrastructure runs a MinIO service in the first place. pgAdmin is there because you are going to look at the database while developing, and Bull Board is there because the queues are where the interesting failures are.

One warning is stated in the README and is worth repeating: the credentials in docker-compose.dev.yml are for local development only and must be replaced before exposing anything to a network that is not your own machine. The compose file at the root of the repository is the development one, docker-compose.dev.yml, and there is a docker-entrypoint.js in the tree that the container uses to start the service.

The README is written in Chinese, which is consistent with the product's audience, and the table of contents links to the local development, system boundaries, configuration, notification, command reference and production deployment sections.

Configuration resolution, and why the order matters

The configuration section describes a five-level lookup, resolved in this order: the directory given by `CONFIG_DIR`, then a settings.json in the project root, but only in local or development environments, then /app/config, then config/settings.json in the current working directory, and finally ~/.streamer-helper/settings.json.

Two properties of that list are worth pulling out. The first is that the project root is only consulted outside production, which is a sensible guard against a development settings file shadowing a deployed configuration. The second is that the last two entries are per-user locations, so a single server can carry different settings for different accounts, and a developer can keep a personal configuration without touching the deployed one.

Environment variables take precedence over file configuration, which is the usual arrangement and the reason the configuration directory concept exists at all.

For production, the README is specific: production configuration should be generated through the infra configuration tool, and keys, cookies and access tokens should not be committed to the repository. That is the difference between a configuration system designed for a container and one designed for a developer machine, and it is why the five-level lookup exists at all: the same code has to run in a developer's checkout and in a container that Infra has populated.

The API surface is organised by resource. The controller table lists /api/streamers for streamer configuration and live status, /api/jobs for job query, control and recovery, /api/bilibili for Bilibili authentication, upload and submission, /api/douyin for the Douyin login and verification flow, /api/text for text and subtitle processing, and /api/system for system status and runtime configuration.

The front end reaches these endpoints through an Nginx proxy, and the README notes that in production the Backend port does not need to be exposed directly. That is a small architectural decision with a real consequence, because the API can then create recording jobs and drive FFmpeg without being reachable from outside the machine, and the front end talks to it through the same origin.

The notification service and its fatigue suppression

Notification is handled by a NoticeService that provides a unified entry point. Business code can send notifications directly, and a logging Transport can also capture exception logs by level.

The flow is drawn in the README as three steps: a business event or a log goes to the NoticeService, then to fatigue suppression, then to a notification channel. Fatigue suppression is the interesting part, and it is a real operational requirement rather than a nicety.

The default fatigue time for log notifications is 300 seconds, and the same event within the fatigue window is only sent once. That matters because the alternative is a storm: a failing transcoder that logs the same error every few hundred milliseconds would otherwise generate a notification per log line, and the person receiving them would either mute the channel or stop reading it. Suppressing duplicates inside a window keeps the channel meaningful.

The failure behaviour is stated too, and it is the right choice: a send failure is written to a separate internal log and does not interrupt the business flow. A notification is a side effect of the work, and a notification service that can fail a recording job because a webhook was down would be a serious design error.

The environment variables are the configuration surface: NOTICE_ENABLED turns the unified notification on, NOTICE_LOGGER_ENABLED turns log capture on, NOTICE_LOGGER_LEVEL sets the minimum level that is captured, NOTICE_LOGGER_FATIGUE_SECONDS sets the default fatigue window, and SERVERCHAN_SENDKEY carries the SendKey for the Server酱 channel, which is the only channel built in today.

The extension point is explicit: to add an email or SMS channel you implement NoticeChannel and register it with NoticeService, and the logging integration layer does not need to change. That is the right shape for a notification system, because the capture side stays simple and the delivery side grows.

Where the web-server is the wrong thing to adopt

There are three limits, and the first one is the boundary of the repository itself.

The first is that this is a component of a product, and the product's documentation, front end and deployment all live elsewhere. The README says the complete product description is in the StreamerHelper repository and that containerised deployment is managed by the infra repository, and it explicitly advises against maintaining a separate production Compose configuration of your own, so that ports, volumes and migration order do not drift. That advice is correct and it is also a constraint: you are adopting a component whose operational story is defined by another repository.

The second is the platform adapters, and it is the one that will break. The platform adaptation module reads live status and stream information from Bilibili, Huya, Douyu and Douyin. Those are four separate services, each with its own web interface, and the ones that render their state client-side are read with Puppeteer. Any of them can change its markup, its API shape or its anti-bot behaviour, and the project will not get a notification when that happens. A tool whose value depends on scraping four live-streaming sites inherits the maintenance cost of all four, and the browser automation makes it worse rather than better, because a headless browser is exactly the thing those sites change their defences against.

The third is the scope of the media pipeline. Recording, segmentation, danmaku collection, ASS subtitles, ASR speech recognition, transcoding and audio extraction, each of which is a real piece of engineering that FFmpeg and the subtitle tools do competently but that needs configuration, and none of which this repository documents in detail. The README's command reference is about the project's own commands, not about how a recording is configured.

And a practical one: the environment expects Node.js 16 or newer, Docker Compose v2, FFmpeg, a PostgreSQL, a Redis, an S3-compatible store and a browser for Puppeteer, plus Puppeteer needs a Chromium you point it at. That is a substantial machine for a development environment, and the tool is a good deal further from a single binary than the BullMQ design might suggest.

Editorial conclusion

Adopt this backend if you are extending StreamerHelper itself, because it is the service half of a product with a separate front end and a separate infrastructure repository, and the documented deployment path runs through that infra tool. Do not adopt it as a standalone streaming recorder, since the platform adapters, the Bilibili integration and the front end are all part of the wider product. Verify first that the platform adapters still match the current behaviour of Bilibili, Huya, Douyu and Douyin, since those change without notice, and that your environment provides FFmpeg and Docker Compose v2 before the documented development sequence will run.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. StreamerHelper/web-server on GitHub