# AI-YinMei: a Python backend for an AI streamer with barrage, speech and a Live2D avatar

> AI-YinMei is a Chinese-language streaming platform backend written in Python that aggregates live chat from several platforms, talks through a large language model, speaks with a choice of TTS engines and drives a Live2D or desktop pet avatar. It suits streamers who want a full AI host stack, and the open source release stops at version 1.8.1 without the administration console.

**worm128/AI-YinMei** — AI吟美-人工智能主播-Vtuber-桌宠-智能体

- Repository: https://github.com/worm128/AI-YinMei
- Website: https://www.yinmei.vip
- Stars: 1,028 · Forks: 140
- Language: Python
- License: BSD-2-Clause
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/worm128-ai-yinmei

## A streaming host, not a language model

The project name translates roughly as AI Yinmei, and the repository description lists it as an AI streamer, a VTuber, a desktop pet and an agent. The key point for evaluation is that this is the platform around a model, not the model itself. The primary language is Python, the API entry point is api.py, and the repository layout shows the shape of the system: config.yml and a config/ directory for settings, func/ for the feature modules, background/ for scheduled work, html/ for the web interface, output/ for produced media, and a test/ directory.

The centre of the product is a livestream host that listens to chat, understands what a viewer is asking, answers with a large language model, speaks the answer aloud, and moves an avatar while it does so. The README describes the scope as a general purpose artificial intelligence platform supporting aggregated live streaming, a QQ bot and a desktop pet, and emphasises that it is lightweight and ready to use out of the box.

That position in the stack explains the dependency list, which is a good map of the work involved. The requirements pin Flask 3.0.0 with Flask-APScheduler 1.13.1 for the API and the scheduled jobs, edge-tts 6.1.6 for one of the speech paths, pyvirtualcam 0.11.0 and opencv-python for the virtual camera and camera analysis, obs-websocket-py 1.0 for controlling OBS, bilibili-api-python 16.1.1 for the Bilibili side, and websocket-client, aiohttp, requests, PySocks and duckduckgo_search for network work.

## The open source release stops well short of the product

This is the first thing to sort out, because the README is explicit about it. Under the project statement there are two versions. The open source version is 1.8.1, and the note says that this version was open sourced on GitHub but has no administration interface. The complete version is 3.0.0-Pre, which does have the full management console, with intent analysis, emotion analysis, voiceprint recognition, diffusion thinking, a points system and a user system, and the README states plainly that the complete version is not open source.

The header of the README carries a different number again, version 2.4.4, which is the version of the downloadable package described in the quick start section. So there are three version lines in play: the open sourced 1.8.1, the packaged 2.4.4, and the closed 3.0.0-Pre. The GitHub releases list matches the older line, with v1.8.1 from 2024-09-01, v1.8.0 from 2024-07-27 and v1.7.7 from 2024-07-16.

The licence of the repository itself is BSD-2-Clause, so the code that is published is permissively licensed, but the permission does not extend to the closed parts. Anyone evaluating this should decide early whether the 1.8.1 feature set is enough, because the console that makes the product operable for a non-developer lives in the version that is not open.

## Chat from nine sources, and 27 tools the model can call

The aggregation layer is what distinguishes this from a chatbot. The README describes aggregated barrage supporting nine source channels: the Bilibili developer platform, napcat for QQ bots, barragefly covering Douyin, Huya, Kuaishou and Douyu, WeChat Live, the desktop pet itself, and backend chat conversations. A separate note lists supported platforms as Douyin, Bilibili, Kuaishou, Douyu, Huya, WeChat Live and QQ bots.

On top of the chat sits intent analysis, which the README describes as a 10 millisecond multi-head attention mechanism that produces an intent classification and an emotion classification together, and which can be trained on custom corpora. The distinction matters in practice: knowing that a message is a song request, a complaint, or small talk lets the system route it differently before the language model is involved at all.

The tool set is large and is the other half of the product. The README lists 27 tools, including camera observation, scheduled patrol, visual capability, image search, greeting, Wikipedia, TMDB movie ratings, browser control, a calculator, a points lookup, advanced search, chain of thought, product lookup, a stocks tool, timezone conversion, random numbers, a cooking guide, changing the character's outfit, moving and rotating the character, video lookup and playback, toggling speech synthesis, speech rate control, AI video, AI singing, AI drawing and a search tool. MCP tools are supported, and you can also write your own code tools. The README claims that mounting a hundred large model tools does not cause lag, that cloud service tool selection completes within 500 milliseconds and local model tool selection within 200 milliseconds.

## Speech, voiceprint and the latency that makes it feel live

The audio side is where most of the engineering effort is described. Four TTS routes are supported: cosyvoice2, bert-vits2, gpt-sovits and edge-tts, plus any TTS cloud vendor following the OpenAI specification, which the README puts at about 80 percent of the market. The cosyvoice2 build is a modified, fine-tuned version of the upstream model, published separately as a Docker image under worm128/yinmei-cosyvoice, with vllm acceleration applied.

Two details handle the problem of a talkative room. VAD activity detection is described as powerful noise rejection that stops the AI from mistaking the streamer's own voice, played through the speakers, for a viewer speaking. The desktop pet character carries a microphone with noise suppression, VAD and acoustic echo cancellation, layered with the voiceprint system so that a real-time conversation is not triggered by accident.

Latency is stated rather than implied: the streaming rework is said to bring speech conversation to a 600 millisecond response. Singing is also supported, with the README stating that learning one song takes roughly 150 to 200 seconds and that the singing model can train a voice timbre on its own.

Search and knowledge are delegated outward. The knowledge base uses FastGPT, web search goes through a searxng aggregation platform that the README notes requires a proxy, and image search uses Baidu.

## Memory split between MongoDB, neo4j and FastGPT

The persistence story has three distinct parts, and separating them is the clearest way to understand the design.

Short-term memory is the model context, recorded in a MongoDB database, and the README states that all chat records are persisted in full. Long-term memory is separate: it is woken selectively according to time expressions in a message, so a question about last week pulls last week's context while an unrelated question does not, with configurable counts and similarity thresholds so that memory retrieval does not slow the main flow.

The third part is the graph. Diffusion thinking uses neo4j as a graph database to build relationships between words, and the README says group chat data can be exported to train those relationships, with the resulting graph displayed in neo4j. That is the mechanism behind the open-ended conversation style, where associations rather than a retrieval chain drive the reply.

Alongside memory sits the points system, where chatting, liking, gifting and registering add points, while singing, drawing, video playback, song switching and lotteries deduct them, and a points configuration plan is provided. Other operational features listed at the end include idle tasks, room welcome messages, intelligent installation of docker services, server monitoring and statistics.

## Starting the packaged build on port 9000

The README describes the quick start as downloading a package and running a binary, so this is not a pip-first project. The download path is the 吟美核心 folder inside the distributed package, and the application archive is named AI-YinMei-v2.4.4.zip. Startup is by double-clicking start.bat or the yinmei-core-api.exe executable, and once it is running the interface is served at http://127.0.0.1:9000.

The Python side of that package is defined by the requirements file, which pins exact versions rather than ranges:

```bash
Flask==3.0.0
Flask-APScheduler==1.13.1
APScheduler==3.10.4
edge-tts==6.1.6
pynput==1.7.6
duckduckgo_search==4.1.1
pyvirtualcam==0.11.0
opencv-python==4.8.1.78
numpy==1.26.3
Pillow==10.0.1
websocket-client==1.7.0
aiohttp==3.9.5
obs-websocket-py==1
requests==2.31.0
```

For deployment on a server, the Docker route is published on Docker Hub as worm128/yinmei-core, and the README also lists a cosyvoice2 image there. The documentation site at docs.yinmei.vip carries the setup guides, with separate pages for the core chat configuration, the voice synthesis configuration and starting a conversation. A public demo runs at demo.yinmei.vip.

The distribution model is worth noting before you plan an install. The Windows and Linux builds are offered as a download through a cloud drive, and the feature bundle directory lists separate archives for the analysis module, the desktop pet 2.0, the GPT-SoVITS family, bert-vits2, cosyvoice2, an image moderation module, a Stable Diffusion web interface, Live2D and VTube Studio assets, a bundle of MCP plugins, and an OBS configuration file.

## Where this platform is the wrong tool

Several constraints are structural.

The first is the split between what is open and what is not. If you need the administration console, the points and user management, or the diffusion thinking features as described in the complete version, you are using closed software that the repository does not contain, and the open source release will not get you there.

Second, the packaging is Windows-first. The quick start is a start.bat and a Windows executable, the cloud drive distribution is a Windows archive, and the repository also carries a start.bat file at the root. Running on Linux means using the Docker image instead and accepting that the desktop pet, the virtual camera and some of the screen-control tools behave differently without a desktop session.

Third, the external dependencies are not optional in practice. Search needs a searxng instance and, per the README, a proxy to reach the engines behind it. Image search depends on Baidu, drawing depends on models downloaded from civitai.com, the knowledge base needs FastGPT, and the speech engines need their own model weights. A working installation is an assembly job, not a single command.

Fourth, and this is the one to weigh for a public deployment, the component that talks to your live room and executes tools including browser control and keyboard and mouse control is a large attack surface, and the shipped configuration exposes an HTTP interface on port 9000. The public demo credentials are listed in the README as admin, so the security posture of any instance you run is entirely yours to define.

## Against a plain model API wired into OBS

The alternative is the setup most people start with: an LLM API of your choice, a text-to-speech service, a Live2D or VTube Studio model in OBS, and a chat bot that reads the platform's own danmaku feed. The pieces are all available separately, and a technical streamer can wire them together in a weekend.

The difference in approach is where the integration work lives. In the DIY arrangement, every cross-cutting feature is yours: the chat from several platforms is one script per platform, the avatar is triggered by whatever mechanism you write, the tool set is a dispatch table you maintain, and the memory is whatever the framework you chose already does. When the stream needs a new capability, that is your code.

AI-YinMei packages those integrations as one product with a shared configuration, a documented tool set and a management layer, and it adds the parts a single-model script does not have: intent and emotion classification before the model is called, VAD and voiceprint handling so the bot does not talk over itself, points and user management, and a graph-based association layer.

The cost is the dependency list, the assembly work, the Chinese-language documentation and tooling, and the fact that the most convenient parts are closed source. For a Chinese-speaking streamer who wants the whole stack on day one, that is a good trade. For a streamer who wants exactly one language model and a talking avatar, the DIY route is smaller and fully yours to control.

## Conclusion

Adopt AI-YinMei if you run a Chinese-language stream and want a working AI host with chat aggregation, speech, a Live2D avatar and tool calling, and if you are willing to assemble the surrounding components yourself. Do not expect the open source release to be the product: the README states that the open sourced version is 1.8.1 and has no administration interface, while the complete 3.0.0-Pre version with the console is not open source. Verify first that you can supply the pieces the download does not include, since the package bundles are distributed through a cloud drive and the extras cover separate speech models, Live2D assets and OBS configuration. The licence of the repository is BSD-2-Clause, and the last push was on 2026-10-06.

## FAQ

### Which version of AI-YinMei is actually open source?

The README states that the open sourced version is 1.8.1 and that it has no administration interface, while the complete 3.0.0-Pre version with the management console, intent and emotion analysis, voiceprint and points system is not open source. The repository itself is licensed BSD-2-Clause.

### How do I start AI-YinMei and where is the interface?

Download the package, take the AI-YinMei-v2.4.4.zip application archive from the 吟美核心 folder, then start it by running start.bat or yinmei-core-api.exe. Once running, the interface is served at http://127.0.0.1:9000, and there is also a Docker image published as worm128/yinmei-core.

### Which speech engines does AI-YinMei support?

The README lists cosyvoice2, bert-vits2, gpt-sovits and edge-tts, plus TTS cloud vendors that follow the OpenAI specification, which it puts at about 80 percent of the market. The cosyvoice2 build is a modified and fine-tuned version of the upstream model, published as its own Docker image.

### Where does AI-YinMei store chat history and long-term memory?

Short-term memory, meaning the model context and the chat records, is stored in MongoDB with all chats persisted. Long-term memory is woken selectively by time expressions, and diffusion thinking uses a neo4j graph database to build relationships between words, with the knowledge base handled by FastGPT.

### Can AI-YinMei aggregate chat from more than one platform?

Yes. The README describes aggregated barrage across nine source channels including the Bilibili developer platform, napcat for QQ bots, barragefly covering Douyin, Huya, Kuaishou and Douyu, WeChat Live, the desktop pet and backend conversations.

### What external services does AI-YinMei need?

The knowledge base uses FastGPT, web search goes through a searxng aggregation platform that the README says needs a proxy, image search uses Baidu, and drawing downloads models from civitai.com. Speech needs its own model weights, so a working setup is an assembly rather than a single step.

## Sources

- [License: BSD-2-Clause](https://github.com/worm128/AI-YinMei/blob/master/LICENSE)
- [Project website](https://www.yinmei.vip)
- [README](https://github.com/worm128/AI-YinMei/blob/master/README.md)
- [Releases](https://github.com/worm128/AI-YinMei/releases)
- [worm128/AI-YinMei on GitHub](https://github.com/worm128/AI-YinMei)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/worm128-ai-yinmei