open-xiaoai-bridge: replacing the Xiaoai speaker's cloud backend with your own AI stack

What the bridge actually replaces on a Xiaoai speaker

A stock Xiaoai speaker answers to Xiaomi's cloud. The audio path, the wake word and the text-to-speech all terminate in services you do not control, and there is no supported way to point the device at your own model. This project attacks that from the server side: the README states plainly that it contains only the server, and that a client must first be installed on the speaker. That client is a Rust program from the coderzc/open-xiaoai repository, installed through a patching procedure that itself requires flashing the firmware and enabling SSH, following a guide in idootop/open-xiaoai.

So the audience is narrow and specific. You own a speaker whose model appears in the flashing documentation. You are comfortable with firmware work and SSH. You want the microphone and the speaker to become endpoints for something you run yourself, whether that is a local Ollama instance, a hosted OpenAI-compatible API, or an agent framework. If any of those three conditions fails, the rest of this article does not apply to you, because the bridge has no fallback path that works on an unmodified device.

The project credits idootop/open-xiaoai as its inspiration and says it evolved from that repository's examples/xiaozhi/ example before becoming independently maintained. That lineage matters when you read the install instructions: the client-side documentation lives in a different repository from the server you are deploying here, and the two must agree on the protocol and, if you set one, on the token.

The audio pipeline: VAD, KWS, ASR and where each model file goes

The architecture diagram in the README shows the device capturing PCM audio and shipping it over a WebSocket to open_xiaoai_server on port 4399. From there a GlobalStream carries the audio into a processing chain: VAD for speech onset and offset detection, KWS for wake-word detection, and SherpaASR for offline recognition, with an AudioCodec handling encoding and playback. A MainApp loop tracks device_state, a WakeupSessionManager holds the wake-session state machine, and separate controllers handle continuous conversation for the Xiaoai-native path and for OpenClaw.

That chain is where the model files come in, and the README is explicit about a fork in the requirements. If you enable xiaozhi AI, or if continuous dialogue for OpenClaw, OpenAI-compatible services or QwenPaw runs with local_asr, you download the full VAD + KWS + ASR bundle from the vad-kws-asr-models release tag. If those same continuous-dialogue paths run with xiaoai_asr instead, you only need VAD + KWS, because recognition is delegated back to the speaker's own ASR. That is a real reduction in disk and memory footprint, and it is the first configuration decision worth making.

The VAD-first ordering is a design choice with a cost. The feature table claims it reduces invalid recognition and saves power, and the environment variable AUDIO_INPUT_ENABLE exists to turn audio input off entirely, with the README noting that xiaozhi, KWS and local_asr all stop working when it is disabled. Nothing in the supplied material quantifies the saving, so treat the power claim as unverified. What is verifiable is the coupling: one switch disables three subsystems at once, which is convenient for a deployment that only uses the HTTP API and awkward if you wanted to keep wake-word detection while dropping local recognition.

Wake words as a routing table, not just a trigger

The feature that distinguishes this from a simple speech-to-text relay is that wake words are routers. The README advertises custom wake words in Chinese and English, with different wake words sending the utterance to different AI services or to different OpenClaw agents. The multi-agent row describes one speaker, several wake words, and each wake word bound to a distinct OpenClaw Agent Session, with the switch described as zero-overhead and dynamic.

Mechanically this is the WakeupSessionManager and the config.py before_wakeup and after_wakeup hooks visible in the architecture diagram. The hooks are the extension point: they run around the wake event, which is presumably where you inspect which keyword fired and dispatch accordingly. Continuous dialogue then keeps the session open, and the README says saying the wake phrase again interrupts.

The honest limitation here is that the README describes the routing capability in a feature table and a diagram but the excerpt does not show the config.py contents that implement it. If you need to know the exact key names for mapping a keyword to an agent session, you will be reading config.py after downloading it, not this review. That is a documentation gap, not a defect, but it is the kind of gap that turns a ten-minute setup into an evening.

Deployment: two paths, and the environment variables that gate everything

The README gives two installation routes. The recommended one is Docker Compose. You unpack the model archive into ./models, then fetch two files and start:

curl -O https://raw.githubusercontent.com/coderzc/open-xiaoai-bridge/main/config.py curl -O https://raw.githubusercontent.com/coderzc/open-xiaoai-bridge/main/docker-compose.yml docker compose up -d

The compose file already mounts ./models:/app/core/models, so the model directory layout is fixed by the compose file rather than by an environment variable. For users in mainland China the README suggests swapping the image to ghcr.nju.edu.cn/coderzc/open-xiaoai-bridge:latest. If the container needs to reach an OpenClaw or QwenPaw process running on the host, the README points to its Docker FAQ section, which the excerpt does not include; that is the one piece of setup I would want before starting.

The second route is a local build. You need uv and Rust, plus pkg-config and patchelf on Linux, and the models go to core/models/ instead. The start script takes the feature flags as environment variables:

API_SERVER_ENABLE=1 XIAOZHI_ENABLE=1 OPENCLAW_ENABLE=1 OPENAI_ENABLE=1 QWENPAW_ENABLE=1 ./scripts/start.sh

Every integration is off by default. The table lists XIAOZHI_ENABLE, OPENCLAW_ENABLE, OPENAI_ENABLE, QWENPAW_ENABLE and API_SERVER_ENABLE as disabled unless set, with API_SERVER_HOST defaulting to 127.0.0.1 and API_SERVER_PORT to 9092, LOGLEVEL to INFO, and CONFIG_PATH to ./config.py. Note the loopback default on the API host: exposing the HTTP API beyond the machine is a deliberate change you have to make, which is the right default for a service that can make a speaker in your home play arbitrary audio.

The token, and what happens if you skip it

OPEN_XIAOAI_TOKEN is described as client authentication: when set, only clients holding the same token can connect. The default is no authentication at all. The start script example shows the pairing:

OPEN_XIAOAI_TOKEN=your-secret-token API_SERVER_ENABLE=1 ./scripts/start.sh

The token has to match the value on the speaker side, which means changing it is a two-ended operation, not a server restart. The README does not describe what an unauthenticated client can do once connected, but the WebSocket on port 4399 is the channel that carries audio in both directions and drives playback, so an open port is an open microphone-and-speaker interface for anyone who can reach it. Combined with the loopback default on the HTTP API, the picture is a project that defaults to safe on the HTTP side and open on the WebSocket side. Set the token if the bridge is reachable from anything other than the speaker itself.

Where this is the wrong tool

The clearest failure case is a speaker that cannot be flashed. The README's first prerequisite is updating the firmware and enabling SSH, following an external guide, and the second is installing the Rust client through a patching procedure in a different repository. There is no supported path around either step. If your device model is not covered, or you are not willing to modify it, the bridge is inert.

The second case is a single-assistant deployment. If you want one wake word, one model, no continuous dialogue and no per-keyword routing, then what you are deploying is a WebSocket audio relay plus a VAD and KWS front end, and the parts of this project that justify its configuration surface (the wake-session state machine, the multi-agent routing, the separate OpenClaw conversation controller, the QwenPaw task integration) sit unused. A simpler relay would be less to maintain.

The third case is an environment where the model files are a problem. The local ASR path requires downloading and unpacking a model bundle into a fixed directory, and the Docker path expects it at ./models mounted to /app/core/models. On a small always-on box, that plus a Rust build toolchain for the local-install route is real overhead. The xiaoai_asr option exists precisely to avoid the ASR model, but it still needs VAD and KWS.

How it differs from running xiaozhi-esp32-server directly

The most natural alternative is to skip the bridge and point the speaker at xiaozhi-esp32-server, the project this one integrates with as one of its connectors. The difference in approach is where the routing logic lives. xiaozhi-esp32-server speaks its own realtime audio protocol to a device and is the endpoint. open-xiaoai-bridge sits in front of it, terminating the Xiaoai-specific WebSocket on port 4399, running the local VAD and KWS chain, and then acting as a client to xiaozhi-esp32-server through its XiaoZhi connector.

That extra layer buys three things you cannot get from the server alone: wake-word-based routing to services other than xiaozhi, continuous dialogue against OpenClaw or an OpenAI-compatible endpoint with the option of xiaoai_asr instead of a local model, and an HTTP API on port 9092 for pushing text or audio to the speaker from outside a conversation. It costs you a second process to run, a config.py to keep in sync, and the token pairing. If your only goal is to talk to a xiaozhi server, the bridge is a middleman. If you want one speaker to answer to several wake words backed by different agents, the routing has to live somewhere, and this is where this project puts it.

Maintenance cost, licensing and what to check before you commit

The project is MIT licensed, which permits commercial and private use with the usual requirement to keep the copyright notice and licence text; that is a statement about the licence text, not legal advice for your situation. The release cadence visible here is three releases between late March and mid July 2026, with the most recent push to main in late July 2026, so the project is being touched regularly. Star and fork counts appear in the README badges but say nothing about whether the code works on your speaker model.

The maintenance burden that matters is not the Python code, it is the coupling to three external things: the patched client on the device, the model bundle from the vad-kws-asr-models release tag, and whichever upstream service you enable. A model bundle and a client patch are versioned artifacts you have to re-download when they change, and the README's model instructions differ depending on whether you run local_asr or xiaoai_asr, so a configuration change can invalidate your model directory. Budget for that, not for reading the source.

What to verify first, concretely: open the flashing guide and confirm your speaker model is listed. Then decide local_asr versus xiaoai_asr, because it determines whether you download the ASR model at all. Then check the Docker FAQ section for the host-networking recipe if OpenClaw or QwenPaw runs outside the container, since the README defers that answer rather than inlining it. Finally, set OPEN_XIAOAI_TOKEN to the same value on both ends before the bridge is reachable from anywhere but localhost.

Editorial conclusion

Adopt it if you have already flashed the speaker and installed the Rust client from coderzc/open-xiaoai, because the bridge is the server half of that pair and does nothing on its own. Skip it if you are unwilling to open SSH on the device, or if you only need a single fixed assistant, since the multi-agent routing, the custom wake words and the continuous-dialogue state machine are the parts you would be paying for in setup time. Before committing, verify three things in this order: whether your speaker model is covered by the flashing guide, whether you need the local ASR model at all or can run with xiaoai_asr and only the VAD plus KWS files, and whether the container can reach the OpenClaw or QwenPaw process, which the project addresses in its Docker troubleshooting section.

Official sources

1. coderzc/open-xiaoai-bridge on GitHub
2. Issues
3. License: MIT
4. README
5. Releases