py-xiaozhi: a Python voice assistant runtime for desktops and edge boards
Open-source AI assistant ecosystem with MCP integrations, multimodal workflows, IoT support, and cross-platform voice interaction.
At a glance
- What is it?
- The project ports the xiaozhi-esp32 firmware idea into Python, so the same voice, vision and MCP tooling can run on a laptop or a Raspberry Pi. The pitch is portability; the price is a setup path that assumes you read the docs before you run anything.
- Who is it for?
- Adopt py-xiaozhi if you already work in Python and want the xiaozhi-esp32 interaction model on a desktop or an ARM board without writing your own audio pipeline. Skip it if you need a stable, versioned library with a frozen API, or if your target is a microcontroller, where the original firmware remains the right layer.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 5 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What py-xiaozhi actually replaces
The xiaozhi-esp32 project established a pattern: a small device listens, streams audio to a model, and answers. py-xiaozhi takes that pattern off the microcontroller. According to the README, it is a Python async framework for real-time voice streaming, vision-language tasks and IoT device control, deployable on Windows, macOS, Linux desktops and ARM boards including Raspberry Pi, Horizon Robotics RDK and Jetson Nano. The stated goal is bridging large language models and physical hardware. The audience is therefore narrow and specific: people building a voice-driven device or kiosk who are comfortable in Python and do not want to write Opus framing, wake word detection and WebSocket reconnection themselves. If you only need a chat interface, this is more machinery than the problem requires.
The pipeline: Opus frames in, MCP tool calls out
The architecture described in the README is event-driven, built on an asyncio event loop, with application, protocol and UI layers separated and component lifecycles managed through a bootstrap container. Audio moves as Opus, and the README specifically mentions auto frame detection via RFC 6716 TOC parsing, which is the part that usually breaks naive implementations: you cannot assume fixed frame sizes when decoding a stream. Wake word detection runs on-device through Sherpa-ONNX rather than round-tripping audio to a server, and the README says custom wake words are supported. Tool use goes through MCP, described as a modular JSON-RPC 2.0 tool server with entries for music playback, camera, screenshot, app management, weather and volume control. Transport is WebSocket or MQTT with WSS/TLS and auto-reconnection. The README also claims sub-20ms latency for the voice path, a number I cannot verify from the material and which in any case depends on where the model runs.
Installation and the update trap
The README does not print a full install command sequence, so I will not invent one. What it does state is concrete and worth reading carefully. Python 3.10 through 3.12 is the supported range. You need a microphone and speaker, at least 4GB of RAM with 8GB recommended, a CPU with AVX support, roughly 2GB of free disk for model files and cache, and audio hardware that can sample at 16kHz. Camera work needs OpenCV and a camera device; wake word needs the Sherpa-ONNX models downloaded separately. The README points to the project documentation site for startup tutorials and file descriptions, and there is a Bilibili video walkthrough titled Zero to Xiaozhi Client. The one operational instruction given directly in the README is the important one: the main branch carries the latest code, and you must manually reinstall pip dependencies after every update to pick up new ones. That is a deliberate choice by the maintainers, and it means a git pull alone will leave you with a broken environment.
Where the design gets in the way
Three constraints stand out. First, the Python version window is closed on both ends: 3.10 to 3.12. On a distribution shipping 3.13, you are managing a separate interpreter before you manage the assistant. Second, the manual dependency reinstall after each update is a maintenance burden that scales with how often you pull, and it is the kind of step that gets skipped until something fails at runtime. Third, the feature set is unevenly portable. The README lists PySide6 and QML for the GUI, a CLI mode, and GPIO for embedded use, but a headless Raspberry Pi deployment and a Windows desktop deployment share very little of the surface area. GPIO control, camera capture and the GUI are not interchangeable, and the README does not present a single configuration that covers all three. The README also flags that the main branch is the latest code, which in practice means the stable surface is whatever the most recent release tag froze, currently v2.1.1 from July 2026.
How it differs from the ESP32 firmware and from Home Assistant
The obvious comparison is xiaozhi-esp32, the firmware project this one evolved from and which the README names as its origin. The difference in approach is the runtime, not the interaction model. The firmware runs on the device with tight memory limits and no Python interpreter; py-xiaozhi runs on a host with an operating system, which buys you OpenCV, PySide6, arbitrary MCP tool servers and the ability to swap models without reflashing. It costs you the boot time, the RAM floor and the reliability of a dedicated microcontroller. The other comparison worth making is Home Assistant, which also does local voice and device control. Home Assistant is an integration platform organized around entities and automations, with voice as one frontend among several. py-xiaozhi is organized around a single conversational agent, with device control exposed as MCP tools. If your goal is a house full of coordinated automations, the entity model fits better. If your goal is one device that talks, listens and drives a few GPIO pins, py-xiaozhi is closer to the shape of the problem. The README notes that D-Robotics adopted it as an upstream dependency for xiaozhi-in-rdk, which suggests the RDK board path is exercised by someone other than the original author.
Licence and the cost of keeping up
The project is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is a permissive baseline, and it means you can ship a product built on it without opening your own code. It does not settle anything about the models or services you connect it to: the README lists sponsors who operate API relay services for Claude, Gemini and GPT, and those come with their own terms. The MIT grant covers the code in this repository only. On maintenance, the release cadence visible in the material is three releases between mid-July and late July 2026, with the last push to the repository in August 2026. Frequent releases are good for fixes and bad for anyone pinning a version, especially combined with the README's instruction to reinstall dependencies after every update. Budget for a pinned release tag plus a periodic, deliberate upgrade rather than tracking main.
Who this fits
The fit is a Python developer with an ARM board, a microphone and a specific interaction to build. The README supports that reading with its hardware list and its stated origin as a port of an embedded firmware project. The misfit is anyone who wants a library they can import and forget about, because the update process is manual and the supported Python range is narrow. It is also the wrong layer if your target is a bare microcontroller, where the ESP32 firmware it descends from remains the appropriate choice. The honest position is that py-xiaozhi is a runtime with a real hardware story and a documentation site that carries the setup detail the README omits, and the README itself says to read that documentation before starting. Treat the README as a pointer, not a manual.
Editorial conclusion
Adopt py-xiaozhi if you already work in Python and want the xiaozhi-esp32 interaction model on a desktop or an ARM board without writing your own audio pipeline. Skip it if you need a stable, versioned library with a frozen API, or if your target is a microcontroller, where the original firmware remains the right layer. Before committing, verify three things against the current release: that your Python is inside the 3.10 to 3.12 window, that you have re-run the pip install step after pulling the latest main branch as the README instructs, and that the Sherpa-ONNX wake word model you intend to use actually exists for your language.
Community notes