xszyou/Fay: a Python agent framework for digital humans

The gap Fay is trying to fill

A digital human demo is easy. A digital human that answers a real question, calls a real tool and keeps a memory across turns is a stack of problems: speech recognition, a language model, speech synthesis, a memory store, a tool dispatcher and a transport to whatever renders the face. Fay's README describes itself as a framework that adapts upward to digital human models and downward to large language models, with TTS and ASR replaceable in between. That is the actual pitch. It is not a model. It is the glue layer, written in Python, aimed at people building for single-board computers, apps, websites and large screens rather than at researchers training avatars.

The README lists the endpoints it exposes: text interaction, voice interaction, digital human driving, management control, automatic broadcast and intent. That list is the clearest statement of scope. If your product needs a virtual teacher, a virtual anchor or a news reader that talks on a schedule, the automatic broadcast mode and the scheduler directory in the repository are the parts aimed at you. If you only need a chat completion wrapper, Fay is more machinery than the job requires.

How the pieces connect inside the repository

The top-level layout is the architecture. ai_module, asr, tts and llm are separate directories, which matches the README's claim that each model slot is swappable. core and gui hold the runtime and the interface, faymcp and mcp_servers handle tool exposure over MCP, memory and genagents hold the memory layer, and scheduler plus simulation_engine drive the calendar-style proactive conversation the README calls a daily-schedule digital human.

Configuration is split between config.json, the config directory and system.conf.bak, which the README instructs you to rename to system.conf. The control panel is a Flask application, which is consistent with flask and flask_cors appearing in requirements.txt and with the README pointing a browser at http://127.0.0.1:5000. Tool calling is delegated to MCP, and the README says both sse and studio transports are supported. Two acknowledgements in the README name the references: openclaw for the memory mechanism and skills design, and OpenAI Codex for tool-calling behaviour. That is useful context, because it tells you the memory design is borrowed rather than novel, and the project says so.

Installing Fay and getting the control panel up

The README targets Python 3.12 on Windows, macOS and Ubuntu. On Ubuntu it says gcc and portaudio must be installed first, because pyaudio is in the dependency list. The two apt commands below are copied from the README.

sudo apt update
sudo apt install build-essential
sudo apt install portaudio19-dev

After that, install the Python dependencies from the repository root.

pip install -r requirements.txt

Before the first start, copy the configuration template. The README says to rename system.conf.bak to system.conf in the root directory and fill in its contents.

cp system.conf.bak system.conf

Then launch. The README's quick-start command passes a config_center id, which it describes as a public resource that is very slow, with a note recommending you replace it with your own key.

python main.py start -config_center d19f7b0a-2b8a-4503-8c0d-1a587b90eb69

Once it is running, open the management page in a browser.

http://127.0.0.1:5000

The README also links a prebuilt image on compshare.cn for people who would rather not assemble the environment. There is no PyPI package and no release artefacts in the repository, so source checkout is the documented path.

Where Fay will fight you

The dependency file is the first real constraint. requirements.txt pins tenacity to 8.2.3 and uses compatible-release specifiers for pyaudio, websockets, ws4py, flask, openpyxl and flask_cors, but most entries have no version constraint at all: langchain, langgraph, chromadb, sentence_transformers, opencv-python and the rest float. sentence_transformers and chromadb in particular pull large transitive trees. A fresh install months after a working one can resolve differently. The repository keeps a requirements-legacy.txt alongside, which suggests the maintainers have already hit this and kept an older set rather than pinning the current one.

The README does not document rollback, migration between versions or a versioned configuration schema. It does state that three former versions were merged into one, which means anyone following an older tutorial from the Feishu wiki may be reading instructions for a layout that no longer exists. The README also does not describe a test suite, even though a test directory is present, so there is no documented signal about what is covered.

Finally, the quick-start command points at a shared config_center id. It is a demo credential, and the README itself warns it is slow. Treat it as a smoke test only. Shipping a product on someone else's key is not a configuration the documentation supports.

Fay against wiring the stack yourself

The obvious alternative is not another framework. It is assembling the same pipeline from the individual libraries Fay already depends on: FunASR or another recogniser, an OpenAI-compatible client, edge_tts or a cloud TTS, and your own HTTP layer. That path gives you full control over versions and licensing, and it removes the Flask control panel and the scheduler you may not want. The cost is that you rebuild the memory layer, the MCP tool dispatch and the six interfaces the README lists, and you own the integration drift when any one model changes.

Fay's difference in approach is that it treats the model slots as replaceable parts behind fixed interfaces, and it ships the operational surface (control panel, broadcast mode, intent interface, qa.csv-driven voice commands) rather than just the inference path. If your product is a kiosk or a virtual teacher with a fixed set of intents, that surface is the value. If your product is a single chat endpoint, it is overhead. The README also credits FunASR for speech recognition, so choosing Fay does not exclude using the same ASR you would have picked anyway.

Licence, updates and what maintenance costs you

Fay is GPL-3.0. The README's first feature bullet says fully open source and commercial免责, which in context reads as a claim that commercial use is permitted. GPL-3.0 does permit commercial use, but it also carries copyleft obligations for distributed derivative works, and it is incompatible with shipping the code inside a closed product without meeting those terms. The README does not discuss this trade-off. If your deployment model is a distributed binary or an on-premise appliance, get the copyleft question answered before you build on it, and note that the README offers no alternative licence or commercial exception.

The last push to the default branch was on 2026-08-07. The repository is not archived. There are no retrieved releases, so upgrades happen by pulling main, and the absence of a changelog file in the repository root means the README's link to an external Feishu wiki is the only documented change history. Combined with an unpinned requirements.txt, the practical upgrade procedure is to pull, reinstall dependencies in a fresh virtual environment, and compare system.conf against system.conf.bak by hand. Budget for that, not for a package manager doing it for you.

Editorial conclusion

Adopt Fay if you are wiring a 2.5D or 3D avatar, a mini program or a kiosk into an OpenAI-compatible model and you want the ASR, TTS, memory and tool-calling layers handled in one Python process. Do not adopt it if you need a permissively licensed core, because the repository is GPL-3.0, or if you want a framework that installs cleanly from a pinned lockfile, because requirements.txt carries no version pins on most lines. Before committing, verify three things: that the system.conf.bak template matches the models you actually have keys for, that the public config_center id in the README is not what you ship, and that the Feishu documentation covers the digital human model you intend to drive.

Frequently asked questions

Official sources

1. Issues
2. License: GPL-3.0
3. README
4. xszyou/Fay on GitHub