CyberVerse: A Self-Hosted Voice Agent Stack With an Optional Talking Head
Self hosted, real-time digital human agent platform. Build voice-first AI agents with WebRTC, persona memory, tools, RAG, and optional digital-human video.
At a glance
- What is it?
- CyberVerse is a Python framework that pairs WebRTC voice interaction with persona memory, RAG and a pluggable digital-human video layer. It is aimed at builders who want the whole loop on their own hardware, and the GPU table in the README is the first thing to read before you clone it.
- Who is it for?
- Adopt CyberVerse if you need a full-duplex voice loop plus an optional avatar and you are willing to own the GPU and the provider keys. Do not adopt it if you only need text agents, or if GPL-3.0 does not fit your distribution plan.
- Can I use it commercially?
- Yes, with conditions. GPL-3.0 is a copyleft licence: if you distribute software that includes it, you must release that software's source code under the same licence. Running it internally without distributing it does not trigger that obligation.
- Is it still maintained?
- Yes. The repository last received commits 5 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap CyberVerse is trying to fill
Most agent frameworks assume text. You send a prompt, you wait, you get a block of tokens back. Voice changes the constraints: the user can interrupt mid-sentence, the reply has to start before it is finished, and the agent has to hold a persona across sessions rather than resetting at every turn. CyberVerse is built around that constraint set. Its README describes it as a real-time digital-human agent framework using WebRTC, persona memory, tools, RAG and optional digital-human video, with voice as the centre of the interaction rather than an add-on.
The intended audience is narrower than "anyone building an agent". It is people who want the conversational loop, the memory store and the face all running under their own control, on their own machine or their own rented GPU. The README's framing leans toward personal companions, character-driven assistants and J.A.R.V.I.S.-style setups, and the demo gallery lists named characters (Alice, Lina, Xiaolongnü) that the README explicitly says are examples and are not bundled or licensed for commercial use. That is a useful signal about who the project is written for: individual builders and small teams experimenting with embodied voice agents, not enterprises shopping for a managed contact-centre product.
PersonaAgent in the foreground, SubAgents in the background
The architectural decision that matters most is the split between PersonaAgent and SubAgent tasks. PersonaAgent stays in the foreground and is responsible for keeping conversation fluid, reacting quickly to interruptions and handling context switches. Anything long-running, which the README lists as search, research, material organisation, summarisation and HTML report generation, is handed to background SubAgents that run asynchronously.
That is a direct response to a real failure mode in voice systems: a tool call that takes fifteen seconds to complete will either stall the turn or force the agent to talk over itself. By moving that work off the foreground path, the user can keep speaking, ask follow-ups or change direction, and PersonaAgent returns the SubAgent result when it is ready. Whether the handoff is smooth in practice depends on how the result is surfaced, and the README does not describe that mechanism in detail. Treat the async delegation as the design intent rather than a guarantee about latency.
Memory and retrieval sit alongside this. Each character's conversation history is persisted to local disk and reloaded when you re-enter a conversation, which is what gives continuity across sessions. Separately, you can import knowledge bases, documents and biographical material for a character; the system indexes them for retrieval-augmented generation so answers align with the character's background. Persistence to local disk is a simple choice with a simple consequence: there is no built-in replication story described in the material, so backups are your problem.
The digital-human layer and what it costs in hardware
The video component is optional and, by the project's own numbers, expensive. CyberVerse integrates local FlashHead and LiveAct models and also supports cloud digital-human offerings including Baidu Xiling and Xunfei Digital Human. The README's table is unusually candid about the hardware needed for the local options.
FlashHead 1.3B at Pro quality is listed as real-time on two RTX 5090s at 512x512 and 25+ FPS, or on a single RTX 5090 at 464x464 and 20 FPS. LiveAct 18B is listed as real-time on two RTX PRO 6000s at 320x480 and 20 FPS, or one at 256x417 and 20 FPS. Vidu S1, Baidu Xiling and Xunfei are cloud APIs requiring no local GPU, with resolution and frame rate either fixed by the provider (540P, 25 FPS for Vidu S1) or determined by provider and figure configuration. The README claims full-duplex interaction where the user can interrupt or speak over the digital human at any time.
Read that table before you plan anything. A single 5090 buys you a 464x464 talking head. Two cards buy you 512x512. If your expectation was a 1080p avatar on a workstation GPU, the README does not support it. The cloud routes remove the GPU requirement but move the face, the latency and the per-minute cost to a third party, which undercuts the self-hosted premise for that one component.
Configuration surface: one YAML file and per-modality model directories
CyberVerse is assembled from replaceable modules: brain, voice, hearing, tools, memory and face. Runtime behaviour lives in config/cyberverse.yaml. Provider definitions for omni, LLM, TTS, ASR and embedding models are loaded from the built-in infra/config/*_models/ directories, with optional local overrides under config/*_models/. API keys and service endpoints for different vendors are set in the web UI at /settings, which is how you switch providers and model combinations per scenario.
That layout gives you two layers of override: shipped defaults under infra/, and your own additions under config/. The practical consequence is that upgrading the project will not clobber your provider definitions as long as you keep them in the config/ side, and it means the interesting configuration work happens in a known set of directories rather than scattered through code. The trade-off is that provider wiring is spread across three places (the YAML, the model directories, and the settings UI), and the README does not spell out the precedence rules between them. If you are debugging why a particular model is being selected, that ambiguity is where you will start.
The LiteLLM plugin is the escape hatch for LLM coverage. The README states it adds access to 100+ LLM providers including AWS Bedrock, Azure, Vertex AI, Mistral and Cohere through a single unified interface, which means you are not limited to whatever providers ship in the built-in model directories.
Getting it running
The README's Quick Start leads with cloud images rather than a pip install. It lists a Compshare CyberVerse image and an AutoDL CyberVerse image for people who want to try it without setting up environment dependencies manually. The supplied README text is truncated inside the Quick Start section, so the remaining install steps, the exact run command and any dependency list are not visible in the material I have. I cannot state them, and you should read the repository's Quick Start in full before assuming a particular entry point.
What is confirmed from the material: the primary language is Python, the default branch is main, and the only release listed is v0.1.0 from 2026-05-16. The repository is not archived, and the last push recorded is 2026-08-05. Configuration is centred on config/cyberverse.yaml, with provider definitions under infra/config/*_models/ and optional overrides under config/*_models/, and keys entered through the web UI at /settings. If you are evaluating it, the honest sequence is: read the full Quick Start, decide whether you are using a cloud image or a manual environment, then confirm the GPU situation against the digital-human table before you spend time on the rest.
Where CyberVerse is the wrong choice
Three cases stand out. First, if you only need text agents, the WebRTC transport, the voice pipeline and the digital-human layer are overhead with no payoff. Second, if you cannot supply the GPUs listed for FlashHead or LiveAct and you do not want a cloud digital-human provider, the video feature is effectively unavailable to you, and you are left with a voice agent framework whose distinguishing feature you cannot use. Third, the licence. CyberVerse is GPL-3.0. If you intend to distribute a modified version, or to combine it with code under incompatible terms, that is a constraint you need to evaluate with your own counsel. The README's own note that the demo characters are not bundled and not provided for commercial use is a separate restriction on assets, not on the code.
There is also a maturity question the material cannot answer. One tagged release, v0.1.0, is the entire release history shown. The README describes an ambitious feature set spanning real-time video, multi-agent delegation, persistent memory, RAG and a plugin system. That combination at a 0.1.0 version number means you should expect the configuration surface to move. I have not installed or run this project, so I cannot tell you how stable any of it is in practice.
How it differs from LiveKit Agents
LiveKit Agents is the closest well-known comparison point, and the difference is in what each treats as the core abstraction. LiveKit Agents is built on the LiveKit real-time media stack and treats the voice pipeline (transport, VAD, STT, LLM, TTS) as the thing you compose, with room-based sessions as the organising unit. CyberVerse instead organises around the character: persona memory persisted per character, knowledge imported per character, and a foreground PersonaAgent whose job is to hold that character together while background SubAgents do the slow work. The digital-human face is a first-class module in CyberVerse's stack rather than something you bolt on.
Practically, that means choosing CyberVerse is choosing an opinionated agent shape (character plus memory plus optional avatar) and inheriting its provider configuration model. Choosing LiveKit Agents is choosing a media infrastructure and assembling the agent semantics yourself. Neither is strictly better; they fail in different directions. CyberVerse gives you more of the agent behaviour out of the box and less freedom in how sessions and characters are modelled. If your product is a companion or a character-driven assistant, the CyberVerse shape is closer to what you want. If your product is a support line or a multi-participant room, the room-based model is a better fit and CyberVerse's character-centric design will feel like a constraint.
Maintenance, upgrades and licence obligations
The maintenance picture from the material is thin. One release, v0.1.0, published 2026-05-16, with repository activity recorded through 2026-08-05. That is consistent with an actively developed early-stage project, but it is not enough to judge release cadence. The configuration split between infra/config/*_models/ and config/*_models/ is the part of the design that most directly affects upgrade cost: keeping your provider definitions on the config/ side is what lets you pull new versions without reapplying them, and the README does not describe a migration path for changes to config/cyberverse.yaml itself.
On licensing, GPL-3.0 is a copyleft licence. If you modify CyberVerse and distribute it, or distribute a work derived from it, the licence's terms apply to that distribution. Running it as a self-hosted service for yourself is a different situation from shipping it inside a product, and the two should not be conflated. I am not giving legal advice; the point is that the licence choice is a real adoption factor here, not a footnote, and it is worth a conversation with someone qualified before you build a commercial offering on top of it. The sponsor arrangement with Compshare, which offers GPU rental and model API access and is linked from the README with a referral code, is worth noting as a commercial relationship attached to the project.
Editorial conclusion
Adopt CyberVerse if you need a full-duplex voice loop plus an optional avatar and you are willing to own the GPU and the provider keys. Do not adopt it if you only need text agents, or if GPL-3.0 does not fit your distribution plan. Before committing, verify which digital-human backend you can actually run: check the GPU column in the README table against your hardware, and confirm that config/cyberverse.yaml and the infra/config/*_models/ directories expose the providers you intend to use.
Community notes