Talk to 峰哥: cloning a voice and a personality for sub-second voice chat

What Talk to 峰哥 actually assembles

Most voice projects pick one half of the problem. A real-time conversational agent such as GPT-4o Voice does not let you supply your own cloned timbre. A cloning toolkit such as Bark or XTTS produces speech from text and stops there, with no live turn-taking. Talk to 峰哥 exists to join the two: voice cloning from a short sample, a written persona that shapes phrasing and habits, and a live audio loop. The README frames the target as talking on the phone rather than listening to a reader.

The first complete example is 峰哥亡命天涯, a Bilibili creator. That matters for how you read the repository. The persona files, the reference audio at assets/voice_samples/fengge_ref.wav and the default configuration are all built around one person. The README states the architecture supports swapping in someone else, and the section on doing so is the part a new user should read first, because it tells you what the project really is: a template with one filled-in instance, not a general product.

The audio path from microphone to cloned reply

The flow is a straight chain. Speech goes from the browser over WebRTC to a local LiveKit server, which forwards it to a Python agent worker. The worker runs STT, then the LLM with persona text and optional memory recall injected, then TTS, and the synthesized audio goes back out through LiveKit to the browser. The README diagrams persona and memory feeding the LLM stage, and OpenViking sitting off to the side as an optional memory service.

Three components are swappable through .env.local. STT defaults to Cartesia ink-whisper, with Deepgram nova-2 and Gemini as alternatives. The LLM defaults to MiniMax, with DeepSeek and Gemini listed. TTS has the widest choice: VoxCPM for the best cloning quality on a GPU, MOSS-TTS as a CPU fallback, Cartesia Sonic in the cloud, or MiniMax TTS. A separate web server on port 8766 serves the single-page front end, and livekit-server runs locally in dev mode. That is four processes to keep alive during a session, which is the practical shape of the project.

Install and first conversation

The README recommends handing the repository to an AI coding assistant and asking it to configure and start the project, since the assistant can read .env.example and walk you through the keys. The manual path is documented and short enough to follow. Clone the repository and install the Python dependencies with uv, then install the LiveKit server binary; the README gives the macOS Homebrew command.

git clone https://github.com/YeJe-cpu/talk-to-fengge.git
cd talk-to-fengge
uv sync
brew install livekit

Copy the example environment file and edit it. The README says you need at least a Cartesia key for STT, a MiniMax key for the LLM, and a decision about TTS.

cp .env.example .env.local

The default TTS setting points at a VoxCPM service on localhost port 8000, which means you need a GPU host running that service before the agent can speak. The repository ships runpod_setup.sh for that machine; the README states the first run downloads the model and takes roughly ten minutes. If you have no GPU, set TTS_PROVIDER=moss in .env.local for the CPU fallback.

bash runpod_setup.sh

Start the three processes in separate terminals, then open the page. The agent worker is started as a module with the LLM provider named explicitly, and the web server serves the front end.

livekit-server --dev --node-ip=127.0.0.1
LLM_PROVIDER=minimax python -m worker.main start
python -m worker.web_server

Open http://127.0.0.1:8766 and speak. On macOS the README also lists a double-clickable launcher, Talk-to-Me-V3.6.command, which runs the same components for you.

Swapping the voice and the persona is manual work

Replacing 峰哥 with someone else is not a configuration change. The README asks for two inputs. The first is 15 to 45 seconds of clean speech with no music or noise, which VoxCPM uses to clone the timbre. The second is a persona description, and the documented method is to open the repository in an AI coding assistant and ask it to generate a new persona by reading docs/persona-*.md and worker/persona.py as templates. The assistant interviews you to extract traits and writes the persona code.

That is an honest description of where the project stands. Persona creation is a prompt-driven authoring task, not a pipeline with a schema you fill in. The .env.example confirms the narrowness: PERSONA_NAME is annotated as currently supporting only fengge. Richer source material such as chat logs, transcripts and stream clips improves the result, according to the README, but nothing in the repository automates collecting or distilling it. The README lists automated persona distillation as a future item rather than a feature.

Where the sub-second claim stops being true

The headline figure is engineering pipeline latency under one second, and the README is unusually direct about the gap between that number and what a caller experiences. It states the felt conversation delay is about 2 to 3 seconds, driven by network conditions and API response times. Treat the sub-second figure as a property of the local chain, not a promise about your session. If your STT, LLM or cloud TTS endpoints sit far from your machine, the felt delay grows.

The memory system has a sharper limitation. OpenViking is described as capturing memories mainly from events and subjects on the user side. In this scenario the AI talks more than the caller, so the README concludes that little gets deposited. A memory feature that works poorly precisely in the configuration the project ships is a real constraint, and it is the reason the memory layer is optional rather than central.

Deployment is the third rough edge. Each of STT, LLM and TTS needs its own key or service, and the README states there is no one-command deployment. The front end is a single page with a starfield effect and no avatar. None of this is hidden, but it means the project is closer to a working reference than to something you hand to a non-technical colleague.

LiveKit Agents versus a hosted voice API

The nearest alternative in kind is a hosted conversational voice API, where you send audio and receive audio from one vendor and the vendor owns the model stack. The difference is control over the timbre. A hosted voice service gives you a fixed catalogue of voices or, at best, a cloning feature behind a paid tier, and you cannot run the speech model on your own hardware. Talk to 峰哥 inverts that: VoxCPM runs on a GPU you supply, so the cloned voice never leaves your machine, but you own the GPU, the model download and the service uptime.

The repository itself offers a middle path. Setting TTS_PROVIDER=cartesia swaps in Cartesia Sonic, a cloud model that requires a Pro subscription at $5 per month to clone a voice. That removes the GPU requirement and keeps the rest of the architecture, which makes the trade-off concrete: run the clone locally and pay in hardware, or rent it and pay in subscription and data leaving your network. The STT and LLM choices follow the same pattern, with MiniMax noted as usable without a VPN from mainland China and Gemini as the alternative that may need one.

Licence, maintenance and what an upgrade costs

The project is Apache-2.0, and the README states it is built on LiveKit Agents, also Apache-2.0, with a NOTICE file covering the details. Apache-2.0 permits commercial use and modification with the usual attribution and notice obligations; the NOTICE file is the part to read before redistributing, and the acknowledgements list third-party components including VoxCPM, OpenViking and two persona-skill repositories whose own terms are separate from this project's licence. That last point is worth checking if you plan to ship a persona derived from someone else's material.

The last push to the default branch was on 2026-07-11, and the repository is not archived. No releases were retrieved, so there is no tagged version to pin against; you are tracking the main branch. Upgrading means re-reading .env.example, because provider keys and defaults live there rather than in a versioned config schema, and re-checking that your VoxCPM service still matches what worker expects. The dependency surface in pyproject.toml is small, which keeps the Python side of an upgrade cheap, but the external services are the part that can change under you.

Editorial conclusion

Adopt it if you want a working reference for a LiveKit voice agent with a cloned timbre and a written persona, and you already have a GPU or are willing to rent one. Do not adopt it if you need one-command deployment, a CPU-only box with good cloning quality, or an application that stores long-term memory about the user, because the README states OpenViking captures little when the AI talks more than the caller. Before you commit, verify three things: that VoxCPM answers on http://localhost:8000 on your GPU host, that your Cartesia and MiniMax keys work from your network, and that the felt round-trip delay on your connection is acceptable, since the project documents 2 to 3 seconds in practice rather than the sub-second engineering figure.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. YeJe-cpu/talk-to-fengge on GitHub