fikrikarim/parlor: an on-device voice and vision assistant in Python

What fikrikarim/parlor actually solves

The README frames this as a reaction to GPT-Live. Its author runs a self-hosted voice AI for English learners and describes OpenAI's release as making that app feel obsolete, so the goal became matching the experience under one constraint: it had to run entirely on a MacBook M3 Pro. Parlor is the attempt. The audience is narrow and specific: people who want a spoken, camera-aware assistant whose audio never leaves the machine, and who are willing to run a model server locally to get it.

The project also states plainly that it is a research preview and that rough edges and bugs are expected. It carries an AI disclosure: development leaned heavily on Claude, with humans leading ideas, testing and debugging, and the README says outright that if you are unhappy with AI-developed code, this software is not for you. That is an unusual amount of self-description for a repository, and it is worth taking at face value rather than treating the v2.0.0 tag as a stability signal.

The cascade behind the conversation

The README publishes a data-flow diagram, and it is the clearest statement of what Parlor is. A browser captures microphone audio and camera frames and pushes PCM audio and JPEG frames over a WebSocket to a FastAPI server. Four components sit behind that socket. smart-turn-v3, described at roughly 20 ms, decides whether you finished your thought. Gemma 4 E4B runs through llama.cpp with a QAT q4_0 quantisation, hears and sees, and streams the reply; a separate action head on the same model emits grammar-forced JSON for timers, modes and research. Kokoro TTS speaks sentence by sentence, on MLX on macOS and ONNX on Linux. An optional background reasoner delegates research to a frontier model.

Turn-taking is split across two places, which is the design choice worth noticing. Silero VAD runs in the browser with a silence cutoff of about 200 ms, so there is no push-to-talk button. Then smart-turn-v3 judges whether the pause was a real end of turn: mid-thought pauses are held silently until you continue, or answered once you stay quiet. VAD alone cannot distinguish a thinking pause from a finished sentence, and Parlor does not pretend otherwise.

Two smaller mechanisms explain a lot of the feel. Everything streams: the transcript appears on screen immediately, and the README claims committing to the transcript first measurably improves accuracy, while the camera frame and speech are pushed through llama.cpp's prompt cache while you are still talking. Actions never ride the speech. Timers, mode switches and research requests are decided by a separate grammar-forced JSON request over the same prompt cache, hidden under TTS playback, so control markup cannot leak into the spoken reply. The README cites benchmarks/archbench.py for a recall of 1.0 against 0.955 for in-band tags, and benchmarks/timerprobe.py for the argument that the server, not the model, must own the clock because a turn-based model cannot ring into silence.

Installing Parlor and having a first conversation

The README's quick start assumes Python 3.12 or newer, llama.cpp, and uv. On macOS, llama.cpp comes from Homebrew; the README points other platforms at llama.cpp's own install guide. The clone and install sequence is short:

git clone https://github.com/fikrikarim/parlor.git
cd parlor

# Install uv and llama.cpp if you don't have them
curl -LsSf https://astral.sh/uv/install.sh | sh
brew install llama.cpp

uv sync
uv run parlor

After uv sync resolves the dependencies, uv run parlor starts the FastAPI server. Open http://localhost:8000, grant camera and microphone access, and start talking. The README notes that models download automatically on first run: roughly 5.7 GB for Gemma 4 E4B QAT plus its multimodal projector, plus the TTS models. Budget for that download before you judge the latency.

Configuration lives in your shell or a .env at the repository root, and .env.example documents the keys. The voice model size is the first dial:

# Model size: e2b | e4b | 12b (Google's official QAT q4_0 GGUFs,
# auto-downloaded from HuggingFace on first run). e4b is the default.
MODEL=e2b

The README's configuration table gives e4b about 1.8x the latency of e2b, and e2b fits in roughly 4 GB of RAM against about 6 GB for the default. If you prefer your own weights, .env.example shows MODEL_PATH and MMPROJ_PATH, which must be set together, and LLAMA_SERVER_URL to use an external llama-server instead of spawning one. Background research stays off unless REASONER_API_KEY is set, so a .env without it keeps the whole loop local.

Where Parlor breaks, and where it is the wrong tool

The hard requirement is llama.cpp build b9503 from June 2026, or newer. The README states that older builds lack Gemma 4 audio or crash loading its mmproj, and that MODEL=12b needs b9512. This is not a soft recommendation. A packaged llama.cpp from a distribution repository is likely to be older than that, and the failure will look like a model-loading crash rather than a version complaint.

Platform support is the second boundary. The README lists macOS with Apple Silicon, or Linux with a supported GPU. Windows is not mentioned. The dependency list in pyproject.toml makes the split concrete: mlx-audio, misaki[en] and num2words are marked sys_platform == 'darwin', while kokoro-onnx is marked sys_platform == 'linux'. The two platforms do not run the same TTS stack, and KOKORO_ONNX=1 exists to force the ONNX backend even on Apple Silicon.

Memory is the third. The default e4b model wants about 6 GB free, and MODEL=12b needs about 8 GB. On a machine that is already running a browser, an editor and a container runtime, that is not a small ask. Finally, the README's own framing matters: this is a research preview, and the AI disclosure says the code was developed with strong assistance from Claude. If you need a supported product with a maintenance commitment, or if you are not comfortable running AI-assisted code, this is the wrong tool regardless of how well the architecture fits.

Parlor versus a plain speech-to-text and TTS pipeline

The obvious alternative is assembling the same pieces yourself: a VAD library, a speech-to-text model, an LLM call and a TTS engine, glued together with your own turn logic. Parlor's difference is not the model list, it is the turn logic and the action channel. A hand-rolled pipeline typically treats a silence threshold as the end of a turn, which cuts people off mid-thought; Parlor adds smart-turn-v3 on top of the browser VAD to make that judgement. A hand-rolled pipeline also usually lets the model emit control instructions in the same stream as the speech, which is why in-band tags leak into TTS. Parlor forces actions into a separate JSON request over the same prompt cache, and the README reports the recall difference between the two approaches.

Parlor's own README describes an earlier attempt that went the other way: fine-tuning Gemma 4 12B to behave like a full-duplex model, grafting a decision tick and a speech head onto it. That failed after multiple trials, and the README's conclusion is that a classic cascade is still better until a frontier lab releases a full-duplex model on par with GPT-Live. That is a useful piece of evidence about where the state of the art sits, and it also tells you what Parlor is not: it is a cascade, with the latency that implies, not a true full-duplex model.

Modes, timers and the background reasoner

Beyond the default conversation, Parlor exposes behaviour through spoken commands rather than settings. Live translation mode is entered by saying "Translate everything I say into English" and turns the system into a consecutive interpreter: each utterance is rendered after a short silence, with no conversational replies, in any language Gemma understands, until you say "stop translating" or press the stop chip. Just-listen mode is entered by asking it to listen for a while, and makes it a silent scribe that transcribes every utterance on screen and speaks nothing back until you address it again.

Timers are owned by the server, not the model, on the argument in benchmarks/timerprobe.py that a turn-based model cannot ring into silence. The model announces the ring out loud in any mode, and a countdown chip with a cancel button tracks it. The model is also told how much quiet preceded a turn, how long research took, and when the session started, so a question like "how long was I gone?" has something real to answer from.

Background research is the one feature that can leave the machine. When a question needs web search or deep research, the voice model hands it to a frontier model on any OpenAI-compatible endpoint and weaves the answer back into the conversation while it continues. The defaults in .env.example point at OpenRouter, with REASONER_MODEL defaulting to anthropic/claude-sonnet-4.5 there and to openai/gpt-5.6-luna in the README's configuration table, and web search enabled by appending ':online' to the model name. REASONER_TIMEOUT defaults to 90. The README is explicit that the feature is off unless REASONER_API_KEY is set, and that without it Parlor stays fully on-device.

Licence, maintenance and what an upgrade costs

Parlor is Apache-2.0, which permits commercial use and modification, but the repository's LICENSE file is the document that governs, and this is not legal advice. The practical licensing question is not Parlor's own code but the models it downloads: Gemma weights and Kokoro TTS carry their own terms, and the README does not restate them. If you plan to ship something built on this, read those model licences separately.

On maintenance, the facts are the dates. The last push to main was on 2026-08-03. The repository is not archived. Two releases exist, v1.0.0 on 2026-07-29 and v2.0.0 on 2026-08-02, which is a fast early cadence rather than a long track record. The README's own research-preview label is the honest summary of what that cadence means for a production dependency.

Upgrade cost is dominated by the llama.cpp build requirement and the model download. Moving from e2b to e4b changes the answer quality and the latency, and moving to 12b requires build b9512 and about 8 GB of RAM. Pointing MODEL_PATH and MMPROJ_PATH at your own GGUFs means you own the quantisation choice, and the README does not document a rollback path for a model swap, so keep a working .env before you change it. The CHANGELOG.md at the repository root is where the release-to-release changes are recorded.

Editorial conclusion

Parlor suits engineers with an Apple Silicon Mac or a supported Linux GPU who want to inspect a full-duplex-style voice loop without sending audio to a vendor, and who accept a research preview with rough edges. It is the wrong choice if you need a supported product, Windows, or a hosted service. Before committing, verify that your llama.cpp build is b9503 or newer (b9512 for MODEL=12b), that roughly 6 GB of RAM is free for the default e4b model, and that a .env with no REASONER_API_KEY really keeps research off.

Frequently asked questions

Official sources

1. fikrikarim/parlor on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases