local-voice-ai runs a private voice assistant on your own hardware

Speech recognition, a language model and speech generation in one process

The problem is that building a voice assistant means stitching together a speech recogniser, a language model and a speech synthesiser, each with its own runtime, its own model format and its own latency profile. local-voice-ai puts the three together in one supervised application, and the README describes it as a private, low-latency voice assistant that runs on your hardware, powered by LiveKit Agents. The package metadata says the same in more technical terms, describing it as a single supervised process bundling LiveKit Agents, llama.cpp, Nemotron STT and Kokoro TTS. The application includes a browser voice interface, local streaming speech recognition with Nemotron Q8, local language models through llama.cpp, local speech generation with Kokoro, automatic setup for CPU, NVIDIA, Apple Silicon and Jetson, and a remote client mode for devices that run without a local browser.

A launcher that reads the hardware and picks the model stack

The selection logic is the interesting part of the design, and it is not a config file you have to write. The launcher shows the detected hardware, the memory budget and the recommended models, and you accept the recommendation or select a different profile. Automatic selection uses the device type to pick a runtime, CPU, CUDA or Metal, and then uses the memory budget to pick a model profile. There are four profiles, and the table pairs each with a memory target, a language model, a context size, the recogniser and the voice: lean and jetson-realtime target about 4.7 GB with Qwen3 1.7B at 4K context, compact targets about 5.5 GB with Gemma 4 E2B at 4K, and balanced targets about 6.5 GB with Gemma 4 E2B at 16K. The memory values are planning targets rather than hard limits, and the automatic mode keeps memory available for the operating system and active conversations.

Nemotron streams partial transcripts, Whisper waits for the end

The recogniser choice is a latency decision, and the README is explicit about the difference. All profiles use the native streaming Nemotron Q8 runtime, and the launcher selects the CPU, CUDA or Metal runtime for the device. The default language is English, for which the application uses the English-specific Nemotron model, and for another supported language it uses Nemotron 3.5. The language is set in .env.local with the STT_LANGUAGE key, and if the speaker language can change you use the auto value, which selects the multilingual model. Whisper remains available as a manual fallback through the STT_PROVIDER key, but it waits for a complete utterance before transcription, whereas Nemotron sends partial transcripts while the user speaks, so Nemotron has lower voice latency.

The launcher and the common commands it exposes

Everything is driven through a single Python entry point, run.py, and the README lists the command set. Running it with no arguments configures and starts the application, there is a configure subcommand to select a different profile, a plan subcommand to show the selected runtime and models without starting anything, status to show service readiness, logs to follow the application logs, and down to stop the Docker application. There is also a client subcommand that takes a server argument to run the interface for a remote server. For a non-interactive start the launcher accepts a profile, a memory budget and a yes flag, which is what you want in a script:

python3 run.py start --profile auto --yes

The launcher saves the selected profile in a .local-voice-ai.toml file, which is local to the device and is not committed to Git, so the choice of profile is machine state rather than repository state.

Container first on Linux and NVIDIA, native tools on Apple Silicon

The installation is split by platform, and the split is not cosmetic. For Linux CPU you need Docker Engine with Docker Compose, for desktop NVIDIA you need Docker Engine, Compose and the NVIDIA Container Toolkit, and for a Jetson Orin you need JetPack 6.2, L4T 36.4 and the NVIDIA Docker runtime. Apple Silicon is the odd one out, needing Python 3.11 to 3.13, uv, livekit-server and llama-server, installed with Homebrew along with the project dependencies through the uv sync command:

brew install livekit llama.cpp
uv sync --extra ml --extra dev

The Dockerfile is a three stage build, a frontend stage that produces a Next.js static export, a binaries stage that pulls the livekit-server and llama-server binaries from upstream images, and a runtime stage that is Python 3.11 with the dependencies, the binaries and the frontend together. The compose file has GPU and Jetson overlays for the other two platforms.

A Jetson as the voice server with the browser on a laptop

The Jetson setup runs the voice stack on the Jetson and the browser interface on a laptop, which gives the browser a localhost address for microphone access, and that is the point of the arrangement. The Jetson setup needs approximately 29 GB of free disk space, and the first build compiles native components so it takes longer than later builds. The client is installed with Node.js 20 on the laptop, cloning the repository, enabling corepack and running the launcher in client mode against the server address:

git clone https://github.com/ShayneP/local-voice-ai.git
cd local-voice-ai

and the client installs its frontend packages on the first start. The Jetson needs four ports opened for the laptop, 8080 for connection details and status, 7880 for the LiveKit connection, 7881 as the WebRTC fallback media and 7882 over UDP for WebRTC media. The README includes the exact ufw commands to permit only the local subnet, and warns in capitals not to expose the ports to the public internet because the default service uses development credentials.

The compose file is where the environment surface is visible

Reading docker-compose.yml tells you more about the configuration than the prose does. The service builds with build arguments for the llama image, the LiveKit image, the Python base image, the torch index URL and the NeMo speech backend, and it loads two env files, .env and an optional .env.local, with a comment explaining that per-machine settings such as MANAGE_LIVEKIT and LIVEKIT_NODE_IP would otherwise be lost at the container boundary. The environment block then lists the recogniser and synthesis settings with their defaults, among them STT_PROVIDER defaulting to nemotron-cpp, STT_LANGUAGE defaulting to en, a Nemotron model name and model id, a Whisper model for the fallback path, a TTS voice of af_nova and a TTS provider of kokoro. The Python dependencies are similarly split, with the heavy machine learning packages in an optional ml extra so that torch is not forced into every development environment, and a separate whisper extra for the fallback.

Editorial conclusion

local-voice-ai suits someone who wants a voice assistant that runs on their own machine, whose audio never leaves it, and who is prepared to give the box several gigabytes of memory and a first run with a network connection. It does not suit a team that needs a hardened deployment today, because the default LiveKit credentials are development ones and the README warns against exposing the ports publicly, and it does not suit a machine without enough memory for the chosen profile, since the smallest one still targets about 4.7 GB. Before deploying, pick the profile with run.py plan, set the memory budget with the memory-gb flag rather than relying on automatic detection, and note that the last push to the repository was on 2026-08-31 with no GitHub releases published.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. ShayneP/local-voice-ai on GitHub