local-talking-llm: a Whisper + Ollama + Chatterbox voice loop you assemble yourself
A talking LLM that runs on your own computer without needing the internet.
At a glance
- What is it?
- The repository is a tutorial-grade Python script that chains speech recognition, a local or cloud LLM, and Chatterbox text-to-speech into a spoken conversation loop. It is a working starting point for offline voice assistants, not a packaged product, and the README is honest about the dependency management being the hard part.
- Who is it for?
- Adopt this if you want a readable, MIT-licensed reference for wiring Whisper, a Langchain LLM backend, and Chatterbox TTS into a spoken loop, and you are comfortable managing Python dependencies yourself with uv. Do not adopt it if you need a packaged application, a stable API, or a support commitment, because the README itself warns that the pinned requirements.txt may not install across systems.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 164 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap between a chatbot and something you can talk to
Most local LLM projects stop at text. You type, the model answers, and the loop is complete. local-talking-llm extends that loop with two more stages: speech in and speech out. The README frames the goal as a voice assistant "reminiscent of Jarvis or Friday from the iconic Iron Man movies, which can operate offline on your computer." The audience is developers who already run a model locally and want to add a microphone and a speaker without routing audio through a cloud API. The repository is Python, MIT-licensed, and organized around a single app.py entry point. It is not a library you import. It is a script you run, read, and modify. That distinction matters for anyone evaluating it as a dependency rather than a reference.
Three stages, one loop, and where the LLM actually runs
The architecture is a straight pipeline. Speech recognition uses OpenAI Whisper to convert spoken language into text. That text goes into what the README calls the conversational chain, a Langchain interface with a pluggable backend. The backend is either a local model served by Ollama (the README names Gemma3 and Llama-4 as examples) or a cloud model from MiniMax (MiniMax-M2.7). The generated response then goes to Chatterbox TTS, which produces audio output the user hears. The README describes the workflow plainly: record speech, transcribe to text, generate a response using an LLM, and vocalize the response using Chatterbox. The pluggable backend is the most interesting design decision here. It means the same script can run fully offline with Ollama or fall back to a cloud provider when local hardware is not enough. The trade-off is that the cloud path breaks the offline promise for the LLM stage while speech recognition and synthesis stay local. The README states this explicitly: no Ollama installation is needed when using MiniMax, and "the LLM runs in the cloud while TTS and STT still run locally." So the offline claim is conditional, not absolute.
Getting it running: uv, NLTK data, and an Ollama pull
The README recommends uv over pip and gives a specific reason: the requirements.txt file was generated by uv pip freeze and "contains pinned versions that may not install correctly across different systems." That is an unusually candid warning, and it should shape how you approach setup. The uv path is a short sequence: install uv with the curl script or brew on macOS, clone the repository, run uv sync, activate .venv, then download NLTK data with python -c "import nltk; nltk.download('punkt_tab')". The pip alternative installs from pyproject.toml with pip install -e . and downloads the punkt tokenizer instead of punkt_tab. Note the tokenizer name differs between the two paths, which is easy to miss. For the LLM backend you install Ollama, then run ollama pull gemma3 or another model. For the cloud path you export MINIMAX_API_KEY and skip Ollama entirely. Usage is a single command, python app.py, with flags for voice cloning (--voice path/to/voice_sample.wav), emotion and pacing (--exaggeration 0.7 --cfg-weight 0.3), model selection (--model codellama), and saving samples (--save-voice).
Voice cloning and the watermark that comes with it
The May 2025 update replaced Bark with Chatterbox TTS, and the README lists four consequences: voice cloning from a short audio sample, emotion control, a 0.5B parameter model with faster inference, and built-in neural watermarking. The watermark is the detail worth pausing on. Chatterbox embeds a neural watermark in generated audio, and the README presents this as a feature for authenticity. In practice it means audio produced by this pipeline is marked, which is useful if you care about provenance and potentially inconvenient if you wanted clean output for further processing. The README does not explain how to detect or remove the watermark, and it does not say whether the watermark survives downstream editing. That is a gap. Voice cloning is described as needing a 10 to 30 second sample, passed through the --voice flag. The README does not discuss consent, storage, or what happens to the reference sample after cloning. For a feature that reproduces a person's voice, that silence is notable. The --exaggeration and --cfg-weight flags control emotional expressiveness, but the README gives no guidance on sensible ranges beyond the example values.
Where this breaks down, and who should not use it
The README's own dependency warning is the first real limitation: pinned versions from uv pip freeze are not portable, and the project tells you to prefer uv sync over requirements.txt. If your environment cannot use uv, you are on the pip path with pyproject.toml, which the README presents as an alternative rather than the supported route. Second, there are no releases. The repository has no retrieved releases, so there is no versioned artifact to pin against. You track the main branch, and the README shows the project has already made a breaking change by swapping Bark for Chatterbox, preserving the old implementation in the archive-2025-05-29 branch. Anyone depending on Bark behavior must check out that branch. Third, the offline claim is only true on the Ollama path. Choosing MiniMax sends your prompts to a cloud provider, which changes the privacy and network profile entirely. Fourth, this is a script, not a service. There is no mention of an API, a daemon mode, concurrency, or error recovery. If the microphone fails or the model returns something unexpected, the README does not describe what happens. For a demo or a personal assistant, that is acceptable. For anything you intend to run unattended, it is not.
What you would use instead
The most direct alternative is to build the same pipeline yourself from the three components the README already names: openai-whisper, a Langchain LLM interface, and Chatterbox TTS. That is essentially what this repository does, so the difference is that you would own the glue code, the error handling, and the dependency pinning. The benefit is control over exactly which versions you lock and how failures surface. The cost is that you reimplement the recording, tokenization, and playback logic that app.py already contains. A second alternative is a cloud voice assistant stack, which trades the offline requirement for managed speech recognition and synthesis. That path removes the Whisper and Chatterbox installation burden, including the model downloads and the NLTK data step, but it sends audio off your machine. The README's MiniMax option is a partial version of this: cloud LLM, local speech. The comparison is not about which is better in the abstract. It is about which stage of the pipeline you are willing to move off your hardware. This project lets you choose per stage, which is its main structural advantage over a monolithic cloud assistant.
Maintenance, licensing, and what to check before you commit
The repository is MIT-licensed, which permits commercial and private use with the usual attribution and warranty disclaimer. That covers the code in this repository. It does not automatically cover the models you pull: Ollama models, Whisper weights, and Chatterbox TTS each carry their own licenses, and the README does not summarize them. Check each model's terms separately before shipping anything. On maintenance, the last push is dated 2026-04-04 and the README documents a significant migration from Bark to Chatterbox in May 2025, which suggests the author does update the stack. There are no releases, so upgrades mean pulling main and re-running uv sync, then re-testing the pipeline end to end. Budget for that: a TTS or Whisper change can alter audio output in ways that are hard to catch without listening. The NLTK punkt_tab versus punkt discrepancy between the two install paths is the kind of detail that will bite on a fresh machine. Before adopting, run uv sync on your target platform, confirm Chatterbox and Whisper install cleanly, decide between Ollama and MINIMAX_API_KEY, and confirm which branch you are on if you need the Bark behavior.
Editorial conclusion
Adopt this if you want a readable, MIT-licensed reference for wiring Whisper, a Langchain LLM backend, and Chatterbox TTS into a spoken loop, and you are comfortable managing Python dependencies yourself with uv. Do not adopt it if you need a packaged application, a stable API, or a support commitment, because the README itself warns that the pinned requirements.txt may not install across systems. Before committing, verify that Chatterbox TTS and openai-whisper build on your platform, confirm whether you are pointing at Ollama or MiniMax via MINIMAX_API_KEY, and check which branch holds the current code versus the archived Bark implementation.
Community notes