acon96/home-llm: A Local LLM Integration for Home Assistant and the Home Models That Feed It

The Problem: Voice Control Without Shipping Your Floor Plan to a Cloud API

Home Assistant already has a conversation layer, but the assistants that plug into it are mostly cloud services. Sending a request like "turn off the kitchen lights" to a hosted model also sends context: the names and states of the entities you have exposed. The README states plainly that external providers "will be able to see the status of all exposed entities in your Home Assistant instance, which can potentially include your current location." That is the problem home-llm targets. It is a Home Assistant custom component plus a set of small models, both maintained in the same repository, so the prompt format the integration emits and the format the models were fine-tuned on stay in sync. The audience is narrow and specific: Home Assistant users who want chat or voice control over lights, switches, fans, covers, locks, climate, media players, vacuums, buttons, timers, todo lists, and scripts, and who are willing to run inference themselves. If you are happy with a cloud assistant and its privacy terms, this project solves a problem you do not have.

Two Halves: The Integration and the Home Models

The repository ships two distinct things that are easy to conflate. The first is the Local LLM Integration, a Home Assistant custom component that connects a language model to the conversation and AI Task systems. The second is the Home model family, small models under 5B parameters fine-tuned specifically for smart home control. The current table lists Llama 3.2 at 3B and FunctionGemma at 270M, with older StableLM-Zephyr, TinyLlama, and Phi-based versions kept under a collapsed section. The integration does not require the Home models. The README describes in-context learning as the fallback: "You can use any instruction-tuned model with in-context learning (ICL). The integration provides examples that teach general-purpose models (like Qwen3, Llama 3, Mistral) how to control your smart home." That is the real architecture split. Fine-tuned Home models are expected to emit Home Assistant service calls more reliably at small sizes; general models lean on prompt examples instead, which costs context window and, in practice, accuracy. The README does not quantify the difference, and no benchmark is given, so treat the fine-tuned route as the intended path and ICL as the escape hatch.

Backends: Where Inference Actually Runs

The integration abstracts the model server behind a backend choice, and the table in the README is the most decision-relevant part of the project. Built-in llama.cpp runs models directly inside Home Assistant, which is the only option that keeps everything in one process. Ollama is positioned for "easy setup on a separate GPU machine." A generic OpenAI-compatible API covers LM Studio, LocalAI, and vLLM. The llama.cpp server backend is called out for "heterogeneous (non-uniform) GPU compute setups, including CPU + GPU inference," which is a more specific claim than most projects make. There are also OpenAI Responses-style and Anthropic Messages-style backends for cloud services, plus text-generation-webui for "advanced users with existing setups." The split matters because it determines your failure modes. A built-in llama.cpp backend competes with Home Assistant for RAM and CPU on the same host. An external server on another machine adds a network hop and a second service to keep alive, but isolates the resource contention. The cloud backends reintroduce exactly the data exposure the project exists to avoid, which the README flags in a note rather than hiding.

Getting It Running: HACS, Core Version, and Model Pinning

Installation goes through HACS. The README opens with a My Home Assistant badge pointing at the repository home-llm under owner acon96, category Integration, which is the standard one-click route. The hard requirement stated is Home Assistant 2026.5.0 or newer. That is a recent core version, and it is the first thing to check, because a custom component that targets newer conversation and AI Task APIs will not load on an older core. Detailed configuration lives in docs/Setup.md, which the README links but does not reproduce, so the exact config keys for each backend are not visible in the material supplied here and I will not guess at them. One version constraint is stated explicitly and is worth repeating: models v1 (3B) and earlier are only compatible with integration version 0.2.17 and older. That is a hard coupling between a model artifact and a component release. If you are running a model from the older list, you cannot simply upgrade the integration. The same coupling logic applies going forward: pin your model and your integration version together, and read the release notes before moving either.

The Version Coupling Is the Real Maintenance Cost

Release cadence is visible in the tags: v0.4.9 in May 2026, v0.4.10 in July, v0.4.11 in September. Roughly every six to eight weeks. For a custom component that is active but not frantic, and each release is a chance for the prompt format or the entity exposure logic to shift under a model that was fine-tuned against an earlier format. The repository mitigates this by keeping the training data and scripts in-tree: the Home-Assistant-Requests-V2 dataset on HuggingFace, the data/ directory, and train/README.md. If a release breaks your model's output, you have the material to re-fine-tune rather than wait. That is a genuine advantage over integrations that consume a black-box model. It is also work most users will not do. The practical cost is that you should expect to re-read release notes on upgrade and to keep a known-good model and integration pair recorded somewhere, because the v1/0.2.17 note shows the project has already made one breaking model-to-integration split. The licence is listed as unknown in the repository metadata, which is a problem worth resolving before you build anything on top of the code or redistribute a modified component. I cannot tell you what the licence permits; check the repository files directly.

Where It Breaks Down: Determinism, Exposure, and Small-Model Failure

The wrong tool case is straightforward. If you need a command path you can audit and reproduce, a language model is the wrong layer. A fine-tuned 270M or 3B model will sometimes pick the wrong entity, emit a malformed service call, or hallucinate a device that does not exist. The integration's job is to constrain that with prompt structure and a fixed device-type vocabulary, but the README offers no accuracy figures, no evaluation harness description, and no statement about what happens when the model returns an invalid call. That silence is the gap. A second limitation is exposure scope. The integration sees the entities you expose, and the README's privacy note implies exposure is the security boundary: anything exposed is visible to whichever backend you chose. Curating the exposed entity list is therefore not optional configuration, it is the control. Third, the "runs on low-power devices" claim is attached to the Home models specifically, and the README does not say what latency to expect on a Raspberry Pi. A 270M model and a 3B model will not feel the same in a voice pipeline, and voice streaming is listed as a feature, which suggests response time is treated as a design concern rather than an afterthought. Test latency on your hardware before wiring it into a voice assistant you use daily.

The Alternative: Intent Matching Instead of Generation

The obvious comparison is Rhasspy, now part of the Home Assistant voice ecosystem, and the difference is architectural rather than cosmetic. Rhasspy-style assistants map utterances to intents using grammars, sentence templates, and trained intent classifiers. The set of things you can say is bounded by what you wrote down, and the system either matches an intent or it does not. There is no generation step, so there is no hallucinated entity and no malformed service call. The cost is that phrasing must be anticipated: "turn on the kitchen light" works because you wrote a template for it, and "it's a bit dark in here" does not. home-llm inverts both properties. It handles phrasing you never anticipated, because the model reads the request and the entity list together, and it can fail in ways a grammar cannot. There is also a middle ground the README gestures at: run a general instruction-tuned model with in-context learning rather than a Home model. That keeps one model for multiple jobs, at the cost of a larger context and, per the project's own framing, less reliable service-call output. Choose based on whether your failure budget tolerates a wrong light turning on occasionally.

A Second Role: AI Tasks, Not Just Conversation

The integration is not limited to the conversation agent slot. The README lists AI Task handling as a separate mode: "Create automations that use AI to process data and generate structured responses." That is a different risk profile from voice control. A conversation agent that misreads a command is annoying; an automation that calls a model to extract structured data and then branches on the result can propagate a bad value into the rest of your home logic. The README points to docs/AI Tasks.md for examples but does not reproduce them, so the shape of the structured output and the failure handling are not visible in the supplied material. If you plan to use this mode, read that document before building automations on top of it, and consider whether the task needs a model at all. Many data-extraction automations are better served by a template or a regex, and adding a 3B model to that path buys flexibility you may not need at a latency and reliability cost you did not budget for.

Editorial conclusion

Adopt home-llm if you run Home Assistant 2026.5.0 or newer, already have hardware that can host a small model (a Raspberry Pi is named as sufficient for the Home models), and want natural-language control without sending entity states to a cloud provider. Do not adopt it if you need a deterministic, auditable command path, or if you are unwilling to pin a model version against an integration version. Before installing, verify three things: your Home Assistant core version, whether you will use the built-in llama.cpp backend or an external server, and which exposed entities the model will see, because the README warns that external providers receive the status of every exposed entity, including location.

Official sources

1. acon96/home-llm on GitHub
2. Issues
3. README
4. Releases