Home Generative Agent: a LangGraph agent for Home Assistant that keeps the LLM out of the safety loop

An agent that acts on the home, not a chat passthrough

Home Generative Agent, or HGA, is a Home Assistant integration that puts a generative agent inside a smart home. The README draws the line it is built against: most AI conversation integrations are prompt passthroughs that forward your words to an LLM and read the answer back. HGA instead uses tools to control entities, query history, watch cameras and write real Home Assistant automations.

The intended user is a Home Assistant owner who wants conversational control that does something, creating an automation by describing it, asking a camera what it sees, getting an alert when something is wrong, rather than a bot that only answers questions. It is distributed through HACS, the community integration manager, and licensed MIT.

The reason it is worth studying is the architecture underneath. The README says it is a full agent built on LangGraph that keeps long-term semantic memory in pgvector so it remembers preferences across conversations, and that everything runs against the model provider you choose, including fully local, so no data has to leave your home. That last point is the difference between a cloud gadget and something a privacy-minded user can actually run.

Sentinel: deterministic safety with the LLM as advisor

The design decision that stands out is how HGA handles anything consequential. The README states that its Sentinel anomaly engine keeps safety decisions deterministic, with the LLM advising but never actuating. Rules, not a language model, decide whether a lock left open or an unknown person is a problem.

This is the right instinct for a system that can open doors. An LLM is probabilistic and can be steered by unexpected input, so wiring it directly to a lock is a category of risk. HGA keeps the actuation path deterministic and lets the model triage and suggest. The same caution appears in automation creation: with the Critical Action PIN enabled, an automation that would open a door lock or a garage door is held for PIN confirmation before it is installed, the README says, the same as a direct command.

Sentinel's scope is broad and concrete. Beyond locks left open and open entries, the README describes a built-in Home Assistant security audit that flags new admin accounts and access tokens, old long-lived tokens, failed logins, public webhooks, offline security devices, Z-Wave locks included without S2 security, an open Zigbee2MQTT join window and pending coordinator firmware, with no router integration required. Asking "is my home secure?" runs the same checks on demand and reports which ones it could not run, which is an unusually honest touch.

LangGraph tools, pgvector memory, and a local voice pipeline

The capabilities trace to specific components. Conversational control and automation writing are LangGraph tools that call Home Assistant services, so when you describe an automation the agent writes the YAML, registers it, and it appears in the HA automation editor. Camera analysis works with Axis, Ring via ring-mqtt, Reolink, UniFi Protect and any camera exposing a motion entity, with optional face recognition through an external face-service that also powers Sentinel's unknown-person rules.

Long-term memory is semantic search over past conversations held in pgvector, which the README provides as a bundled Home Assistant app so you do not stand up PostgreSQL separately. That is what lets the agent remember preferences across sessions rather than treating each chat as new.

The voice pipeline can stay entirely on your network. The README describes built-in speech-to-text and text-to-speech providers for Assist pipelines, backed either by the OpenAI Whisper and speech APIs or by a local OpenAI-compatible server such as Speaches running faster-whisper, with one Speaches container serving both STT and TTS using Kokoro or piper voices. Combined with Ollama for the LLM, the whole loop, listening, thinking and speaking, can run without a cloud call.

Installing through HACS, and the developer path

For users, HGA installs the way Home Assistant integrations do. The README's requirements table lists Home Assistant 2025.5.0 as the minimum, with 2026.4.0 or newer for streaming responses, HACS for the recommended install path, PostgreSQL with pgvector provided as a bundled HA app, and at least one model provider among OpenAI, Gemini, Anthropic, Ollama or any OpenAI-compatible server. You add the repository in HACS, install the integration, set up the bundled pgvector app, and configure a provider, all through the Home Assistant UI rather than a shell.

Contributors work from a checkout, and the repository's `Makefile` defines the setup. A virtual environment and the dependency sets are created with make targets:

make venv
make devdeps
make test

`make venv` creates the `hga` virtual environment and packaging tooling, `make devdeps` installs the development requirements, and `make test` runs the suite. There is also `make lint`, which regenerates the pinned runtime requirements and checks formatting with Ruff. The runtime dependency list is generated from a manifest by `scripts/gen_manifest_requirements.py`, so runtime pins are produced rather than hand-edited, which the lint target enforces with a `git diff --exit-code`.

The cost of running it: dependencies and optional services

HGA's honest limitation is that its full feature set assumes real infrastructure. The pgvector-backed memory needs a PostgreSQL app running, face recognition needs an external face-service, and the local voice pipeline needs an STT/TTS server such as Speaches, with the README recommending roughly 2 GB of VRAM for `faster-whisper-large-v3-turbo`. Fully local LLM inference needs an edge GPU server running Ollama, vLLM, llama.cpp or LiteLLM. None of these is required to start, but the advertised experience is a stack, not a single add-on.

The unknown-person Sentinel rules are a clear example of the conditional design: the README says they never fire unless face recognition is enabled, with the `face_recognition` option off by default and the external service configured. So a user who skips the face-service simply does not get those alerts, which is the correct default but a real limitation on what works out of the box.

Home Assistant version sensitivity is the other cost. Streaming responses need 2026.4.0 or newer, and the requirements note specific schema-core support, so keeping HGA working means keeping Home Assistant reasonably current. On an older HA install, some features are unavailable rather than degraded.

Against a plain conversation-agent integration

The alternative most people reach for is Home Assistant's own conversation integration pointed at an LLM, or a similar prompt-passthrough add-on. That is far simpler to set up: connect a model, talk to it, and it answers questions about your home.

The difference in approach is agency and safety. A passthrough reads your words to the model and the reply back; it does not keep memory across sessions, write real automations, run deterministic safety rules or watch cameras. HGA does all of that, at the cost of the LangGraph agent, the pgvector memory and the optional services described above. Crucially, HGA's Sentinel design means the parts that can affect physical safety are rule-driven with the LLM only advising, which a naive LLM-in-the-loop setup does not guarantee. Choose the simple passthrough if you want conversational answers and nothing that acts on its own. Choose HGA when you want an agent that creates automations, remembers context and enforces safety deterministically, and you are willing to run the supporting pieces, ideally locally, to keep your home's data at home.

MIT licensing and keeping it current

HGA is MIT licensed, so it can be forked, modified and reused freely with attribution, which suits a HACS integration that people will adapt to their own setups. Because it is distributed through HACS, upgrades arrive as integration updates you apply in Home Assistant rather than manual reinstalls.

The maintenance cost is coupling to two fast-moving things: Home Assistant releases and the model providers. The requirements table already tracks specific HA versions for specific features, so an HA upgrade can change what HGA can do, and the Ruff-enforced, manifest-generated runtime pins in the repository show the maintainers taking dependency drift seriously. Anyone running fully local also inherits the upkeep of Ollama or a similar server and, if used, the face-service and Speaches.

The concrete first step is to decide the provider and locality question before installing, because it shapes everything else: pick whether you are running cloud models or fully local via Ollama, stand up the bundled pgvector app for memory, and enable the Critical Action PIN if any automation could open a door lock or a garage door, since that is the guard that holds a physical action for confirmation.

Editorial conclusion

Adopt Home Generative Agent if you run Home Assistant and want an agent that writes real automations from plain English, remembers context in pgvector, and can run fully local via Ollama so no data leaves your home: the deterministic Sentinel engine, which advises with the LLM but actuates only through rules, is the reason to prefer it over a chat passthrough. It is a poor fit if you want a one-click add-on, since the full experience needs pgvector, optional face and voice services, and a reasonably current Home Assistant. Before installing, decide cloud versus local models, stand up the bundled pgvector app, and enable the Critical Action PIN if any automation could open a door lock or a garage door.

Frequently asked questions

Official sources

1. goruck/home-generative-agent on GitHub
2. License: MIT
3. Project website
4. README
5. Releases