LLM Vision: multimodal camera analysis inside Home Assistant
Visual intelligence for your home.
At a glance
- What is it?
- A HACS integration that routes snapshots, video files, live feeds and Frigate events through an LLM provider and writes the result back into Home Assistant sensors and a timeline. The value is the plumbing, not the model.
- Who is it for?
- Adopt LLM Vision if you already run Home Assistant with cameras or Frigate and you want descriptions and extracted values as sensors without writing your own provider client. Do not adopt it if you need on-device inference with no external API dependency, or if you are not prepared to mount and back up the /media folder.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 11 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap LLM Vision fills in a Home Assistant camera setup
Home Assistant can tell you a camera detected motion. It cannot tell you whether the thing that moved was a delivery van, a fox, or a neighbour walking a dog. The integration exists to close that gap: it sends images, video files, live camera feeds and Frigate events to a multimodal model along with a prompt you write, and returns a description or an answer. The README frames the project as an integration that "uses multimodal large language models to analyze images, videos, live camera feeds, and Frigate events." The intended user is someone who already has cameras and automations in Home Assistant and wants the detection layer to say something more specific than motion. It is not a standalone service, not a model, and not a replacement for Frigate's own detection. It sits on top of hardware and software you already run, and its output is only as good as the frames it is handed.
What happens between a camera event and a sensor update
The architecture is a Home Assistant integration with a provider abstraction underneath. The README lists the supported backends: OpenRouter, OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure, Groq, Ollama, Open WebUI, LocalAI, and "any provider with OpenAI compatible endpoints." That last clause is the design decision that matters. Rather than writing a bespoke client per vendor, the integration targets the OpenAI-compatible request shape and treats everything else as a configuration variant. Adding a provider is mostly a matter of pointing the base URL at a different endpoint.
On the Home Assistant side, the integration stores snapshots in the /media folder, which the README calls "the more secure" option for that purpose. Analysis results are not just displayed. The README states the integration "updates sensors based on data extracted from camera streams, images or videos," which means the model's response is parsed into state rather than left as free text. Alongside that, it keeps "a timeline of camera events" that can be shown on a dashboard through an optional Timeline Card or queried through Assist. There is also a memory feature: the README says it "remembers people, pets and objects," so a recurring subject can be recognised across events rather than described from scratch each time. A blueprint is shipped for the common case of summarising camera event notifications.
Installing it: HACS, a restart, and the /media mount
The README gives a six-step quick start. Install LLM Vision from HACS (it is in the default repository), restart Home Assistant, then find LLM Vision under Settings, Devices and services and press submit to accept the defaults. Step five is the one that breaks installs: set up the media folder, because LLM Vision uses /media for snapshots. The README warns that if you run Home Assistant Container you may need to mount a folder to /media in your container settings. Then return to the integration page and press Add Entry to register your first AI provider.
Provider configuration is documented externally rather than in the README, at llm-vision.gitbook.io under getting-started/setup/providers. That split is worth noting: the repository tells you the integration exists and how to install it, while the per-provider credential and endpoint details live on the docs site. Debug logging is enabled on the integration's settings page, which is what the bug report instructions ask you to attach. If you are evaluating this before installing, read the provider page for whichever backend you intend to use first, because the install steps themselves are provider-agnostic and will not tell you whether your endpoint works.
Where the design costs you: external models, storage and parsing
The honest limitation is that this is a wrapper, and every weakness of the layer below shows through. Most of the listed providers are hosted APIs. That means camera frames leave your network unless you deliberately choose Ollama, LocalAI or Open WebUI, and the README does not present any of those as the default path. Latency is whatever your provider's latency is; there is no batching or local pre-filter described in the material, so a busy camera produces a request per event you route to it. Cost scales with event volume, not with the number of cameras, which is easy to underestimate on a driveway that sees forty cars an hour.
The /media requirement is a second constraint. It is a deliberate security choice, but it moves the integration's working data outside the Home Assistant configuration directory. Container users have to mount it, and anyone with an existing backup routine built around the config folder should check whether snapshots are covered. Finally, sensor updates depend on the model returning data in a shape the integration can extract. The README asserts that sensors are updated from extracted data but does not describe the parsing contract. If your prompt produces prose instead of a value, the sensor is the place that failure surfaces, and it surfaces quietly.
How it differs from Frigate alone or a hand-rolled automation
Frigate is the obvious comparison point, and the difference is one of layer rather than feature. Frigate does detection and object classification locally, on your hardware, and its output is bounding boxes and object labels. LLM Vision consumes Frigate events as one of its four input types, which tells you the relationship: it is downstream of Frigate, not a substitute for it. If you want to know that a person is present, Frigate answers that without an API call. If you want a sentence describing what the person is carrying, Frigate does not attempt it and LLM Vision does.
The other alternative is writing the integration yourself: an automation that grabs a snapshot, POSTs it to a provider, and sets an input_text. That is a weekend of work and it is genuinely viable. What you would be rebuilding is the provider abstraction across eleven backends, the timeline store, the memory of recurring people and objects, the Timeline Card, and the blueprint. The trade is control versus maintenance. A hand-rolled automation breaks when you change providers and you fix it yourself. LLM Vision breaks when a provider changes and you wait for a release, which the release history suggests arrives: v1.7.2 and its beta landed within four days of each other in late August and early September 2026.
Upgrade and licence position
The repository is active, not archived, with a last push of 2026-09-05 and a maintenance badge reading yes/2026. The recent release pattern is small and frequent: v1.7.1 in early August 2026, then v1.7.2-beta.1 and v1.7.2 in the following weeks, all labelled as bug fixes or bug fixes and performance improvements. Nothing in the supplied material describes a migration step or a breaking change between those versions, so I cannot tell you whether upgrading is risk-free; the beta-then-stable cadence suggests the maintainer does test before tagging, but that is an inference from dates, not a documented policy.
The licence is Apache-2.0, which permits commercial and private use and modification, and includes a patent grant. This is not legal advice and I am not your lawyer. The practical point is that the licence is not the constraint here. Your constraint is the terms of whichever model provider you connect, plus the recurring per-request cost. Running the integration against a local Ollama endpoint changes that cost profile entirely, and it is the configuration worth checking first if API spend is the reason you are reading this.
Who should install it, and what to check before you do
Install it if you run Home Assistant with cameras, you already have Frigate or a working snapshot pipeline, and you want descriptions and extracted values as first-class entities rather than as notifications you read and forget. The timeline and the sensor updates are the parts that make it more than a notification formatter, and they are the parts you would spend the most time rebuilding.
Skip it if your requirement is fully local inference with no external dependency at all. Ollama, LocalAI and Open WebUI are listed as supported, so a local path exists, but the README does not present it as the primary configuration and you will be working from the external provider docs to set it up. Skip it too if you cannot mount /media, or if you are unwilling to let snapshots accumulate somewhere your existing backup does not reach.
Before you commit, verify in this order. Confirm the provider endpoint is reachable from the Home Assistant host, not just from your laptop. Confirm /media is writable, which for container installs means checking the mount rather than the integration. Then write one prompt and check what lands in the sensor, because that is where the parsing contract either holds or does not. If the sensor stays empty while the description looks fine, the problem is your prompt shape, not the provider.
Editorial conclusion
Adopt LLM Vision if you already run Home Assistant with cameras or Frigate and you want descriptions and extracted values as sensors without writing your own provider client. Do not adopt it if you need on-device inference with no external API dependency, or if you are not prepared to mount and back up the /media folder. Before committing, verify three things: that your chosen provider is reachable from the Home Assistant host, that /media is actually writable in your container setup, and that your prompt returns the structured values your automations expect. The integration is Apache-2.0, so the licence question is not the blocker; per-image API cost is.
Community notes