LocalAGI: a self-hosted agent platform with an OpenAI Responses-compatible API

The problem LocalAGI targets: agents without cloud keys

Most agent tooling assumes two things: a hosted model endpoint and a Python environment. LocalAGI rejects both. The README frames the project as a reaction to "AI wrappers calling out to cloud APIs, risking your privacy", and the pitch is that you bring a GPU or even just a CPU and a browser, and get customizable assistants, automations, chat bots and agents that run locally. No agentic Python libraries, no cloud service keys.

The audience is therefore narrow and specific. It is for someone who wants an agent reachable from Slack, Discord, Telegram, IRC or GitHub Issues, backed by a model running on their own hardware, configured through a web UI rather than a Python file. It is not aimed at teams whose main constraint is model capability. A locally served model is a different product from a frontier hosted model, and the README does not claim otherwise.

The second audience is subtler: anyone who wants an OpenAI-compatible Responses endpoint backed by their own model. The README states that every agent created will support the OpenAI Responses API out of the box. That makes LocalAGI usable as a compatibility layer, not only as an agent builder.

How LocalAGI is put together: Go core, web UI, MCP, skills

The repository is a Go module, github.com/mudler/LocalAGI, on Go 1.26.0, with main.go and cmd/ at the top level, core/ and pkg/ for the implementation, services/ and tests/ alongside. The HTTP layer is gofiber/fiber/v2 with a template engine, and the CLI is built on spf13/cobra. The agent reasoning layer comes from github.com/mudler/cogito, and the knowledge base is backed by github.com/mudler/localrecall, the same library the README says the embedded Knowledge base section uses. Vector search sits on blevesearch/bleve and philippgille/chromem-go.

Connectors are ordinary Go dependencies, and the go.mod file makes that concrete: bwmarrin/discordgo for Discord, slack-go/slack for Slack, go-telegram/bot for Telegram, google/go-github/v69 for GitHub, thoj/go-ircevent for IRC, and maunium.net/go/mautrix for Matrix. The README lists Discord, Slack, Telegram, GitHub Issues and IRC as built-in connectors, so the Matrix dependency is present without being advertised in that list.

Two design choices stand out. Custom actions are written in Go and interpreted through traefik/yaegi, which the README describes as "interpreted, no compilation". That removes a build step but ties your action code to what yaegi can interpret. Skills follow the skillserver format, via github.com/mudler/skillserver, and the README says they can be created, imported or synced from git, with skill tools and the skill list injected into an agent when Skills are enabled for it. Scheduling uses robfig/cron/v3, which is what the cron-like periodic tasks are built on.

The data flow for a typical deployment is visible in docker-compose.yaml: a LocalAI container serves the model on port 8080 internally, mapped to 8081 on the host, with a healthcheck against /readyz; a Postgres container from quay.io/mudler/localrecall provides the knowledge base storage on port 5432; and the compose file also defines an sshbox service and a docker-in-docker daemon, which is a heavier footprint than a plain agent runtime.

Installing LocalAGI with Docker Compose

The README gives the clone-and-compose path as the quickstart. The default compose file is the CPU setup and pulls localai/localai:master with a default model of gemma-3-4b-it-qat, plus granite-embedding-107m-multilingual for embeddings. Run this from the repository root:

git clone https://github.com/mudler/LocalAGI
cd LocalAGI
docker compose up

The first start is slow because the LocalAI container downloads models into the models volume. The compose file sets a healthcheck interval of 60s with 120 retries, which tells you the maintainers expect a long warm-up rather than a fast boot. Once the stack is up, the README says agents are managed at http://localhost:8080.

GPU users pick a different compose file. The README lists one per vendor:

docker compose -f docker-compose.nvidia.yaml up
docker compose -f docker-compose.intel.yaml up
docker compose -f docker-compose.amd.yaml up

You can also override the model without editing the compose file, because the model name is passed as a container command argument through an environment variable:

MODEL_NAME=gemma-3-12b-it docker compose up

The README points at models.localai.io for available model names, or localai.io if you want any Hugging Face model, and shows a multi-model form that also sets MULTIMODAL_MODEL and IMAGE_MODEL for the NVIDIA file. If you prefer to build the binary yourself, the Makefile builds the React UI in an oven/bun:1 container and then runs go build -o localagi ./; make run does the same and starts the server, and make run-nokb sets KBDISABLEINDEX=true for a run without knowledge base indexing.

Where LocalAGI gets in your way

The most honest limitation is the one the project chooses: local inference. The default model in docker-compose.yaml is gemma-3-4b-it-qat, a small quantized model. The README's CPU profile says it supports text models only, which means vision and image generation require a GPU compose file and additional model configuration. If your agent task depends on strong multi-step reasoning, the local model is the ceiling, and no amount of agent scaffolding raises it.

Operationally, the stack is not light. The compose file brings up LocalAI, Postgres, an sshbox service and a privileged docker-in-docker daemon. The sshbox service is built from Dockerfile.sshbox with SSH_USER and SSH_PASSWORD both set to root and DOCKER_HOST pointing at the dind daemon. That is a development convenience, not a hardened deployment, and anyone running this outside a laptop should read the compose file before exposing port 8080. The README does not document authentication for the web UI.

Custom actions are interpreted, not compiled, which is a trade-off rather than a pure win: the README states actions are scripted in Go with no compilation, and the yaegi interpreter defines what that Go code can do. Expect to test action behavior at runtime rather than rely on the compiler.

Finally, the documentation is uneven. The README covers installation, connectors, skills, memory and observability at a feature level, but it does not document rollback of agent configuration changes, migration between releases, or a supported upgrade procedure. Treat the release notes as the only signal for what changed between v2.8.0, v2.8.1 and v2.9.0.

LocalAGI against a code-first agent framework

The natural comparison is a Python agent framework such as LangChain, which appears in go.mod as tmc/langchaingo, the Go port LocalAGI uses for parts of its model and chain handling. The difference in approach is where the agent lives. In a Python framework, the agent is code you write and version: you compose tools, prompts and memory in a script, and the runtime is whatever you deploy it into. In LocalAGI, the agent is a record in the web UI, and the runtime is a server that already knows how to talk to connectors, a knowledge base, cron schedules and an OpenAI-compatible endpoint.

That flips the cost. A Python framework gives you arbitrary control and no UI; LocalAGI gives you a UI and a fixed set of extension points, of which custom Go actions and skillserver-format skills are the two the README documents. If your agent logic is a one-off research script, the framework is lighter. If your agent needs to live in Slack and Telegram and answer on a schedule, LocalAGI has already written that plumbing, and the go.mod dependency list is the evidence.

The second alternative is running LocalAI alone and calling the model directly. That is simpler and skips the agent layer entirely, but you lose the connectors, memory, scheduling and the per-agent Responses endpoint.

Maintenance, licence and upgrade cost

LocalAGI is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are included. The repository is not archived, and the last push was on 2026-08-25, so the project is being worked on. The release cadence visible in the repository is modest: v2.8.0 and v2.8.1 both landed on 2026-02-16, and v2.9.0 followed on 2026-05-08. There is no published upgrade guide in the repository.

The practical maintenance cost is the model server, not the agent binary. The compose file pins localai/localai:master, a moving tag, while the knowledge base image is pinned to ${LOCALRECALL_VERSION:-v0.5.2}-postgresql. Mixing a floating tag with a pinned one means a docker compose pull can change your inference backend without changing your agent version. Pin the LocalAI image to a specific tag if you want reproducible stacks.

Builds require Go 1.26.0, per go.mod, and the React UI is built through an oven/bun:1 container, so a source build needs Docker even if you never run the compose stack. The MIT licence covers the code; it says nothing about the licences of the models you download, which is a separate question from the model sources the README points to.

Editorial conclusion

Adopt LocalAGI if your agents must run on hardware you control, you already have a local model server such as LocalAI, and you want the web UI plus an OpenAI Responses-compatible endpoint rather than a Python framework. Do not adopt it if you need frontier-model reasoning quality, if you cannot run Docker Compose or keep a model server warm, or if you expect the README to tell you how to roll back a configuration change. Before committing, verify three things on your own machine: that your chosen compose file starts and the web UI answers on port 8080, that your model responds through the agent's Responses endpoint, and that the connectors you plan to use are covered by the credentials you are willing to store in the agent configuration.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. mudler/LocalAGI on GitHub
4. README
5. Releases