LocalRecall: a local memory layer for AI agents

The premise: the memory layer does not have to be remote

The tagline is the argument, and it is short: your AI, your hardware, your rules.

LocalRecall is described as a lightweight, no-frills RESTful API for managing knowledge bases and files stored in vector databases, with no GPU, no internet and no cloud services required. The abstraction layer is generic and sits between an agent and its storage, and the intent is to handle both long-term and short-term memory seamlessly for agents and chatbots.

The practical reason this exists is in the surrounding project family. LocalRecall is one of a set of local tools by the same author, and the readme positions it as a companion to LocalAI, a free open source drop-in replacement for the OpenAI API that runs local inference, and to LocalAGI, an agent management platform that is a drop-in replacement for the OpenAI Responses API.

So the intended deployment is a machine with a model server next to a memory server, and an agent that talks to both over the local network. The compose file in the repository shows exactly that: a LocalAI container for embeddings, a PostgreSQL container, and the RAG server, wired together with a database URL pointing at the local database and an OpenAI base URL pointing at the local model server.

The features list is short and modest. A REST interface for knowledge management, offline operation, retrieval-augmented generation compatibility across multiple vector databases, memory management for applications that need an abstraction, and file support for Markdown, plain text and PDF with more formats promised. There is also a web interface for convenient file management including raw text inputs, and the readme says it integrates easily with LocalAI, LocalAGI and other agent frameworks.

Two backends, and the choice is about where the data lives

The project is described as batteries included, and it supports two vector database engines.

The first is Chromem, a local file-based vector store, and it is the default. That default is the reason the project is easy to try: there is no database to install, no extensions to compile and no connection string to get wrong, and the store is a directory of files you can back up with a copy command.

The second is PostgreSQL, which the readme calls production-ready, with TimescaleDB, pgvector and pgvectorscale for hybrid search combining BM25 with vector similarity. The reason to move is stated plainly: PostgreSQL provides better performance, scalability and hybrid search capabilities.

The compose file in the repository shows the full production shape, and it is more than a database. There is a LocalAI service that the compose file starts with a specific multilingual embedding model as its command, so the embedding model is downloaded or loaded lazily on the first request. There is a PostgreSQL service built from a separate Dockerfile, with a health check that runs the readiness probe every ten seconds with five retries, and a RAG server that depends on that service being healthy.

The health check and the dependency condition are the details that make this work. The RAG server waits for the database to report healthy rather than merely to be running, which avoids the class of failure where the application starts, cannot connect, and exits.

The two Dockerfiles are also instructive. The main one builds the Go binary in a builder stage and copies it into a scratch base image with the certificate store, and the comment in the file explains why the base image is a newer Go than the module directive asks for, because a dependency swap for PDF handling pulled in newer x/* dependencies and the older base image stopped being able to download them.

Hybrid search is the part that needs explaining

The PostgreSQL features are worth reading closely, because hybrid search is where a knowledge base either feels smart or feels like a keyword grep.

The setup is: hybrid search combines BM25 keyword search with vector semantic search with configurable weights, and the weights are environment variables, with examples that set both the keyword and the vector weight to the same value. The indexing is described as advanced, with GIN indexes for full-text search, BM25 indexes for keyword search, and DiskANN or HNSW indexes for vector similarity.

The extensions involved are a pg_textsearch extension providing BM25 keyword search, a vectorscale extension providing advanced vector search with DiskANN, pgvector as the fallback for vector similarity, and TimescaleDB for time-series capabilities. So there is a fallback chain inside the database itself, which is a sensible way to make the same deployment work whether or not a given machine can run the newest index.

Why hybrid matters is worth stating in plain terms. Vector search finds things that mean the same thing, which is what you want when the user's wording differs from the document's wording. Keyword search finds things that contain the exact terms, which is what you want for identifiers, error codes, product names and anything where a synonym is wrong. A knowledge base for a support or engineering audience needs both, and the weights are how you tune the balance for your corpus.

The default of half and half is a reasonable starting point, and the fact that it is a variable rather than a fixed behaviour is the point.

The raw vector API for callers who already have embeddings

There is a second API, and it exists for a specific reason.

The caller-supplied vectors API, under a raw path, is PostgreSQL-only and stores vectors without invoking an embedding model. It supports vector, BM25 and hybrid retrieval in a namespace separate from the legacy collections, and there is a separate guide covering deployment, request examples, consistency limits and verification.

The use case is obvious once you say it. If your agent already produces embeddings, for example from a model you run yourself or from a service you already pay for, then having the knowledge base re-embed your documents on the way in is both slow and a needless dependency. The raw API lets you push the vectors in directly, and the separate namespace keeps that data away from the collections the server manages itself.

The mention of consistency limits in the guide is the part to notice. A store where the server computes embeddings can guarantee that the vector for a document matches the model it will query with, because there is only one model. A store where the caller supplies vectors cannot check that, and the caller has to be responsible for not mixing vectors from different models in one namespace. That is a real footgun, and the fact that the documentation has a section for it is a good sign.

So the two APIs split along a sensible line: the default path for people who want the server to own the whole pipeline, and the raw path for people who already have a model and want control.

Building it and the file formats it handles

The quick start is three commands, and the prerequisites are a Go toolchain and optionally Docker.

git clone https://github.com/mudler/LocalRecall.git
cd LocalRecall

go build -o localrecall

./localrecall

After that the web interface is at localhost on port 8080.

The dependencies in the module file describe the feature set. The web framework is Echo. The Chromem library is the file-based vector store, and the PostgreSQL driver is pgx. The OpenAI client is there because the embedding model is reached through an OpenAI compatible API, which is how it talks to LocalAI. PDF handling goes through a PDFium binding, and an HTML to text converter handles pages that are HTML rather than PDF. A sitemap parser is present, and Git is implemented as a Go library, which suggests the service can ingest content straight from a repository rather than only from uploaded files.

Tests are written with Ginkgo and Gomega rather than the standard library, and the module requires Go 1.25.

For containerised deployment there is a prebuilt image published to a container registry, and the compose route is one command:

docker compose up -d

That starts LocalAI for embeddings on one port, PostgreSQL with the extensions on another, and the RAG server on the third, and the readme explains which of the three is which. The compose file also shows the configuration through environment variables, which is the same set the documentation describes: the collection database path, the embedding model, the file assets directory, the database URL, the vector engine selection, the OpenAI key and base URL, and the two hybrid search weights.

The Makefile documents a race worth knowing about

The build file is ordinary until you read the comment next to the embedding model, and that comment is the most useful line in the repository.

The Makefile pins the embedding model the test suites exercise, and explains that the compose file starts LocalAI with that model but it is downloaded or loaded lazily on the first request. So the test harness has to wait for the model to actually answer before running the specifications, and the file points at a wait target for exactly that reason. Otherwise, the first embedding call races the download and the whole suite fails with a model not found error.

That is a specific, real failure mode, documented in the build file where the person hitting it will find it.

The rest of the Makefile is a well organised test setup. There are targets for the unit, integration, end to end and coverage runs, and a combined target for all of them, plus a target to start the LocalAI service, one to stop it and one to wait for it. There is a separate target for a raw image test, and the version is stamped into the binary at build time with a linker flag pointing at a versioning variable in the internal package, with the version string derived from the most recent tag.

The CGO environment variable is disabled by default, which is what lets the final image be a scratch base with nothing in it but the binary and the certificate store.

Where the project stands

Three things to check before you commit to it.

The first is the version and the pace. The releases are on the 0.6 line, with 0.6.2 in May 2026, 0.6.3 in June and 0.6.4 in July, and the last push was on 2026-10-12 with the repository not archived. The project is moving and it is pre-1.0, which usually means the API and the configuration surface are still moving too.

The second is that the memory layer is only half of the system. LocalRecall does the retrieval, but something has to produce the embeddings, and the documented setup points that at LocalAI with a multilingual Granite embedding model. If you want a genuinely self-contained deployment, you are deploying two services, and the compose file is the reference for how they are wired.

The third is the honesty about the no-frills part. The readme says lightweight and no-frills, and the feature list is short. There is no mention of authentication on the API, and the web interface is described as a convenience for file management rather than as a hardened admin console. For a service bound to a local network in a trusted home or lab, that is a reasonable trade. For anything exposed beyond that, you would be adding a layer that the project does not provide.

The licence is MIT, the language is Go, and the family of local tools around it is the larger story: a local model server, a local agent platform, and now a local memory, all speaking the OpenAI API shape so they can be swapped for the hosted services without changing the client.

Editorial conclusion

LocalRecall suits a team that wants retrieval for their agent to stay on their own hardware, and its Chromem backend makes a first run almost free of setup. It is a poor fit if you need a hosted service with guarantees, and a poor fit if your data is large enough that the file-based store becomes the bottleneck, since that is the moment you need the PostgreSQL path and its extensions. Before you deploy it, decide which backend you want, since the raw vector API exists only on PostgreSQL, and read the Makefile note about the embedding model being downloaded lazily, because that race is the thing most likely to make your first test run fail.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. mudler/LocalRecall on GitHub
4. README
5. Releases