AI LocalBase: a single-instance RAG stack in Go, Qdrant and SQLite

The problem AI LocalBase picks, and who it is built for

Most retrieval-augmented generation demos assume you will rent a vector service, ship documents to a hosted embedding API, and keep the conversation history in someone else's database. AI LocalBase goes the other way. It is a local-first knowledge base: you upload files, it chunks and embeds them into a Qdrant instance running next to the app, and chat sessions land in a local SQLite file. The README lists the target audience plainly, as personal or small-team use in a local or self-hosted environment, and the use cases it names are personal knowledge bases, internal team document Q&A, self-hosted RAG prototyping and retrieval-strategy experiments.

The document types it accepts are TXT, Markdown, PDF (text), xlsx and csv. Note the qualifier on PDF: the README says PDF, and the description of the project narrows that to text PDFs, so scanned pages are not the target. That is a real boundary rather than a bug, and it decides a lot about who this fits. If your corpus is a folder of contracts that were scanned to image, this is the wrong starting point.

The second audience is agents. AI LocalBase embeds an MCP server, so an external tool such as Cherry Studio, Claude Desktop or Cursor can call into the same knowledge base and session services the web UI uses. That is the part that makes the project more than a chat front end: the retrieval pipeline is exposed as callable tools, not only as a page you click.

How the pieces fit: Go backend, Qdrant, SQLite, JSON state

The architecture is stated in the README without much ambiguity. The backend is Go with Gin. The front end is React, Vite and TypeScript. Qdrant is the vector store. Model access goes through Ollama or any OpenAI-compatible API. Deployment is Docker Compose, with a single-process local run as a supplement. The repository also carries docs/architecture.md, which is where the full picture lives; the README only sketches it.

State is split three ways, and that split is the source of most of the operational constraints. Uploaded files live in a data directory controlled by .env. Application state and model configuration are written to local JSON files. Chat messages go to a local SQLite database. Vectors go to Qdrant, with its own persistence directory. Four stores, one host, no shared coordinator.

The retrieval side is more elaborate than a naive embed-and-cosine loop. The README lists automatic text splitting and batch embedding, dynamic candidate recall, a keyword-coverage reranking pass, MMR selection for de-duplication, and a second, wider recall when confidence is low. On top of that there are optional stages: hybrid search, a semantic reranker, query rewrite and context compression. Embedding caching is present, with an optional semantic cache. The project can also generate a RAG evaluation dataset from documents already in a knowledge base, which is the honest way to check whether any of those optional stages help on your corpus rather than someone else's.

Two details in that list are worth flagging as design choices rather than features. MMR de-duplication and keyword-coverage reranking are heuristics that trade recall for diversity and lexical overlap. They will help on repetitive document sets and can hurt on corpora where the answer is deliberately phrased differently from the question. The evaluation-dataset generator exists precisely because that trade-off is corpus-specific.

Getting it running: the commands and the config keys that matter

The shortest path in the README is two commands. Copy the environment template with cp .env.example .env, then run docker compose up --build. The front end answers on http://localhost:4173, the backend on http://localhost:8080, Qdrant's HTTP API on http://localhost:6333 and its gRPC port on 6334. Model configuration happens in the Settings page of the web UI, not in the compose file.

For a prebuilt image, the README gives AI_LOCALBASE_IMAGE_TAG=v1.4.6 docker compose -f docker-compose.prod.yml up -d. There are two other compose files: docker-compose.app.yml for the application orchestration alone, and docker-compose.dev.yml, which mounts local source as volumes and is described as the recommended path for daily development and interface debugging.

Several environment keys carry more weight than their names suggest. QDRANT_VECTOR_SIZE must match your embedding model's dimension. If you switch to a model with a different dimension, the README says to use a new QDRANT_COLLECTION_PREFIX or rebuild the old collection. The same advice applies before enabling ENABLE_HYBRID_SEARCH, because Qdrant then needs named dense and sparse vectors in the collection.

Authentication is on by default in the normal and production compose files. ENABLE_AUTH=true turns on web login and API key checks; AUTH_USERNAME defaults to root; AUTH_PASSWORD creates the root user on first start; and AUTH_SETUP_TOKEN is the alternative guard if you do not set a password. The README warns that with ENABLE_AUTH=true and no AUTH_PASSWORD, the first visit to the web page opens an initialization wizard, so a server deployment should set one of the two values before anyone else reaches the port. Development compose is the exception: it needs ENABLE_AUTH=false set explicitly to skip login.

Upload limits come in pairs. MAX_UPLOAD_BYTES defaults to 26214400, which is 25 MiB, and NGINX_CLIENT_MAX_BODY_SIZE defaults to 32m. The proxy limit has to stay above the file limit to absorb multipart overhead. If you raise one and forget the other, uploads fail at the proxy with no useful error from the backend.

Qdrant binding is the other pair to get right. QDRANT_BIND_ADDRESS defaults to 127.0.0.1, so the vector store is reachable only from the host. Changing it to 0.0.0.0 requires QDRANT_API_KEY and firewall rules; the README states the Qdrant container refuses to start on a non-loopback address without an API key. Similarly, BACKEND_BIND_ADDRESS keeps the backend on the host by default. If a controlled HTTPS reverse proxy sits in front, set TRUST_EXTERNAL_PROXY_HEADERS=true and keep X-Forwarded-Proto and X-Forwarded-Host intact; leave it false when the front end port is exposed directly.

The MCP server, and the transport detail that breaks clients

MCP is off by default. ENABLE_MCP defaults to false, and the README says to enable it only after ENABLE_AUTH=true on a server, because the MCP surface exposes knowledge base and session capabilities to outside agents. The mount point is MCP_BASE_PATH, default /mcp, with GET /mcp, GET /mcp/tools and POST /mcp as the entry points. Rate limiting sits at MCP_REQUESTS_PER_MINUTE, default 120, and MCP_REQUEST_TIMEOUT_SECONDS defaults to 15.

Authorization uses API keys carrying mcp:* scopes. The README recommends mcp:read, mcp:upload and mcp:eval, adding mcp:danger only when delete-class tools are needed. Tools are graded read-only, write, or dangerous, and dangerous tools require a one-time confirmation. There is an audit log. The older MCP token scheme is deprecated and rejected by default; ENABLE_MCP_LEGACY_TOKEN=true is a migration-only switch, and the README notes that the legacy token is equivalent to full MCP permissions.

The transport is where I would expect the most support traffic. The README describes the implementation as a JSON-response subset of Streamable HTTP. It does not issue an MCP session ID and does not provide SSE long connections. Clients must accept application/json; declaring only text/event-stream will not work. The Cherry Studio example in the README sends Accept: application/json, text/event-stream and points at http://127.0.0.1:8080/mcp with a bearer token. The Docker front-end proxy handles /mcp and nested paths ending in /mcp, such as /agent/mcp. Any other path needs your own reverse proxy rule or a client pointed straight at the backend port.

The tool list is broader than a search endpoint. It includes a capability self-check tool called get_mcp_capabilities, document detail, retrieval debugging, structured queries, evaluation-set generation and index rebuild. There is an asynchronous job workflow for imports with status polling, cancellation and a recent-task list, plus composite tools for answering with sources, knowledge base quality checks, retrieval-mode comparison, evaluation sample creation and document summarization. The composite tools are the interesting part: they encode a retrieval workflow rather than a single primitive, which means an agent can ask a question and get citations without orchestrating four calls itself.

Single instance is a design decision, not a missing feature

The README states it directly: the application layer is designed as a single instance. Local SQLite chat records, the application state file and in-memory MCP jobs do not support shared writes from multiple backend replicas. It goes further and names the command not to run: docker compose scale backend=2. Production compose pins a released image version, and upgrades or rollbacks go through AI_LOCALBASE_IMAGE_TAG explicitly, with a note not to depend on latest.

That constraint ripples outward. MCP jobs live in memory, so a backend restart loses in-flight import state; the README does not describe a durable job queue, and I would not assume one exists. Chat history in SQLite means you cannot put two backends behind a load balancer and expect consistent sessions. The JSON state file means configuration writes are not coordinated across processes. None of this is a defect for the stated audience of one operator or a small team, but it is a hard ceiling, and the README is unusually candid about placing it there.

The resource and process hygiene around that ceiling is more careful than the average side project. Production compose configures container restart policies, log rotation and resource limits. The backend runs as a non-root user and, on first start, corrects ownership of the persistent data directory and writes a migration marker. Backups are documented in docs/backup-restore.md, and the README advises backing up the upload directory, application state, the chat SQLite file and the Qdrant persistence directory before an upgrade or migration. Four paths, not one, because state is split four ways.

The other limitation is corpus size. Everything runs on one host: Qdrant, the backend, the model endpoints if you self-host Ollama, and the uploaded files. There is no sharding story in the README and no mention of a remote Qdrant cluster. If your document set outgrows the machine's disk or RAM, the answer is a different architecture, not a configuration change.

Where it sits against AnythingLLM and Dify

The closest comparison in this space is AnythingLLM, which also targets local and self-hosted RAG with a web UI and multiple vector store backends. The difference in approach is the state model. AnythingLLM is built around a workspace abstraction and supports several vector databases and deployment shapes, with a desktop build as a first-class option. AI LocalBase commits to one vector store, Qdrant, and to a single-instance backend with SQLite and JSON state. That is a narrower bet, and it buys a smaller surface to reason about: one vector database to back up, one place chat history lives, one configuration file format.

Dify is the other reference point, and it is a different category of tool. It is a platform for building LLM applications with visual workflows, multiple model providers and a broader orchestration layer. AI LocalBase does not try to be that. Its retrieval pipeline is opinionated and fixed: chunking, embedding, dynamic recall, keyword-coverage reranking, MMR, low-confidence re-recall, with optional hybrid search, semantic reranking, query rewrite and compression. You tune parameters and enable stages; you do not draw a graph.

The distinguishing feature against both is the embedded MCP server with scope-graded tools and an asynchronous job workflow. AnythingLLM and Dify both expose APIs, but the README describes AI LocalBase's MCP surface as reusing the same knowledge base, session, configuration and retrieval services as the web UI, with a tool list that includes evaluation-set generation and retrieval-mode comparison. For an agent-first workflow, that is a specific reason to pick this project over a general chat platform. For a human-first workflow with a large team, it is not.

Licence, upgrades and the cost of keeping it current

The licence is MIT, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is the whole of what the repository metadata tells you. I am not a lawyer and this is not legal advice; if you redistribute AI LocalBase inside a product, read the LICENSE file in the repository and confirm the notice requirements against your own distribution model.

The upgrade cost is where the four-store split shows up again. Releases are frequent: v1.4.10 on 2026-09-02, v1.4.9 the day before, v1.4.7 on 2026-08-23. That cadence means you should pin AI_LOCALBASE_IMAGE_TAG rather than track latest, which the README already instructs. Two upgrade paths carry extra work. Changing embedding model dimension requires a new QDRANT_COLLECTION_PREFIX or a collection rebuild, because QDRANT_VECTOR_SIZE is fixed per collection. Turning on ENABLE_HYBRID_SEARCH also wants a new prefix and a reindex, since Qdrant then needs named dense and sparse vectors. Both are reindex operations over your whole corpus, so their cost scales with document count and embedding throughput, not with the size of the release.

Backup before upgrade is not optional here, and the README names the paths: uploads, application state, the chat SQLite database and the Qdrant persistence directory. The backend's first-start routine corrects directory ownership and writes a migration marker, which is the kind of step that is invisible when it works and awkward when it does not. If you run the production compose file, the notes in DOCKER_DEPLOY.md cover the resource variables and image tagging; TROUBLESHOOTING.md is the other file to read before filing anything. The retrieval-improvement plan is explicitly not in Git, so do not expect to read it from a clone.

Editorial conclusion

Adopt AI LocalBase if you are one person or a small team indexing your own documents on a single host, and you want Qdrant plus Ollama or an OpenAI-compatible endpoint behind a web UI and an MCP server you can point an agent at. Do not adopt it if you need horizontal scale, multi-tenant isolation, or a knowledge base larger than one machine's disk and RAM can hold: the README states the application layer is designed as a single instance, and that SQLite chat records, the application state file and in-memory MCP jobs do not support shared writes across replicas. Before you commit, verify three things against your own environment: that your embedding model's dimension matches QDRANT_VECTOR_SIZE, that you have set AUTH_PASSWORD or AUTH_SETUP_TOKEN before the first boot so nobody else claims the initialization window, and that your upgrade path pins AI_LOCALBASE_IMAGE_TAG to a release tag rather than latest.

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. veyliss/ai-localbase on GitHub