AnythingLLM: a self-hosted RAG workspace you install with Docker

The problem AnythingLLM actually solves

Most retrieval-augmented generation stacks start the same way: pick a document loader, pick a chunker, pick an embedding model, stand up a vector database, write the retrieval loop, then build a UI on top. Each of those choices is a separate dependency with its own upgrade path. AnythingLLM collapses that chain into one application. The README describes it as an all-in-one AI app with built-in agents, multi-user support, vector databases and document pipelines, with no extra configuration required.

The audience is narrower than the marketing suggests. This is for teams that have documents they are not willing to send to a hosted chat product, and that want a working chat interface over those documents without writing retrieval code. It is also for individuals who want the same thing on a laptop. The desktop builds for Mac, Windows and Linux exist for exactly that case. If you are building a product and need retrieval as a component inside your own service, AnythingLLM is the wrong shape: you would be adopting an application and then working around it.

How the workspace, collector and vector store fit together

The repository layout tells you most of the architecture before you read any documentation. There are three top-level application directories: server, frontend and collector. The root package.json defines a setup script that installs dependencies in all three and copies environment files, and separate dev scripts for each, plus a combined dev script that runs them together with npx concurrently. That split is the data flow.

The collector is the document pipeline. It is a separate process, which means ingestion is decoupled from serving chat requests: you can restart the API without losing a running document job, and a large PDF import does not block the chat UI. The server holds the API, the agent runtime and the database layer, and the repository ships Prisma migrations and a seed script under the server directory, so the relational schema is managed rather than hand-rolled. The frontend is the chat interface, the workspace administration screens and the embeddable widget.

The retrieval layer is configurable rather than fixed. The README lists embedder options including an AnythingLLM native embedder as the default, along with OpenAI, Azure OpenAI, Gemini, LocalAI, Ollama, LM Studio, Cohere, Voyage AI, Mistral and generic OpenAI-compatible embedding APIs. The LLM list is much longer, covering local runtimes such as llama.cpp builds, Ollama, LM Studio, LocalAI and KoboldCPP alongside hosted providers including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini, Groq, Mistral, DeepSeek, Cohere and OpenRouter. The practical consequence is that you can run the whole stack offline with a local model server, or mix a local embedder with a hosted chat model, without changing the application.

Installing AnythingLLM with Docker and running a first document chat

The project publishes a desktop build for Mac, Windows and Linux, and a Docker deployment for servers. The multi-user instance support and the embeddable chat widget are both marked Docker version only in the README, so if either matters to you, the desktop build is not the path. For a server install, clone the repository and use the docker directory, which contains an .env.example that the setup script copies into place.

git clone https://github.com/Mintplex-Labs/anything-llm.git
cd anything-llm
cp -n ./docker/.env.example ./docker/.env

The .env file is where you set the storage location and any provider credentials the container needs. The README does not reproduce every key, so read the example file rather than guessing at variable names. The Docker deployment is documented in the docker directory of the repository and on the project documentation site at docs.anythingllm.com.

If you want to run from source instead, the root package.json defines the whole sequence. Node 18 or newer is required by the engines field.

yarn setup
yarn dev:server
yarn dev:collector
yarn dev:frontend

The setup script installs dependencies in server, collector and frontend, copies the environment files, and runs the Prisma setup. It then prints an instruction to run the three dev commands in separate terminal tabs, which is why they are listed individually above rather than as a single command.

Once the interface is up, the first real use is short. Create a workspace, open its document upload panel, and drag a PDF or DOCX into it. The collector parses and embeds the file, and the README states that the chat UI supports drag-and-drop uploads with source citations, so answers should carry references back to the chunks they came from. Then ask a question whose answer is only in that document. If the answer cites the file, ingestion and retrieval are both working. If it does not, the problem is almost always the embedder or the vector store configuration, not the chat model.

Where AnythingLLM gets in your way

The most concrete limitation is stated in the README itself: multi-user support and the embeddable chat widget are Docker-only. Anyone who installs the desktop build expecting to hand out accounts will not find them. That is a real fork in the deployment decision, and it is easy to miss because the feature list reads as one product.

The second constraint is the vector store. The README lists supported LLMs, embedders, speech models and vector databases, but the excerpt available here is truncated before the vector database section. If you already run a specific vector database in production, check the supported list before you plan a migration, because AnythingLLM manages the store on your behalf rather than acting as a client to an arbitrary one.

The third is scope. AnythingLLM is an application with an opinionated data model: workspaces, documents, threads, users. The README advertises a full developer API for custom integrations, and that is the right escape hatch for automation. But if your requirement is a retrieval function you call from your own backend, you are paying for a UI, an agent runtime and a permission system you will not use. A library such as a standalone vector store client plus your own embedding code is less code overall in that scenario, not more.

AnythingLLM compared with LM Studio and Open WebUI

The two comparisons people search for most are LM Studio and Open WebUI, and they solve different problems.

LM Studio is a local model runner with a chat interface. Its job is to download and serve models on your machine. AnythingLLM lists LM Studio as a supported LLM provider, which tells you the relationship: they are complementary, not competing. If your goal is to run a model locally and talk to it, LM Studio alone is simpler. If your goal is to run a model locally and talk to your documents with citations, AnythingLLM is the layer that adds ingestion, embedding and retrieval on top of whatever serves the model.

Open WebUI is closer in ambition. It is also a self-hosted chat front end with document handling and multi-user support. The meaningful difference visible here is breadth of the ingestion and provider configuration rather than the chat surface. AnythingLLM's README emphasizes the document pipeline, the agent builder, scheduled tasks, memories, MCP compatibility and dynamic model routing as first-class features, and it ships a separate collector process for ingestion. If you already run Open WebUI and it meets your retrieval needs, switching buys you those features, not a different category of tool. Evaluate on the specific feature you are missing rather than on the general shape.

Release cadence, licence and upgrade cost

The repository is not archived, and the most recent push recorded is 2026-08-27, which corresponds to the v1.16.1 release. The release list shows v1.16.0 on 2026-08-13 and v1.15.0 on 2026-06-25. That is a steady stream of minor releases, and it cuts both ways: you get fixes and new provider integrations, and you inherit a moving target. Pinning a version and reading the release notes before upgrading is the sane default for a self-hosted deployment that other people depend on.

The licence is MIT, which is permissive and places few obligations on how you deploy or modify the software. Two things are worth separating from the licence. First, the repository also contains a TERMS_SELF_HOSTED.md file, which is a separate document from the licence and governs use of the self-hosted product. Read it rather than assuming the MIT grant covers everything in the repository. Second, the models you connect are governed by their own terms: running a hosted provider through AnythingLLM does not change that provider's data handling, and a local model's licence is a separate question from AnythingLLM's. None of this is legal advice; if the distinction between the MIT licence and the self-hosted terms matters to your organisation, have someone read both.

Editorial conclusion

Adopt AnythingLLM if you need a private document chat workspace with agents and per-user permissions, and you are willing to run the Docker image rather than a library. Skip it if you want to embed retrieval inside your own service, since the useful surface here is an application, not a package. Before committing, verify that the multi-user and embed-widget features are available in the build you plan to run, that your chosen vector database and embedder are in the supported list, and that your team is comfortable with the release cadence the repository shows.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes