PrivateGPT: An API Layer for Local AI That Skips the Model Runtime
Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.
At a glance
- What is it?
- PrivateGPT is an open-source API server that turns any OpenAI-compatible inference engine into a Claude-style application backend. It adds RAG, tools, MCP, and database access without running models itself.
- Who is it for?
- Adopt PrivateGPT if you already run an OpenAI-compatible inference server and want a ready-made application API with RAG, tools, and MCP, without building those primitives yourself. Skip it if you need prompt caching, OAuth, or organizations, since the compatibility table marks those as unsupported.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What PrivateGPT Actually Sits Between
PrivateGPT is not another model runner. The README states it does not run models itself. It connects to any OpenAI-compatible inference server through OPENAI_API_BASE, and if that server implements /v1/chat/completions and /v1/models, it works. The architecture diagram shows your app, agent, or workflow on top, PrivateGPT in the middle, and the inference server below. That position matters. Most local AI projects bundle a runtime, but PrivateGPT deliberately leaves that out. The project targets developers who already have Ollama, llama.cpp, or vLLM running and need the layer above it: a standard messages API, file ingestion, retrieval with citations, tools, and database access. The built-in workbench UI at /ui is for demos and testing, not for end users. The README calls the API the actual product.
The Claude API Compatibility Table Is the Real Spec
PrivateGPT follows the Claude API as its reference model. The compatibility table in the README is the clearest statement of scope. It lists supported capabilities side by side: messages, streaming, async batch processing, token counting, file and artifact ingestion, retrieval with citations, tool use, built-in web search, web fetch, custom tools, database querying, CSV analysis, MCP in the API, remote MCP servers, structured outputs, vision, and extended thinking. Two notable gaps stand out: prompt caching is marked as not supported, and OAuth or organizations are also not supported. Token-based auth is supported. That table tells you exactly what you get and what you lose. If your application depends on prompt caching to cut costs, PrivateGPT will not help. If you need multi-tenant OAuth, you must build that yourself.
Installation Paths and the Two Environment Variables
The quickstart gives three installation routes. On macOS you use Homebrew: brew tap zylon-ai/tap and brew install private-gpt. On Linux and Windows the project uses uv, the Python package manager, with a custom wheel index at wheels.privategpt.dev. The Linux command is uv tool install --python 3.11 --find-links https://wheels.privategpt.dev/packages/ "private-gpt[core]". Windows uses the same command with PowerShell syntax. After installation you need a running LLM server. The README example pulls qwen3.5:35b for the LLM and mxbai-embed-large for embeddings, then runs ollama serve. To start PrivateGPT you set two environment variables: OPENAI_API_BASE pointing to the LLM server's /v1 endpoint, and OPENAI_EMBEDDING_API_BASE pointing to the embedding server's /v1 endpoint. Then run private-gpt serve. The API listens on port 8080, and the UI is at /ui.
What the Built-in Tools and Data Access Cover
PrivateGPT ships with tools that mirror the Claude API: web search, web fetch, and code execution. It also has built-in database querying and CSV analysis, which the README positions as a differentiator. The compatibility table marks database querying as built-in in PrivateGPT, whereas Claude does it via tools. CSV and tabular analysis are likewise built-in. That means you can point PrivateGPT at a database or a CSV file and query it through the messages API without writing a custom tool. The UI lets you configure databases, MCP connectors, skills, and custom tools per chat. Skills support is marked as basic, not full. So if you need sophisticated skill orchestration, expect to do more work. The tool system is not a toy: the README lists custom tools and MCP connectors as first-class features, and remote MCP servers are supported.
The Workbench UI Is a Demonstrator, Not the Product
The README is explicit about the UI's role. It is a demonstrator, not the core product. Developers are expected to build their own applications on top of the API. That said, the UI is polished enough for demos, videos, internal pilots, and quick local usage. It lets you send messages, select models from /v1/models, upload documents, test retrieval with citations, enable tools per chat, and inspect requests through an API Debugger. That debugger is a practical feature for development. The distinction matters for adoption. If you are evaluating PrivateGPT for a production system, the UI is not the deliverable. The API is. The UI is a convenience for testing the API's behavior, not a replacement for your own frontend.
Integration Reach Beyond the API
PrivateGPT positions itself as a drop-in backend for existing tools. The README lists integration guides for Claude Code, Claude Desktop and Cowork, Claude for Microsoft 365, and OpenCode. Any tool that works with a local OpenAI-compatible provider should also work with PrivateGPT. The README names n8n, OpenClaw, Hermes Agent, VS Code, and Cline as examples. That is a strong claim, but the mechanism is straightforward: since PrivateGPT exposes an OpenAI-compatible endpoint, clients that already speak that protocol can point at it. The Claude Code integration is interesting because it lets you use local models as the backend for agentic coding in the terminal. The breadth of integrations is a practical advantage, but it depends on the compatibility of each tool with PrivateGPT's specific implementation of the messages API. The README says the list is non-exhaustive, so you should test your own tool against it.
Licence, Maintenance, and the Zylon Connection
PrivateGPT is licensed under Apache-2.0, which permits commercial use, modification, and distribution with attribution. The project is not archived, and the latest release is v1.0.1 from June 2026, with v1.0.0 released two weeks earlier. That recent activity suggests an active maintenance cycle. The README notes that PrivateGPT powers Zylon, an on-premise AI platform for enterprises. That relationship cuts both ways. Zylon has a commercial interest in keeping PrivateGPT solid, but it also means the open-source project may prioritize features that serve Zylon's enterprise customers. The skills support is marked as basic, which signals where development effort is not concentrated. If you adopt PrivateGPT, you inherit that roadmap. The Apache-2.0 licence gives you the right to fork and modify, but you are responsible for tracking upstream changes and applying security patches yourself.
Editorial conclusion
Adopt PrivateGPT if you already run an OpenAI-compatible inference server and want a ready-made application API with RAG, tools, and MCP, without building those primitives yourself. Skip it if you need prompt caching, OAuth, or organizations, since the compatibility table marks those as unsupported. Before committing, verify that your inference server implements /v1/chat/completions and /v1/models, and check whether the basic skills support meets your agentic needs. The project is Apache-2.0, so you can fork it, but the maintenance burden falls on you if Zylon shifts focus to its commercial product.
Community notes