RESTai: A Self-Hosted AIaaS Platform for Teams That Need a REST API Over Their Models

The gap RESTai fills between a model endpoint and a usable internal product

A raw inference endpoint answers one question. Most teams quickly need a second question answered: which project, which prompt version, which documents, which token budget, and which person is allowed to call it. RESTai positions itself as an AIaaS layer that answers that second question. The README describes it as a platform where you create AI projects and consume them through a simple REST API, with RAG, agents, a block-based graphical language, and inference all in one place.

The intended user is not an individual developer prototyping a chatbot. It is a platform or infrastructure team that has been asked to give several internal groups access to LLMs without letting each group stand up its own vector store, its own key management, and its own usage reporting. The feature list points that way: teams, RBAC, OAuth and LDAP, TOTP two-factor authentication, token tracking, per-project rate limiting, and white-label branding. Those are the concerns of a service owner, not of a notebook author.

The project is written in Python, targets Python 3.11 and above, and is licensed under Apache-2.0. The most recent release listed is v6.4.1, published in August 2026, with v6.4.0 and v6.3.28 before it. That version numbering is worth reading carefully: a major version of six, with point releases arriving weeks apart, tells you the maintainers treat the API surface as something that still moves.

How a request flows through projects, LLM backends and vector stores

The organising unit in RESTai is the project. According to the README, each project carries its own LLM, system prompt, tools and configuration, and can be tested in a built-in chat playground before it is exposed. That means the platform is not a single global model configuration with per-caller overrides; the project is the boundary that holds prompt, model choice, retrieval settings and access rules together.

On the model side, the README lists OpenAI, Anthropic, Ollama, Gemini, LiteLLM, vLLM and Azure among the supported backends. The topics attached to the repository include ollama, openai, llamadb, langchain, llamaindex and transformers, which matches a design where the platform sits above several client libraries rather than committing to one vendor. For local inference the practical path is Ollama or vLLM behind the project configuration.

Retrieval is where the design gets more opinionated. RAG projects support multiple vector stores, reranking through ColBERT or an LLM-based reranker, a sandboxed mode described as reducing hallucination, and evaluation via deepeval. There is also an optional path that connects a MySQL or PostgreSQL database and translates natural language questions into SQL. A separate, opt-in Knowledge Graph layer runs ingested documents through a NER pipeline, with dslim/bert-base-NER as the default model, and persists extracted people, organisations and locations into a graph alongside the vector store. Entity extraction runs as a background task so that ingestion is not blocked, and a custom postprocessor boosts retrieved chunks whose source documents mention entities found in the query. That is a concrete mechanism, not a marketing line: the graph changes which chunks are ranked higher, rather than merely being displayed next to them.

Agents are built as zero-shot ReAct agents with built-in tools, and can connect to MCP servers over HTTP/SSE or stdio. The README also mentions an agentic browser that gives agents a headless Chromium instance, though the supplied material cuts off mid-sentence at that point, so the details of how browser sessions are isolated are not something I can confirm.

Installing RESTai from PyPI, source or Docker

There are three documented installation routes, and they differ in how much of the stack you take responsibility for.

The PyPI route installs a package named restai-core and then runs three commands in sequence:

pip install restai-core restai init restai migrate restai serve

The README states that the published package includes a pre-built React frontend, so Node.js is not required on the host. The server comes up on port 9000 by default, with the admin interface at /admin and default credentials of admin / admin. You can pass an environment file and adjust the bind port and worker count:

restai serve -e .env -p 8080 -w 4

Running from source uses a Makefile instead. The README gives git clone followed by make install and make dev, which also lands on port 9000. Docker is the shortest path: docker run -p 9000:9000 apocas/restai:latest, with the same default credentials. The image is published as multi-architecture for linux/amd64 and linux/arm64, and also to GitHub Container Registry as ghcr.io/apocas/restai:latest. The README explicitly suggests pinning a version tag such as :6.2.13, or the shorter :6.2 and :6, instead of :latest, which is sensible advice for anything you intend to run in production. A bundled compose stack is available through docker compose --env-file .env up --build.

The default credentials are the detail that matters most here. admin / admin on a service that holds API keys for paid model providers is a configuration you must change before the port is reachable from anywhere but localhost, and the README presents it as the normal first-run state.

Upgrades, migrations and what the release cadence costs you

RESTai does not pretend upgrades are free, and the documentation is unusually direct about the mechanics. For a PyPI install, the update procedure is pip install --upgrade restai-core followed by restai migrate -e .env. For a source checkout, it is make update, which the README says fetches the latest release tag from GitHub, installs dependencies, runs database migrations, and rebuilds the frontend, auto-detecting GPU to select GPU-specific dependencies.

Two implications follow. First, there is a database with a schema that changes, and migrations are a step you must run rather than something the server handles silently at startup. If you skip the migrate step, the failure mode is a version mismatch between code and schema, which is exactly the class of problem that surfaces in production rather than in a test. Second, the auto-detection of GPU dependencies in make update means the dependency set is not identical across machines. A host without a GPU and a host with one will resolve different packages, so an upgrade validated on a developer laptop is not automatically validated on the deployment target.

The release history supplied shows v6.4.1 in August 2026, v6.4.0 earlier the same month, and v6.3.28 in June 2026. That is a fast cadence for a platform that owns your database schema. Pinning a version tag, as the README suggests for Docker, is the natural response, and it means you will periodically need to plan a migration window rather than tracking the tip of master.

The licence is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant. That covers the RESTai source itself. It does not automatically cover the models you point it at, the vector store you choose, or the frontend dependencies bundled into the published package. Those carry their own terms, and the repository material does not enumerate them, so this is something to check rather than assume.

Where RESTai is the wrong tool

The clearest limitation is scope. RESTai is a platform, and platforms impose structure. If your actual need is to embed retrieval into an existing Python service, adopting a separate HTTP service with its own database, its own admin user, its own migration command and its own frontend means you now operate two systems where one would do. The project's own framing supports this reading: it is an AIaaS product with a dashboard, not a library you import.

The second limitation is the coupling between releases and your database. Because migrations are explicit and the version cadence is fast, the cost of staying current is a recurring operational task, not a one-time setup. Teams that cannot schedule migration windows should pin aggressively and accept falling behind on fixes.

The third is the Knowledge Graph layer, which is opt-in and available exclusively for RAG projects. Enabling it on a populated project requires a rebuild, which the README describes as re-extracting the entire graph from existing sources. That rebuild is a batch job over your whole corpus, and its duration scales with document count rather than with query volume. Entity extraction also depends on a specific default NER model, dslim/bert-base-NER, and the material does not describe how to substitute a domain-specific model. For corpora full of product codes, part numbers or internal jargon, a general-purpose NER model is likely to produce entities you then have to clean up through the merge and disambiguation tools.

Finally, the README's own feature list is broad enough that depth is a fair question. Token tracking, cost analytics, latency charts, OAuth, LDAP, TOTP, white-labelling, MCP, block-based logic, image generation, audio generation and SQL translation are a lot of surface area for one project to maintain well. The material does not let me assess how evenly that attention is distributed, and I would not assume uniform maturity across all of it.

How RESTai differs from assembling LangChain or LlamaIndex yourself

The obvious alternative is not a competing product but a stack you build: LangChain or LlamaIndex for orchestration, a vector store client, FastAPI for the HTTP layer, and your own tables for users, keys and usage. Both libraries appear in the repository topics, which is a fair signal that RESTai itself is built on top of them rather than replacing them.

The difference is where the work sits. With the library approach, you write the project abstraction, the prompt versioning, the token accounting, the rate limiter, the RBAC checks and the admin interface. You get exactly the semantics you want, and you own every line. With RESTai, that layer already exists and is exposed over HTTP, but its semantics are the project model the maintainers chose. If your tenancy model does not map onto a project, or your prompt versioning needs differ from what the platform offers, you are adapting your requirements to the tool rather than the reverse.

A second alternative worth naming is a managed inference gateway. Those remove the operational burden of running the service and its database, at the cost of sending prompts and documents to a third party and paying per call. RESTai's selling point in that comparison is locality: with Ollama or vLLM behind a project, prompts and embeddings stay on infrastructure you control. That matters for regulated data, and it is also the reason the operational burden lands on you.

The honest summary is that RESTai trades flexibility for a working starting point. You give up control over the internal abstractions and take on a service to operate. In exchange you skip building the dashboard, the RBAC, the token accounting and the chat playground, which is a substantial amount of undifferentiated work.

What to verify before you commit to RESTai

Start with the backends. Confirm that your intended LLM provider and vector store appear in the project's supported list, and that the reranking option you want (ColBERT or LLM-based) is available for the store you pick. The README lists categories of providers rather than a compatibility matrix, so the specific combination you need is worth testing on one project before you plan a rollout.

Next, run the migration path against a copy of real data. The commands are restai init, restai migrate and restai serve for a fresh install, and pip install --upgrade restai-core plus restai migrate -e .env for an upgrade. Exercising the upgrade path on a database snapshot before you need it is the only way to know how long a maintenance window will actually take.

Then check the licence surface beyond Apache-2.0. The RESTai source is Apache-2.0, but the models you serve, the vector store you run, and the frontend dependencies shipped in the published package are separate. The repository material does not list them, so this is a review task rather than a lookup.

Finally, decide your pinning policy. The README already recommends pinning Docker tags rather than using :latest, and the same reasoning applies to restai-core versions. Given the gap between v6.3.28 and v6.4.1, a policy of pinning and upgrading deliberately is more realistic than tracking releases as they land. The live demo at ai.restai.cloud uses a restricted account that can browse and chat but cannot create or modify projects, which is enough to judge the interface, and not enough to judge the operational behaviour you will actually be responsible for.

Editorial conclusion

Adopt RESTai if you need a self-hosted, multi-tenant layer that turns several LLMs, vector stores and tools into per-project REST endpoints with a dashboard, and you are willing to run a Python 3.11+ service plus its database and migrations. Do not adopt it if you only need a retrieval library inside an existing application, or if you cannot absorb the cost of following a release cadence that has already produced v6.4.1. Before committing, verify three things against your own environment: that your chosen vector store and LLM backend are among those the project documents, that the Apache-2.0 licence and any bundled model or dependency licences are acceptable to your legal reviewers, and that the migration path from your current database works with the restai migrate command on a copy of your data.

Official sources

1. apocas/restai on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases