ORBIT: A Self-Hosted Gateway That Puts Adapters Between Your Models and Your Data

The gap ORBIT fills: one contract, many backends

Most teams that want retrieval over private documents end up writing the same middleware twice. Once to talk to a model provider, once more when the provider changes. Then a third time when the data source is not a vector store but a Postgres table, a MongoDB collection, or an internal HTTP API. ORBIT's stated position is that it sits between your applications and the models, data, and tools they need, and that you can move from a local prototype to a governed deployment without replacing the architecture. The README frames the audience as developers who want private RAG, natural-language data access, and tool-calling agents behind one API. The concrete deliverable is an OpenAI-compatible endpoint plus a REST API, so existing clients that speak the OpenAI chat format can point at ORBIT instead of a vendor. That is the pitch: the client contract stays stable while the thing behind it is swapped.

How the adapter layer actually works

The mechanism the README describes is YAML-configured adapters. Files, SQL, NoSQL, vector stores, Elasticsearch, REST and GraphQL APIs, and MCP tools are all brought in through that single configuration surface, and the admin panel is described as the control plane where adapters, API keys, prompts and system operations are managed without touching server code. That is a meaningful design choice. Rather than exposing a plugin API in Python and asking you to subclass something, ORBIT asks you to declare a data source in configuration, and the server handles the retrieval or query path. The README also mentions hot adapter reloads, which implies adapters can be changed at runtime rather than only at boot. For the database case the README is specific about the shape: parameterized queries across SQL, MongoDB, Elasticsearch and composite data sources, and the HR demo is described as using reviewed intent templates. That phrase matters. It suggests natural-language questions are mapped onto a set of pre-approved intents rather than translated into free-form SQL on every request. Composite sources are also supported, with the Customer 360 demo described as joining billing data from SQL, support SLA data from HTTP, and CRM context from MCP tools in a single request.

Protocol surface: MCP in both directions, A2A, WebSocket voice

ORBIT is not only an HTTP gateway. The README lists it acting as an MCP server, connecting to external MCP servers as a tool-using client, providing an A2A endpoint, serving requests through REST and OpenAI-compatible APIs, supporting real-time speech-to-speech over WebSocket, and being integrated asynchronously through a message-queue interface. There is also a Node.js client under clients/node-api and a separate OrbitChat npm package used for the demos. Acting as both MCP server and MCP client is the more unusual part of that list. It means an ORBIT deployment can expose its own capabilities to an external agent host while simultaneously consuming tools from other MCP servers, with the README describing scoped access to MCP servers and bounded tool loops for agent construction. Bounded tool loops are worth noting as a stated constraint rather than an open-ended agent runtime. The speech-to-speech path is documented as low-latency voice chat with responses grounded in knowledge sources, and the realtime-voice client lives in its own directory.

Getting it running: tarball, Python 3.12, Ollama by default

The documented install path is a release tarball rather than a package manager. The README gives the v2.17.8 download URL and shows a curl command against the release asset; the second line of that snippet is truncated in the material I have, so treat the extraction and startup steps as something to read from the release directory itself. Prerequisites are stated as Python 3.12+ and an internet connection for dependencies. The default configuration uses Ollama for inference, so Ollama needs to be installed if you keep the default provider. Windows users are pointed at a separate guide under install/windows.md. Configuration is YAML, and per the admin panel description, adapters, API keys and prompts are managed through the UI rather than by editing server code. The README also points at docs/tutorial/http-apis.md for the HTTP surface, docs/tutorial/mcp-tool-calling.md for MCP client usage, docs/tutorial/message-queue-async.md for the async path, and docs/mcp_protocol.md and docs/a2a-protocol.md for the two protocol endpoints. Those files are where the actual config keys live; the README itself does not enumerate them.

Where the design constrains you

The reviewed intent templates are the first real limitation. If database access runs through intents that someone has to define and review, then a question outside the intent set has no path to an answer, and adding a new class of question is a configuration task, not a prompt. That is a defensible safety posture for HR or billing data, and the README's own examples lean that way. It is a poor fit if you want an analyst to ask anything of an unfamiliar schema on day one. The second constraint is the adapter inventory itself. The README names SQL, MongoDB, Elasticsearch, vector stores, REST and GraphQL, DuckDB and SQLite in the demo list, but the presence of a demo is not the same as a general adapter for your particular engine. Verify against the repository before assuming coverage. Third, the YAML adapter model means the set of things you can connect is bounded by what the project has implemented, not by what you can code. A team comfortable writing a Python retrieval service may find that boundary more annoying than the abstraction is worth. Fourth, this is a fast-moving codebase. Three patch releases landed in the four days before the last push recorded here, which tells you the surface is still shifting and that pinning a version is advisable.

How it differs from a plain model proxy

The obvious comparison is a lightweight OpenAI-compatible proxy such as LiteLLM. LiteLLM's job is provider normalization: you send an OpenAI-shaped request, it routes to Anthropic, Bedrock, Gemini or a local server, and it returns an OpenAI-shaped response. ORBIT does that too, listing Ollama, llama.cpp, vLLM, OpenAI, Anthropic, Gemini, Bedrock, Microsoft Foundry and OpenRouter among its routing targets. The difference is what sits on the other side of the request. A proxy hands the call to a model and stops. ORBIT is documented as owning the retrieval and query path: it decides which adapter answers, runs parameterized queries against SQL or MongoDB, joins composite sources, and enforces the tool loop for MCP calls. It also ships governance that a bare proxy does not, namely RBAC, OIDC/SSO, identity allowlisting, per-key quotas, audit logs, moderation and file encryption, plus provider fallbacks, retries and circuit breakers on the reliability side. If all you need is provider failover, ORBIT is a much larger dependency than the problem requires. If you need the data plane as well as the model plane, the comparison flips.

Maintenance, licensing and what to check before you commit

The licence is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant, with the usual requirements around preserving notices and stating changes. That is a permissive licence and it does not impose copyleft on your own code; it also means no vendor is obligated to support you. Support here comes from the repository, the docs directory and the release cadence. The release history shows frequent patch versions, so budget for periodic upgrades rather than a set-and-forget deployment, and read the release notes before moving between minor versions because adapter configuration is the kind of surface that changes. The security documentation referenced in the README, a NIST SP 800-53 and OWASP Top 10 mapping plus a capability matrix, is the place to look if you need to justify the deployment internally. One thing the material does not settle is how the admin panel's role-based access interacts with the API keys it issues; that is worth confirming in the docs rather than assuming. The practical check before adopting is narrow: confirm your database engine has an adapter, confirm your chosen provider is in the routing set, and start the pinned tarball once on your own Python 3.12 environment.

Editorial conclusion

Adopt ORBIT if you need one governed endpoint in front of several model providers and several private data sources, and you are willing to describe those sources as YAML adapters rather than write bespoke glue. Do not adopt it if you only need a thin OpenAI proxy, or if you want the model to write arbitrary SQL against your schema without a review step; the intent-template path is the documented route for database access and it assumes someone curates those intents. Before committing, verify three things against your own environment: that your target database has a matching adapter in the repository, that the provider you intend to use is listed in the supported routing set, and that the release tarball for the version you pin actually starts with your Python 3.12 interpreter. The admin panel is the control plane, so check whether its adapter management covers the credentials and quotas your deployment needs.

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. schmitech/orbit on GitHub