Otari: a self-hosted OpenAI-compatible gateway with virtual keys and pre-dispatch budget checks

The problem Otari solves is credential sprawl, not model access

Most teams do not struggle to call a model. They struggle to call it from six services without copying provider keys into six places. Otari's answer is to hold provider credentials behind the gateway and hand applications a revocable key instead. Those keys carry scope: the README describes them as scoped by user, workspace and model, which means a key issued to a batch job can be limited to one model and one workspace rather than inheriting everything the organisation has access to. The audience is therefore platform or infrastructure engineers inside an organisation that has already accumulated more than one provider relationship. A solo developer calling OpenAI from a script gets nothing from this. A team that has an OpenAI key in one repo, an Anthropic key in another, and no shared view of what either is spending gets the whole product.

Where Otari sits in the request path

Otari is a process between your application and the provider. A request arrives at an OpenAI-compatible route, Otari authenticates it, resolves the provider credentials, checks budgets, and only then dispatches. Provider calls are made through any-llm, a separate Mozilla AI library, so the provider adapters are not implemented inside Otari itself. After the call settles, Otari writes a usage record. That ordering is the design claim worth examining: a budget check that runs before dispatch is a different guarantee from a spend report that runs afterwards. The README states the gateway 'enforces budgets before dispatch, and records usage afterwards'. What the material does not specify is what happens to an in-flight request when the budget is exhausted mid-call, or how concurrent requests against a nearly exhausted budget are serialised. If you need a hard ceiling under concurrency, that is a question for the docs rather than something the README answers.

Routes, modes, and what the dashboard exposes

Three core completion routes are listed: POST /v1/chat/completions, POST /v1/messages, and POST /v1/responses. The second is the Anthropic-shaped endpoint, which is why the README calls the surface OpenAI and Anthropic compatible rather than OpenAI compatible alone. A running server publishes Swagger UI at /docs and the OpenAPI document at /openapi.json, so you can inspect the actual route set rather than trusting the summary. Standalone mode additionally serves a broader OpenAI-compatible and management API. Deployment mode matters here. Standalone runs management and inference in one process against local storage. Hosted is a multi-tenant control plane where inference happens on connected gateways. Hybrid is a data-plane gateway that resolves credentials locally and reports usage back to otari.ai. The mode selection rule is explicit: when OTARI_MODE is unset, the presence of OTARI_AI_TOKEN selects hybrid, otherwise Otari defaults to standalone. That implicit fallback is worth knowing about, because a stray OTARI_AI_TOKEN in an environment changes which mode you boot into.

Getting a gateway running in two commands

The fastest path is an ephemeral container. The README gives this example: docker run --rm -p 8000:8000 with OTARI_MASTER_KEY set to a master key, OPENAI_API_KEY set to a provider key, and OTARI_CONFIG_YAML set to 'default_pricing: true', running the image mzdotai/otari:latest with the command otari serve. On a first, empty database Otari creates a bootstrap API key and prints it once, prefixed gw-, with the message that you should save it now. Requests then go to http://localhost:8000/v1/chat/completions with that key as a bearer token and a body naming the model as 'openai:gpt-4o-mini'. The provider-prefixed model string is the routing instruction. Existing OpenAI clients work by pointing base_url at http://localhost:8000/v1. The container uses SQLite inside the container and is deleted on stop, so nothing survives. For persistence the README points at the Compose setup: clone the repository, copy config.example.yml to config.yml, set a master key, provider credentials and pricing, then docker compose pull and docker compose up -d. Compose runs PostgreSQL. Optional profiles add the code sandbox, web search and guardrail services via --profile code-exec --profile web-search --profile guardrails. The dashboard is served at http://localhost:8000/, and to store provider keys through it you must set OTARI_SECRET_KEY to a Fernet key produced by otari gen-secret-key. That last detail is the one people miss: the dashboard will not hold provider credentials without it.

The pricing table is the load-bearing part of the budget feature

Budgets are only as accurate as the prices Otari uses to compute spend. In the quickstart that table comes from OTARI_CONFIG_YAML with default_pricing: true, and in the Compose path the README instructs you to set pricing in config.yml alongside credentials. Nothing in the supplied material describes how default_pricing is maintained, how often it is refreshed, or what happens when a provider changes a price and the table does not. That is a real operational exposure for anyone treating the budget as a financial control rather than a rate limiter. If you route through a provider with negotiated or tiered pricing, the default table will not reflect your invoice. The configuration doc is the place to check whether per-model overrides exist. Until you have confirmed that, treat Otari's budget enforcement as a guardrail against runaway loops and a runaway key, not as a reconciliation system.

Standalone is the honest default; the hosted modes are a different product

The README frames three runtime modes as equal options, but they are not equivalent in what they ask of you. Standalone is self-contained: one process, local storage, your database, your keys. Hybrid keeps inference and credential resolution on your side while usage reporting goes to otari.ai, which means metadata about your traffic leaves your network. Hosted is a multi-tenant control plane, which is a managed offering rather than a self-hosted gateway. The tagline is 'you own and run', and standalone is the mode that matches it. If you are evaluating Otari because you want provider credentials to stay inside your perimeter, the hybrid mode's reporting path is the thing to read carefully in docs/modes.md before you enable it. The material does not describe what fields the usage report contains, so that question is open.

How this differs from LiteLLM

The repository topics list litellm-alternative, so the comparison is intended. LiteLLM is the incumbent for provider normalisation: a Python SDK and proxy that translate many provider APIs into an OpenAI-shaped one. Otari also normalises, but it does so by delegating provider calls to any-llm rather than implementing the adapters itself, and it puts virtual keys, per-user and per-workspace scoping, and pre-dispatch budget enforcement in the core rather than in a separate spend-tracking layer. The practical difference is where the identity model lives. With a thin proxy you typically authenticate applications with the provider keys themselves and add spend visibility on top. Otari inverts that: applications never see provider credentials, and the gateway's own key is the unit you revoke, scope and budget. If you already run LiteLLM and only need routing, switching buys you little. If your actual pain is that you cannot answer who spent what, or cannot revoke one team's access without rotating a shared provider key, that is the gap Otari targets. Note also the extras the README lists as optional: code execution, web search, MCP, guardrails and file understanding, which are separate Compose profiles rather than always-on features.

Licence, upgrade surface, and what to check before you deploy

Otari is Apache-2.0, which permits commercial use and modification; the repository ships a LICENSE file and the README points to it. This is not legal advice, and if you redistribute Otari or offer it as a service, read the licence text and your own obligations rather than relying on a summary. On maintenance cost, the release cadence visible in the material is active but not frantic: v0.5.1 and v0.5.0 both landed on 2026-09-08, with v0.4.0 before them on 2026-07-24. Two releases on the same day suggests the second was a patch for the first, which is normal at this stage and also a reason to pin a version rather than track latest. The pre-1.0 version number is the more useful signal: configuration keys and route behaviour can still move between minor releases, so an upgrade is a thing you schedule with a changelog open, not a thing you let happen on a pull. The development workflow is standard Python: uv sync --dev, copy config.example.yml to config.yml, then uv run otari serve --config config.yml, with sqlite+aiosqlite:///./otari.db as the database_url if you want to skip PostgreSQL locally. Checks are make test, make lint, make typecheck, plus pnpm --dir web run lint for the frontend. The concrete thing to verify first is provider coverage: Otari routes through any-llm, so the providers you actually use must be supported there, and the README's 40+ figure is a claim about that library's reach, not a list you can audit from this page.

Editorial conclusion

Adopt Otari if you are running several applications against several providers and you need per-key budgets and revocation without handing provider credentials to each app. Skip it if you only talk to one provider from one service, or if you cannot operate PostgreSQL and a secret-encryption key, because the persistent deployment expects both. Before committing, verify three things against your own setup: that your target providers are covered by any-llm, that your config.yml pricing entries match what you actually pay, and that the standalone mode's local storage is the right choice rather than the hybrid data-plane mode. The budget check only means something if the pricing table behind it is correct.

Official sources

1. License: Apache-2.0
2. mozilla-ai/otari on GitHub
3. Project website
4. README
5. Releases