Experiential: a self-hosted OpenAI-compatible gateway that turns agent traces into a router you own

The problem Experiential targets: many providers, one agent, no shared control point

Most agent stacks end up with the same shape. A coding agent talks to Anthropic, a batch job talks to OpenAI, someone's experiment points at a self-hosted vLLM box, and OpenRouter fills the gaps. Each client holds its own key, each provider bills separately, and nobody can answer a simple question: which model did this user actually call, and what did it cost? Experiential puts a gateway in front of all of them. The README describes it as an open source gateway and router for agent workflows with three jobs: one OpenAI-compatible API across hosted, BYOK and local models; per-user and per-agent control over which models are allowed, for which use cases, and at what spend; and turning production traffic into a custom router or model tuned for quality, speed and cost.

The audience is narrower than the tagline suggests. This is for teams that already have agent traffic worth measuring, usually people running Claude Code, Cursor, Codex or Aider against paid APIs, and who care about spend attribution enough to route through a local process. A solo developer calling one provider from one script gets little from it. The setup wizard is built around provider connections and a budget, not around a single key.

How the gateway and the native data plane fit together

The architecture splits into a Python control surface and a compiled data plane. The CLI is the `exp` command, installed from the `experiential` package. The request path is `exp-gateway-native`, a compiled gateway data plane declared as a dependency with the bound `exp-gateway-native>=0.3.82,<0.4`; the pyproject comment notes that checkouts build it via `[tool.uv.sources]`. The justfile exposes a `native` recipe that runs `uv run maturin develop --uv --release --manifest-path exp/runtime/gateway/native/Cargo.toml`, after which the gateway can be started with `uv run exp --engine rust`. So the request-handling layer is Rust, and the Python side owns catalog, identities, budgets and the optimizer.

Requests arrive on the gateway, are matched to a public alias, and are forwarded to whichever provider connection backs that alias. The README's opening example chooses a public alias such as `opus-5`, which is a name you define, not a provider model id. That indirection is the whole point: you can repoint an alias at a different provider without touching agent configuration. The Python client path goes further. `exp.load_router("my-project")` returns an official OpenAI client backed by its own private gateway, so a fitted project router is loaded as a context manager and called with `model="my-project"`.

The optimization loop is the part worth understanding before adopting. You export OpenTelemetry traces from your current agent, run `exp build support-agent` to build a simulation and fit a router against it, and later run `exp optimize model support-agent` to fine-tune an open source model through Tinker. The README points at a public OTLP dataset hosted on Hugging Face for people who want to try the pipeline without their own traces. That dataset is a sample, not a substitute for real traffic; a router fitted on terminal-task traces will reflect terminal-task traffic.

Installing Experiential and making a first gateway call

The README's getting-started path is two commands. On first run the setup wizard uses shared provider, model and reasoning-effort selectors, persists every provider connection you select, then shows defaults for the public alias, identity and a `$50.00` command budget before printing a one-time key.

pip install experiential
exp

Choose a public alias such as `opus-5` when prompted, and capture the key, because it is issued once. Then export it and send a chat completion to the local gateway on port 8000.

export EXP_GATEWAY_KEY=...
curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Authorization: Bearer $EXP_GATEWAY_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"opus-5","messages":[{"role":"user","content":"Help me"}]}'

The response should be a standard OpenAI-shaped completion. From Python, the same router can be loaded as a client.

import exp

with exp.load_router("my-project") as client:
    response = client.chat.completions.create(
        model="my-project",
        messages=[{"role": "user", "content": "hello"}],
    )

If you would rather not run anything locally, the hosted platform at `platform.experientiallabs.ai` serves the same OpenAI-compatible and Anthropic Messages API at `https://api.experientiallabs.ai/v1`, and SETUP.md contains copy-paste prompts you hand to a coding agent so it runs the setup for you. Working from a checkout instead, `just setup` copies `.env.example` to `.env` if one does not exist and runs `uv sync --extra dev`. The `.env.example` lists the keys you may need, including `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`, `AZURE_OPENAI_API_KEY` with `AZURE_OPENAI_ENDPOINT`, the AWS Bedrock chain, `TINKER_API_KEY`, `E2B_API_KEY`, and `EXP_ENDPOINT_API_KEY` with `EXP_ENDPOINT_BASE_URL` for a self-hosted vLLM or other trusted host.

Telemetry defaults and the local settings file

Anonymous aggregate PostHog product telemetry is on by default. The README states it never includes prompts, traces, actions, observations, paths, model names, credentials or raw customer content. Three subcommands manage it.

exp config telemetry status
exp config telemetry disable
exp config telemetry enable

The preference is stored locally in `.exp/settings.toml`. For anyone routing production prompts, the default is worth changing before the first real call rather than after. The claim about what is excluded is a documented boundary, not something this review can verify from the repository listing; if that distinction matters to your compliance review, read the telemetry code path in `exp/` before you point agents at the gateway.

Where Experiential is the wrong tool

The optimizer depends on traces. If your agents do not emit OpenTelemetry spans you can export, the build step has nothing to fit against, and you are left with an expensive proxy. The README's fallback is the public terminal-tasks OTLP dataset, which is fine for a trial and useless for production routing decisions about your own workloads.

There is a packaging constraint too. `requires-python` is `>=3.12`, and the request path depends on the compiled `exp-gateway-native` wheel. On a platform where that wheel is unavailable, the pyproject comment says checkouts build it via `[tool.uv.sources]`, which means a Rust toolchain and a maturin build; the justfile's `native` recipe confirms the PoC framing of that path. If your deployment target cannot compile Rust extensions, budget time for that before committing.

Finally, the fine-tuning step is not self-contained. `exp optimize model` depends on Tinker, declared as the optional `sft` extra with `tinker>=0.23,<0.24` and `tinker-cookbook>=0.4.3,<0.5`, and the `.env.example` carries `TINKER_API_KEY`. The README also mentions E2B sandboxes for distillation rollouts and optimizer or eval runs with the `e2b` backend. Teams expecting a fully offline pipeline will find two external services in the loop. The README does not document rollback behaviour for a router that performs worse than the baseline, so plan your own comparison before repointing production traffic.

How this differs from a plain reverse proxy or LiteLLM

LiteLLM is the obvious comparison and the difference is in what each one optimizes. LiteLLM is a translation and routing layer: you configure model lists, fallbacks and budgets, and it maps OpenAI-shaped requests onto provider APIs. Experiential does that too, but its stated third goal is to turn production traffic into a custom router or model. The build command is the evidence. `exp build support-agent` walks through providers, models and budget, asks for your trace file, and fits a router against a simulation; `exp optimize model support-agent` goes further and fine-tunes an open source model. A proxy does not have a training step.

The cost is operational surface. LiteLLM-style proxies are typically one process and a config file. Experiential adds a Python package, a compiled native data plane, a local `.exp/settings.toml`, trace export, and optionally Tinker and E2B. If your goal is only to hide provider differences behind one endpoint, the extra machinery is dead weight. If your goal is to stop guessing which model is good enough for a given class of task, the trace loop is the reason to accept it.

Maintenance, licensing and upgrade cost

The repository is not archived, and the last push was on 2026-09-17, the same day as the v0.7.89 release. Releases v0.7.89 and v0.7.88 both landed on 2026-09-17, with v0.7.88rc1 tagged as a cache-write billing prerelease. That cadence, multiple releases in a day with prerelease tags for billing changes, tells you two things: the project moves fast, and billing semantics are still being adjusted. Pin your version and read the release notes before upgrading a gateway that sits in front of production traffic.

The licence is Apache-2.0, recorded in the repository as `LICENSE` with the pyproject field `license = { file = "LICENSE" }`. Apache-2.0 permits commercial use and modification and includes a patent grant; it also requires that you keep the licence and notice files with redistributed copies. That is a summary of the licence text, not legal advice, and it says nothing about the hosted platform, the Tinker service or the model weights you might fine-tune, each of which carries its own terms.

Upgrade cost concentrates in two places. The `exp-gateway-native` dependency is bounded below 0.4, so a future major bump will require a coordinated upgrade of the Python package and the compiled plane. And because a fitted router is project state, a version change that alters router serialization would force a rebuild from traces. The README does not describe a migration path for fitted routers, so keep the trace exports that produced them.

Editorial conclusion

Adopt Experiential if you already run coding agents or agent workflows against several providers and want one loopback endpoint plus a router fitted from your own traces. Skip it if you only need a single provider key passed through, or if you cannot export OpenTelemetry traces, because the optimizer has nothing to learn from. Verify first that your Python is 3.12 or newer, that the exp-gateway-native wheel matches your platform, and that you are comfortable with anonymous PostHog telemetry being on until you run exp config telemetry disable.

Frequently asked questions

Official sources

1. experientiallabs/experiential on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases