Agent Router: an Envoy-based control plane for LLM and MCP traffic
Manages Unified Access to Generative AI Services built on Envoy Gateway
At a glance
- What is it?
- Agent Router (formerly Envoy AI Gateway) puts credentials, routing, quotas and failover for generative AI services behind one OpenAI-compatible API, enforced by Envoy Gateway. It is a Kubernetes control plane first, with a standalone `aigw run` mode for local use.
- Who is it for?
- Adopt Agent Router if you already run Kubernetes and Envoy Gateway and need central credentials, routing and quota enforcement for several model providers. Do not adopt it if you want a hosted proxy you sign into with a browser, or if you have no Envoy Gateway deployment and no appetite for CRDs.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Go, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on October 1, 2026, and from our analysis. They are not legal advice.
Editorial analysis
The problem Agent Router solves for platform teams
Application teams want one API shape for every model. Platform teams want the opposite: one place where provider credentials live, where quotas are counted, where a failing provider is retried against a different one, and where usage can be attributed back to a team. Without a gateway in between, each application ends up holding its own provider keys and implementing its own retry logic, and nobody can answer which service spent what.
Agent Router targets that gap. The README describes it as "the open source control plane for AI and agent traffic, powered by Envoy," and states that it gives application teams one consistent, OpenAI-compatible API for every model and tool, from hosted providers to self-hosted inference and MCP servers. The intended audience is therefore platform and infrastructure engineers, not application developers looking for a client library. The repository is a Go project, licensed Apache-2.0, and the last push was on 2026-09-10.
How the control plane and Envoy data plane fit together
The split is stated plainly in the README: "Agent Router controls. Envoy carries." Agent Router itself is the control plane. It watches Kubernetes resources and translates them into configuration that Envoy Gateway applies to Envoy proxies, which are the components actually forwarding request traffic.
The README describes a two-tier gateway pattern. The Tier One Gateway is a centralized entry point and handles authentication, top-level routing and global rate limiting. The Tier Two Gateway handles ingress traffic to a self-hosted model serving cluster, and the README says it provides fine-grained control over self-hosted model access, with endpoint picker support for LLM inference optimization. That division is a design choice worth noticing: the tier that terminates client authentication is separated from the tier that knows about individual inference endpoints, so a self-hosted cluster can be scaled or rescheduled without changing the public entry point.
Configuration is expressed as custom resources. The README names `AIGatewayRoute`, `AIServiceBackend` and `BackendSecurityPolicy`, all in the `aigateway.envoyproxy.io` API group. Those names are unchanged by the rename from Envoy AI Gateway, which means existing manifests keep working. The Go module path in go.mod is still `github.com/envoyproxy/ai-gateway`, and the container images are still published under `docker.io/envoyproxy/ai-gateway-*`. The rename moved the repository and the website, not the deployed artifacts.
Installing Agent Router and running a first request
The fastest path is the standalone CLI, documented in the CLI guide at theagentrouter.ai/docs/cli. The README gives this one-line example, which starts an OpenAI-compatible router on the local machine:
OPENAI_API_KEY=sk-your-key aigw runAfter that, the README says to point any OpenAI-compatible client at `http://localhost:1975/v1`. The port is fixed in the example, so a client configured for the OpenAI base URL works once you change the host. The CLI guide covers installation and provider auto-configuration; the README does not list a package manager command for `aigw`, so treat the CLI guide as the source for installation.
For a Kubernetes deployment, the Getting Started guide covers deploying with Envoy Gateway. The repository ships an `examples/` directory with working configurations, including `examples/basic/`, `examples/provider_fallback/`, `examples/token_ratelimit/`, `examples/mcp/`, `examples/inference-pool/`, `examples/monitoring/`, `examples/cache/` and `examples/aigw/`. Those directory names are the concrete starting points: a request-routing example, a fallback example, a token-based rate limit example, and so on. The README does not reproduce the YAML for any of them, so read the files in the repository rather than reconstructing them.
The supported provider list in the README is broad: OpenAI, Azure OpenAI, Google Gemini, Vertex AI, AWS Bedrock, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, Anthropic and the Tetrate Agent Router Service. Each is presented as a logo and a name; the README does not state per-provider feature parity.
Where Agent Router is the wrong tool
Two limitations stand out. The first is the deployment model. Agent Router is built on Envoy Gateway, so the Kubernetes path assumes you already run, or are willing to run, Envoy Gateway and its CRDs. If your stack is a single container behind a managed load balancer, adding a gateway control plane is a large step up in operational surface for what may be a handful of API keys. The standalone `aigw run` mode lowers that barrier for local work, but the README does not present it as a production deployment.
The second is that the project is a control plane, not a hosted service. It does not supply model credentials, does not host models, and does not provide accounts. Anyone looking for a sign-up flow, a dashboard, or free credits is looking at a different kind of product than what this repository contains. The repository is a Go codebase with `api/`, `cmd/`, `internal/`, `manifests/` and `tests/` directories, a Dockerfile, a Makefile and a Helm-oriented release process, which is the shape of something you deploy yourself.
A third constraint is documentation depth in the README itself. It links out to the docs site for concepts, the CLI and getting started, and it does not document rollback behaviour, upgrade procedures or failure semantics for a provider outage. Those answers live in the docs site and in `RELEASES.md`, not in the README.
How it compares with a plain Envoy Gateway setup
The obvious alternative is configuring Envoy Gateway directly, without Agent Router. Envoy Gateway already handles HTTP routing, TLS and rate limiting for ordinary services, and a team comfortable writing `HTTPRoute` resources can put an OpenAI endpoint behind it today.
The difference is what sits above the routing layer. Agent Router adds AI-specific custom resources: `AIGatewayRoute`, `AIServiceBackend` and `BackendSecurityPolicy` in the `aigateway.envoyproxy.io` group. Those resources exist to express concepts that generic HTTP routing has no vocabulary for, such as which backend serves which model name, how provider credentials are attached, and how token consumption is counted. The `examples/token_ratelimit/` directory is the concrete illustration: token-based limiting is not something a stock `HTTPRoute` expresses.
So the trade-off is real. Direct Envoy Gateway means fewer moving parts and one fewer control plane to upgrade. Agent Router means more resources to learn, in exchange for model-aware routing, provider fallback and token accounting already modelled. Teams that only need to proxy one provider to one backend will not get much from the extra layer.
Maintenance, licence and upgrade cost
The repository is not archived, and the last push was on 2026-09-10, which is recent. Releases are tagged and dated: v1.0.0-rc1 and v1.0.0 on 2026-06-23, then v1.1.0 on 2026-08-21. The README states that the rename from Envoy AI Gateway preserved "same code, same maintainers, same release cadence and Apache 2.0 license," and that CRDs, the API group, the `aigw` CLI, the `envoy-ai-gateway-system` namespace, container images, Helm charts and the Go module path are all unchanged. For anyone already running the previous name, that means the upgrade is a repository and website change, not a manifest migration.
Licensing is Apache-2.0, stated in the README and in the LICENSE file, with SPDX headers in source files such as the Dockerfile and Makefile. Apache-2.0 is permissive and includes an explicit patent grant, which matters when a gateway sits in front of vendor APIs. This is a description of the licence text, not legal advice; review it against your own distribution model if you plan to ship a modified build.
The upgrade cost that is visible is the coupling to Envoy Gateway. The go.mod pins `github.com/envoyproxy/gateway v1.9.0` and a set of `go-control-plane` modules, so Agent Router versions track Envoy Gateway and Envoy API versions. The repository also carries `.envoy-version` and uses `func-e` to download Envoy binaries, and the Dockerfile pre-downloads Envoy into the `aigw` image so the runtime does not need a writable volume for the binary. Upgrading Agent Router therefore means checking the Envoy Gateway version it expects, not only the Agent Router tag.
What to check before you deploy it
Start with the provider list. Confirm the provider you actually call appears among the sixteen named in the README, and then check whether the feature you need (streaming, tool calling, fallback) is documented for that provider rather than assuming parity across all of them.
Next, decide which tier you are deploying. If you only need a centralized entry point with authentication and global rate limiting, the Tier One Gateway alone may be sufficient. The Tier Two Gateway is specifically about ingress to a self-hosted model serving cluster and endpoint picker support, so it earns its place only if you run your own inference.
Finally, read the examples rather than the README for configuration syntax. The README gives the `aigw run` command and the port, and it names the custom resources, but it does not show a complete manifest. The `examples/` directory and the docs site at theagentrouter.ai/docs are where the actual YAML lives.
Editorial conclusion
Adopt Agent Router if you already run Kubernetes and Envoy Gateway and need central credentials, routing and quota enforcement for several model providers. Do not adopt it if you want a hosted proxy you sign into with a browser, or if you have no Envoy Gateway deployment and no appetite for CRDs. Before committing, verify the two-tier gateway split against your own topology, confirm which provider you need appears in the supported list, and check the release notes for v1.1.0 for changes since v1.0.0.
Frequently asked questions
What is Agent Router?
Agent Router is an Apache-2.0 control plane for AI and agent traffic, built on Envoy Gateway. It gives application teams one OpenAI-compatible API for hosted providers, self-hosted inference and MCP servers, while platform teams keep credentials, routing, quotas and failover in one place.
How do I use Agent Router?
For local use, the README gives the command `OPENAI_API_KEY=sk-your-key aigw run` and then points an OpenAI-compatible client at `http://localhost:1975/v1`. For Kubernetes, the Getting Started guide covers deploying with Envoy Gateway, and the repository's examples directory holds working configurations.
How do I use an Agent Router API key?
The README's standalone example passes the provider key as the environment variable `OPENAI_API_KEY` when starting `aigw run`. In the Kubernetes deployment, provider credentials are modelled through the `BackendSecurityPolicy` custom resource in the `aigateway.envoyproxy.io` API group, which is the resource that keeps credentials out of application code.
Is Agent Router safe?
The README does not contain a security audit or threat model, so safety cannot be asserted from it. What is documented is that the project is licensed Apache-2.0, is not archived, and keeps provider credentials in the control plane through `BackendSecurityPolicy` rather than in each application. The repository also carries a SECURITY.md and a Technical Charter.
What is Agent Router used for?
It routes generative AI and agent traffic. The README describes one OpenAI-compatible API for every model and tool, spanning hosted providers, self-hosted inference and MCP servers, with authentication, top-level routing, global rate limiting, quotas, failover and usage attribution handled by the gateway rather than by each application.
How do I use the Agent Router API?
The API is OpenAI-compatible. The README's standalone example starts the router with `aigw run` and then has clients send requests to `http://localhost:1975/v1`. On Kubernetes, the same endpoint shape is served by the Tier One Gateway, which the README describes as the centralized entry point handling authentication and global rate limiting.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/theagentrouter-agent-router)