Agent Router: An Envoy Gateway Control Plane for AI Traffic

The problem Agent Router solves, and who has it

Application teams end up integrating each generative AI provider separately. One team holds an OpenAI key, another wires up Bedrock, a third writes bespoke code for a self-hosted model behind an internal endpoint. Credentials spread out, there is no single place to see who spent what, and failover between providers is written by hand in each service. Agent Router targets exactly that split. The README states it gives application teams one consistent, OpenAI-compatible API for every model and tool, from hosted providers to self-hosted inference and MCP servers, while platform teams keep credentials, routing, quotas, failover and usage attribution in one place. The split of responsibility is the point: applications speak one API, the platform team owns the policy. This matters most to organisations that have crossed from one provider to several, or that run both hosted and self-hosted models and want a single entry point. A team using one provider from one service will get little from it.

The two-tier gateway pattern and where Envoy sits

The README describes a two-tier gateway pattern. The Tier One Gateway is the central entry point and handles authentication, top-level routing and global rate limiting. The Tier Two Gateway handles ingress traffic to a self-hosted model serving cluster, giving fine-grained control over self-hosted model access, with endpoint picker support for LLM inference optimisation. The tagline is blunt about the division of labour: Agent Router controls, Envoy carries. So Envoy and Envoy Gateway are the data plane, and this project is the control plane that configures them. Two consequences follow. First, the request path is Envoy's, not a Go proxy written for this project, which means the operational surface you inherit is Envoy's. Second, the Tier Two tier only makes sense if you actually serve models yourself; if every model you call is a hosted API, you are running the Tier One half of the pattern and can ignore the rest. The README does not explain how the two tiers are configured differently, only what each is for, so the tier boundary is a concept to check against the Concepts documentation before designing around it.

Getting it running: one command, then Kubernetes

The fastest path in the README is a standalone router on a laptop. The example is:

OPENAI_API_KEY=sk-your-key aigw run

Then you point any OpenAI-compatible client at http://localhost:1975/v1. The CLI is still named aigw, and the README points to a CLI guide for installation and provider auto-configuration. For a real deployment the README directs you to the Getting Started guide to deploy on Kubernetes with Envoy Gateway. The configuration objects are Kubernetes custom resources: AIGatewayRoute, AIServiceBackend and BackendSecurityPolicy, all in the aigateway.envoyproxy.io API group. The namespace used by the project is envoy-ai-gateway-system. Container images are published under docker.io/envoyproxy/ai-gateway-*, and the Go module path is github.com/envoyproxy/ai-gateway. Note what this means: the rename from Envoy AI Gateway to Agent Router did not rename any of these. The README states the CRDs, API group, CLI name, namespace, images, charts and module path are unchanged and that old links redirect. If you have manifests from the earlier project, the README's claim is that they apply unchanged.

Provider coverage and the shape of the integration surface

The README lists the supported providers as OpenAI, Azure OpenAI, Google Gemini, Vertex AI, AWS Bedrock, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, Anthropic and the Tetrate Agent Router Service. That list matters less for its length than for what it implies about the abstraction. Each provider has its own authentication scheme and request shape; the project's job is to normalise them behind one OpenAI-compatible surface. The presence of Bedrock and Vertex AI in the same list as OpenAI and Groq suggests the credential handling is per-backend rather than global, which is consistent with BackendSecurityPolicy being a separate CRD. The list is also the first place to check your own stack. If the provider you depend on is not there, the README offers no statement about a generic or custom backend, so treat absence as an open question rather than an assumption that any HTTP endpoint will work.

Where Agent Router is the wrong tool

The dependency on Envoy Gateway is the main constraint, and it is structural rather than incidental. If you are not already running Envoy Gateway, adopting Agent Router means adopting that stack, its upgrade cadence and its configuration model. For a single service calling a single provider, that is a large amount of infrastructure for a problem a library or a few environment variables would solve. There is a second, quieter limitation in the material: the README presents aigw run as a standalone router for a laptop and points to a separate guide for Kubernetes deployment, but it does not state that the two modes expose the same features. Global rate limiting, quota enforcement and usage attribution are described in the context of the platform team's controls, which reads as the Kubernetes path. Anyone planning to start with aigw run and graduate to Kubernetes should confirm feature parity rather than assume it. Finally, the README is silent on what happens when a backend is unreachable or a provider returns errors, beyond listing failover as a capability. The mechanism is not described, so failover behaviour is something to verify in the documentation or in a test environment before relying on it.

The alternative: a single-process LLM proxy

The obvious alternative is a standalone LLM proxy that runs as one process and holds provider keys in its own config, without Kubernetes or Envoy. The difference is not which providers are supported; it is where policy lives and what carries the traffic. A single-process proxy puts routing, retries and key management inside that process, so you configure it in its own file or database and it is the data plane. Agent Router splits those: policy is expressed as Kubernetes custom resources (AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy) and the traffic is carried by Envoy Gateway. The trade is real in both directions. A single-process proxy is faster to stand up and has fewer moving parts, but it does not give you Envoy's filter chain, and it does not fit a platform team that already manages Kubernetes objects as the unit of change. Agent Router costs you the Envoy Gateway dependency and a CRD learning curve, and in return the routing rules, credentials and quotas live in the same declarative system as the rest of your cluster. If your platform team already reviews Kubernetes manifests, that is a genuine fit. If not, it is overhead.

Licence, maintenance and the cost of the rename

The project is Apache-2.0, and the README states the licence is unchanged through the rename, along with the release cadence and the maintainers. The release history shows v1.0.0-rc1 and v1.0.0 in June 2026 and v1.1.0 in August 2026, with the last push to the repository in September 2026. That is a project past its first stable release and moving in minor versions, not a pre-1.0 experiment. The rename has a practical maintenance cost that the README works hard to remove: because the CRDs, API group, CLI, namespace, images, charts and Go module path all keep their old names, existing manifests and imports do not need editing, and old repository and website links redirect. The residual cost is documentation drift. Search results, blog posts, talks and third-party guides will refer to Envoy AI Gateway for a long time, and the name in your own manifests will not match the name of the project. That is a readability problem for new engineers, not a technical one. Apache-2.0 carries no copyleft obligation on your own code, but the usual caveat applies: this is a description of the licence identifier, not legal advice, and if you redistribute or modify the project you should read the licence text and your organisation's policy.

Editorial conclusion

Adopt Agent Router if you already run Envoy Gateway or Kubernetes and need one OpenAI-compatible endpoint with credentials, routing, quotas and failover in one place; the CRDs (AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy) and the aigw CLI are the things to learn. Do not adopt it if you want a single binary in front of one provider, or if you are unwilling to run Envoy Gateway as the data plane. Before committing, verify three things against the current documentation: the supported Kubernetes and Envoy Gateway versions, whether the providers you actually use appear in the provider list, and whether the standalone aigw run mode covers the same features as the Kubernetes deployment, since the README presents it as a laptop quickstart rather than a description of parity.

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. theagentrouter/agent-router on GitHub