Plano: An Envoy-Based Data Plane That Moves Agent Routing Out of Your Application Code

The Middleware You Write After the Demo Works

The README states the problem directly: building agentic demos is easy, shipping them is hard. The specific work it names is the hidden middleware that appears once a prototype meets real traffic. Routing logic that decides which agent handles a request. Guardrail hooks for safety and moderation. Evaluation and observability glue. Model and provider quirks spread across frameworks and application code. Each of those is small on its own. Together they are the reason a working demo takes months to become a service. Plano's answer is to move them out of the application process and into a separate data plane that sits in front of your agents. The intended reader is an engineer who already has agents running as HTTP servers and does not want to reimplement intent classification, provider adapters or trace instrumentation in every one of them. The README is explicit that this is framework-agnostic: the agents are ordinary servers implementing the OpenAI-compatible chat completions endpoint, in any language.

Agents as OpenAI-Compatible Endpoints Behind a Declared Listener

The architecture visible in the material is a proxy in front of your own services. You declare agents by id and URL, and you attach natural-language descriptions to them. The README describes what you do not write: intent classifiers, routing logic, model fallbacks, provider adapters, tracing instrumentation. A listener of type agent binds a port and names a router. In the travel example the listener is travel_assistant on port 8001, with router plano_orchestrator_v1, which the README describes as a 4B-parameter routing model. The listener lists the agents it fronts, each with a description that includes example utterances such as "What's the weather in Paris?" and "Flights from NYC to LA". Model providers are declared separately, with a model string like openai/gpt-4o or anthropic/claude-3-5-sonnet, an access_key read from an environment variable, and a default flag. Your agent code then points its OpenAI client at Plano's LLM gateway rather than at a provider. In the sample weather_agent.py the base URL is http://localhost:12001/v1 with the API key set to the literal string EMPTY, and the request asks for model openai/gpt-4o. So there are two distinct hops: the orchestration listener that picks an agent, and the LLM gateway that picks a provider and model. A tracing block with random_sampling set to 100 controls capture. The README also mentions Filter Chains for jailbreak protection, moderation policies and memory, documented under a separate concepts page.

planoai up and the Config Keys That Matter

The material gives one command and one file. You run planoai up config.yaml after following the prerequisites guide, which covers installing Plano and setting up the environment. The config file opens with version: v0.3.0, which is worth noting because the repository's most recent tagged release is 0.4.36. The README does not state whether that version field is validated against the binary or merely informational, so treat a mismatch as something to check rather than assume. The keys that carry the design are agents (id and url), model_providers (model, access_key, default), listeners (type, name, port, router, agents), and tracing (random_sampling). Access keys are referenced indirectly as $OPENAI_API_KEY and $ANTHROPIC_API_KEY, so provider credentials stay in the environment rather than in the YAML. The listener's agents entries repeat the id and add a description; the description is the routing input, which means the quality of your routing depends on how well you write those descriptions. Nothing in the supplied material documents a schema validation step, a dry-run flag, or a config reload path, so I cannot say how a bad config surfaces.

The Hosted Router and What It Implies for Production

The README carries an important note: Plano and the Plano family of LLMs, including Plano-Orchestrator, are hosted free of charge in the US-central region for a first-run developer experience. To scale and run in production, you either run those LLMs locally or contact the maintainers for API keys. This is the single most consequential operational detail in the material, and it sits in a callout rather than in the quickstart. The default router name in the example config is plano_orchestrator_v1, which points at that family. The practical consequence is that the path of least resistance during evaluation sends routing decisions involving your users' prompts to a hosted endpoint in a specific region. That may be fine for a prototype and unacceptable for a regulated workload. The README does not document the local serving path in the material I have, so I cannot describe how you would run the orchestrator yourself. If you adopt Plano, resolving this question comes before anything else.

Where Plano Is the Wrong Tool

Plano adds a network hop and a configuration surface. For a single agent talking to a single provider, that hop buys you nothing you could not get from a client library, and the YAML becomes a second place to look when something breaks. The routing model is also a dependency you do not control: you are trusting a 4B-parameter model to classify intent from descriptions you wrote, and the README offers no accuracy figures, no evaluation methodology and no fallback behavior for the case where the router picks the wrong agent. That is a real failure mode. A misrouted request in a customer-facing flow is worse than a slow one. The README does not describe a confidence threshold, a rejection path, or a way to force a specific agent for a given request. There is also the release cadence: 0.4.34, 0.4.35 and 0.4.36 landed on consecutive days in August 2026. Frequent patch releases are normal for active projects, but they mean you should pin a version rather than track main. The version field in the sample config, v0.3.0, is well behind that cadence, which suggests config compatibility is not something to take for granted across upgrades.

Compared With Putting a Gateway in Front of Your Providers

The obvious alternative is a conventional LLM gateway or a plain reverse proxy that forwards to providers and handles key management, rate limits and retries. That class of tool solves model agility: one endpoint, many providers, no provider SDKs in your services. Plano does that too, through the same LLM gateway your agent points at. The difference is the orchestration listener. A conventional gateway has no concept of an agent as a routable destination with a natural-language description, so the decision about which service handles a request stays in your application. Plano moves that decision into the proxy and makes it configurable without redeploying agents, which the README frames as adding new agents without modifying app code. If your routing logic is a handful of if-statements that rarely change, the conventional gateway is the smaller dependency. If you are adding agents regularly and each addition means touching a dispatcher, Plano's model is the one that scales with the team rather than with the code.

Licence, Upgrades and What to Pin

Plano is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant. It does not impose copyleft obligations on your own code. This is not legal advice, and the licence text governs. The material does not describe a compatibility policy between config versions and binary versions, and the gap between the sample config's v0.3.0 and the 0.4.x releases is the reason to care. Practically, pin the image tag or the binary version in your deployment, keep config.yaml in version control next to the agents it describes, and read the release notes for the versions you skip. The README also points at a Discord for API keys and a docs site for the full quickstart, so the upgrade path for the hosted router is a conversation rather than a documented procedure. That is a cost worth counting before you build on it.

Editorial conclusion

Adopt Plano if you are running two or more agents behind a single endpoint and you want intent routing, model fallback and trace capture declared in YAML rather than written into each service. Do not adopt it if you have one agent and one provider, because the config file and the extra network hop buy you nothing. Before committing, verify whether the default plano_orchestrator_v1 router resolves to the hosted US-central endpoint or to a locally run model, and confirm what that means for your data path.

Official sources

1. katanemo/plano on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases