Misceo: a local gateway that routes agent traffic to cheaper models safely

The cost-quality trade, made explicit

Every team running AI agents faces the same invoice question: pay strong-model prices for every request, or route to cheaper models and eat the failures. Misceo, distributed as prebuilt binaries through npm, is a local Anthropic-compatible gateway that refuses to answer that question statically. Eligible requests first reach a lower-cost backend; the completed candidate then passes a structural gate that rejects upstream failures, empty output and malformed tool calls; an optional judge can score the visible candidate against the latest user turn; and rejected work is regenerated by a stronger backend whose answer the client actually receives.

The README's strategy table names the alternatives honestly: strong-only pays strong prices always, cheap-only has no recovery, static routers decide before generation and cannot inspect the answer. Misceo's claim is narrower and more defensible than making cheap models equal to strong ones: it makes the cost-versus-quality policy explicit, observable and configurable, with three named postures, quality-first, balanced and savings-first.

Five minutes to a working proxy

Requirements are Node.js 18 or newer and credentials for whichever backends you enable, with no separate Bun, Docker or UI installation, and platform packages covering macOS arm64 and x64, Linux glibc arm64 and x64, and Windows x64. The install and first run:

npm install --global @misceo/cli
mkdir my-misceo
cd my-misceo
misceo init --mode balanced

Provider keys go into a local .env, the README shows ANTHROPIC_API_KEY and ZAI_API_KEY as the examples, then two commands validate and start:

misceo doctor
misceo start

The doctor caveat is worth repeating: it performs static checks and does not contact providers or validate live credentials. The gateway listens on http://127.0.0.1:4141 with a dashboard on http://127.0.0.1:5141, and connecting Claude Code is an environment-variable one-liner that points ANTHROPIC_BASE_URL at the proxy with a local token.

Agent-safe routing: the part that actually matters

Routing agent traffic is different from routing chat, because agents carry state: sessions, tool loops, caches and conversation-shaped expectations. The README devotes its core section to four protections. Conversation continuity groups requests by structured session identity so bootstrap, title-generation and visible requests from one launch stay in one conversation. Cross-model handoff handles provider-specific thinking signatures, cache markers and message invariants at model-family boundaries.

Tool-loop ownership is the sharpest edge: when a model opens a tool loop, that backend keeps ownership until the loop completes, and Misceo will not move an active tool_use and tool_result exchange to another model family halfway. And first-visible-reply protection separates known launch bootstrap traffic from the user's first real request, so the answer a human actually sees comes from the strong backend before the cascade kicks in for later turns. These are the failure modes that make naive cheap-first routing break coding agents, addressed by name.

Observability as a feature

The embedded dashboard shows live traffic, history, cost and deletion controls, and every response is traceable to its decision: the serving backend, the route or cascade note, status, latency, usage and the judge score when a judge ran. Manual backend handoff is available from the CLI or dashboard when you disagree with the policy.

The README's verification recipe is a sixty-second loop: send a normal prompt in Claude Code, send a second small request, open the dashboard, and confirm that the first visible response was protected while a later request exercised the cascade. That is the right way to evaluate a router, by watching its decisions against your own traffic rather than trusting benchmarks, and the design makes those decisions legible enough to audit after the fact.

Where data goes, and what you are trusting

The privacy section draws the boundary precisely: the proxy, dashboard, configuration and traffic logs run locally, but local does not mean offline. Inference requests go to the providers your routing policy selects, and an eligible cascade means two providers see the request, the first provider and the escalation provider on rejection. A configured judge receives the capped latest user turn, the visible candidate and the tool names.

The trust question extends to the distribution model, and the README states it up front: this repository is the documentation, issue tracker and release home, the product ships as prebuilt binaries, and source code is not included. The licence badge reads FSL-1.1-ALv2, a source-available family licence, which GitHub reports as unrecognized. Running a closed binary that intercepts every prompt and tool call is a real decision; the README's own advice, read the privacy doc before using sensitive prompts or exposing the listeners beyond loopback, is the minimum version of taking it seriously.

Against static routers and paying full price

The alternatives frame the value precisely. Paying strong-only is the zero-effort baseline and the budget ceiling; request-only cheap routing saves money and corrupts agent state when a weak model fumbles a tool call; building your own cascade means reimplementing the session identity, tool-loop ownership and handoff logic that Misceo lists as features. Hosted gateway products occupy the same space with multi-tenant convenience, at the price of your prompts transiting their infrastructure, which Misceo's local-first design specifically avoids.

The fit is therefore narrow and real: teams running Claude Code or similar Anthropic-compatible agents at volume, with access to both a cheap backend and a strong one, willing to run a closed-source local binary to cut the bill. For everyone else, the README's strategy table is still worth reading, because it is one of the few places the routing trade-off is drawn as four rows instead of an advertisement.

Editorial conclusion

Misceo fits teams spending real money on Anthropic-compatible coding agents that have a cheaper second backend available, accept running a closed prebuilt binary locally, and will read the privacy doc before routing sensitive work through it. Skip it if you need open source in the path, if your traffic is too small to pay back the gateway, or if your agents cannot tolerate any interception layer. Verify it on real traffic in an hour: install, set balanced mode, connect Claude Code, and read the dashboard's cascade notes against your own prompts before trusting the savings estimate.

Frequently asked questions

Official sources

1. Issues
2. MaySudo/Misceo on GitHub
3. Project website
4. README
5. Releases