Manifest (mnfst/llm-gateway): a self-hosted LLM gateway that routes agents across keys, subscriptions and local models

The problem Manifest solves: many providers, one endpoint

An agent that talks to OpenAI, Anthropic, a local model and a subscription plan needs four clients, four sets of credentials and four places where failures can happen. Manifest's answer is a gateway: you connect API keys, subscriptions and local models, and the README describes the result as one OpenAI-compatible endpoint where each query goes to the right model. The stated goal is avoiding single-provider lock-in.

The audience is narrow and identifiable. It is teams running agents or harnesses that already hold their own provider credentials and want to keep them, rather than buying inference through a reseller. The repository topics name the same concerns: byok, llm-router, cost-tracking, token-tracking, llm-observability. If you have no keys of your own, the value proposition is much weaker, because the gateway is a routing and accounting layer, not a source of model access.

There is a second, less obvious audience: people who need a record of what their agents actually sent. The README advertises full body logs for both successful and error messages, which is more than most hosted dashboards give you by default. That is a compliance and debugging feature as much as a routing one.

Routing, logging and fallback: the mechanism

The architecture visible in the repository is a TypeScript monorepo. The root package.json declares npm workspaces for packages/shared, packages/frontend, packages/backend, packages/manifest and packages/cli, with turbo.json driving builds and tests. The backend is NestJS (the start script runs packages/backend/dist/main.js) and the frontend is SolidJS. So the gateway is a server you run, with a database behind it and a dashboard in front, not a library you import into your process.

Data flow follows from that layout. A client sends an OpenAI-shaped request to the gateway; the gateway selects a provider based on routing rules; the request and its response body are recorded; cost and token figures are attributed to the call. The README lists four capabilities in this order: custom routing across API keys, subscriptions, local models and custom providers; full body logs for success and error messages; dollar tracking with notifications and limits; and fallback to different models when queries fail, which it describes as self-healing bad requests.

Two design choices deserve attention. First, full body logs mean the gateway stores request and response payloads, which is why every deployment path now provisions durable request-recording storage. That is a real operational commitment, not a config flag. Second, fallback is model-level, so a failing provider can be replaced by another model rather than retried in place. The README does not document how fallback ordering is configured or whether a fallback can be constrained to equivalent-capability models, so the quality of a fallback response is something you have to test yourself.

Installing Manifest self-hosted and routing your first request

The fastest path is the Docker image. The README gives a single install command that pulls and starts the stack:

bash <(curl -sSL https://raw.githubusercontent.com/mnfst/manifest/main/docker/install.sh)

After it finishes, the README says to open http://localhost:2099 and sign up, and notes that the first account you create becomes the admin. That port is the default for the self-hosted Docker path. The full self-hosting guide lives at docker/DOCKER_README.md in the repository.

For a first real use, the point of the gateway is that your client code does not change shape. You point an existing OpenAI-compatible client at the gateway's base URL and let the gateway decide which provider serves the call. The README does not print a literal example request, so the concrete step is configuration inside the dashboard: add a provider credential (an API key, a subscription, or a local model endpoint), then route to it. Once a request has gone through, the dashboard is where you should see the logged request body and the cost attributed to that call. If nothing appears there, the recording storage is the first thing to check, because the README states that every deployment path now uses durable request-recording storage.

Where Manifest is the wrong tool

The clearest limitation is storage. Because the gateway keeps full request and response bodies, it needs somewhere durable to put them. The README is explicit that volume-backed templates are single-instance and that you should use S3-compatible storage before scaling horizontally. If your plan is to run several replicas behind a load balancer on a template that mounts a local volume, the recording layer is the thing that breaks first, and the failure is silent until you go looking for a log that is not there.

Second, self-hosting means you own availability. The gateway sits in the path of every model call your agents make, so an outage is not a dashboard outage, it is an agent outage. The README does not document rollback or a degraded mode for the gateway itself, and it does not describe what happens to in-flight requests during an upgrade. That is a gap worth naming rather than papering over.

Third, if you have no provider credentials, Manifest adds a hop without adding access. It routes and accounts for models you already have. Someone who wants a single bill and a single key for many models is better served by a hosted aggregator, because the gateway's whole premise is that you bring the keys.

Finally, the project is in the middle of a direction change. The README carries a banner stating that Manifest is becoming a self-healing layer for APIs, with a new product that fixes failed API requests independently of the gateway, and that the open-source gateway stays available and maintained. That is a commitment in prose, not a guarantee about where future engineering effort goes. The last push to the repository was on 2026-09-10, and releases [email protected] and [email protected] landed on 2026-09-07 and 2026-09-10, so the gateway is currently receiving changes, but the banner is the reason to watch the roadmap rather than assume the gateway is the whole story.

Manifest compared with LiteLLM and OpenRouter

The two comparisons people actually search for are LiteLLM and OpenRouter, and the difference is structural rather than feature-by-feature.

LiteLLM is a Python proxy and SDK. Its centre of gravity is a Python library you can call directly from code, with a proxy mode on top. Manifest's centre of gravity is the opposite: a TypeScript server with a database and a dashboard, where the primary interface is an OpenAI-compatible HTTP endpoint and the primary artefact is the recorded request. If your stack is Python and you want in-process calls, LiteLLM fits more naturally. If your stack is a set of agents and harnesses in different languages that all speak the OpenAI API, a standalone gateway with a UI is the more direct fit.

OpenRouter is a hosted aggregator. You get one key and one bill, and you do not run anything. Manifest is self-hosted and BYOK, so you keep your own provider relationships and your own logs, and you take on the operational work. The trade is control and data locality against convenience. Manifest is not a way to get access to models you cannot otherwise reach.

The related searches also include an AWS angle. The README lists an AWS deployment path using CloudFormation that provisions ECS, RDS and a private recording bucket, so Manifest can run inside an AWS account rather than being an AWS service itself. It is not Bedrock and it does not replace Bedrock; it is a layer you can put in front of providers, including ones you reach through your own cloud account.

Licence, maintenance and upgrade cost

Manifest is MIT licensed, and the root package.json declares "license": "MIT" with the author listed as MNFST Inc. MIT is permissive: you can run it, modify it and ship it inside a commercial product, provided you keep the copyright and permission notice. That is the general shape of the licence, not legal advice, and if you redistribute a modified gateway or bundle it into something you sell, have your own counsel read the LICENSE file in the repository rather than relying on a summary.

The upgrade mechanics are visible in the repository layout. The project uses Changesets: there is a .changeset/ directory, a changeset script, a version-packages script that runs changeset version, and a release script that runs changeset publish. Releases in the [email protected] line arrive within days of each other, so the practical upgrade cost is not the version bump, it is the database and the recording storage. A gateway that stores request bodies accumulates data continuously, and the README does not document retention policy, pruning or a size limit for recordings. Before you put this in front of production traffic, decide how long you keep full bodies, because the default appears to be "all of them" and the storage grows with every call.

The maintenance picture is a moving target by the project's own description. The banner says the open-source gateway stays available and maintained while a separate self-healing product is built. The last push was on 2026-09-10, which is recent, but the roadmap statement is the thing to weigh if you are planning a multi-year dependency.

Editorial conclusion

Adopt Manifest if you already hold provider keys or subscriptions, want an OpenAI-compatible endpoint your harnesses can point at, and are willing to run the Docker image plus durable storage for request recordings. Do not adopt it if you want a fully managed router, or if you plan to scale horizontally on a volume-backed template. Before committing, verify two things in your own environment: that your chosen deployment path gives you the recording storage it expects, and that the routing and fallback behaviour matches how your agents fail.

Frequently asked questions

Official sources

1. License: MIT
2. mnfst/llm-gateway on GitHub
3. Project website
4. README
5. Releases