Portkey AI Gateway: One API for 1,600+ LLMs, But Watch the Guardrail Complexity
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
At a glance
- What is it?
- An open-source TypeScript gateway that routes requests to over 1,600 language, vision, audio, and image models behind a single OpenAI-compatible API. It adds retries, fallbacks, load balancing, and guardrails, but the real cost shows up when you configure those features.
- Who is it for?
- Adopt Portkey AI Gateway if you run multiple LLM providers and need a single OpenAI-compatible endpoint with retries, fallbacks, and content filtering. Skip it if you only call one model and want zero extra infrastructure, because the local footprint and config overhead outweigh the benefits.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 114 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The Problem: Provider Sprawl Without a Single Point of Control
Teams building AI features often end up with code that talks to three or four different model providers. Each provider has its own API shape, its own retry logic, and its own failure modes. Portkey AI Gateway attacks that sprawl directly. The README positions it as a lightweight, open-source, enterprise-ready solution that lets you integrate with any language model in under 2 minutes. The target user is a developer or platform team that wants to swap models without rewriting application code, add reliability features like automatic retries and fallbacks, and enforce content policies before responses reach users. The gateway presents one OpenAI-compatible API, so any client that already speaks OpenAI's protocol can point at it and reach Anthropic, Bedrock, Groq, or any of the 1,600+ models the project claims to support. That claim is broad, and the README does not list the full provider catalog, so you would need to check the docs before assuming your niche provider is covered.
How the Gateway Works: Proxy, Config, and Console
The core mechanism is a reverse proxy. You run the gateway locally, and it listens on http://localhost:8787/v1. Your application sends a normal chat completion request to that endpoint, and the gateway forwards it to the provider you specify. The Python example shows how the client picks a provider: you pass provider='openai' or provider='anthropic' and an Authorization header with that provider's API key. The gateway also exposes a Console at http://localhost:8787/public/, which the README says shows all of your local logs in one place. That console is a differentiator for local development, because most simple proxies just forward traffic and return responses. The gateway adds a visible layer where you can inspect what went through. The routing and guardrail logic lives in a config object that you attach to the client. The README shows a config with a retry attempt count and an output guardrail that denies any response containing the word 'Apple'. That config is not a separate file; it is a Python dictionary passed via client.with_options(config=config). So the gateway reads the config, applies retries and content checks, and only returns a response that passes the guardrails.
Getting It Running: Commands and First Request
The quickstart is genuinely short. You run npx @portkey-ai/gateway from a machine with Node.js and npm, and the gateway starts on port 8787. That single command pulls the package and runs it. Then you install the Python SDK with pip install -qU portkey-ai, create a Portkey client with a provider and an API key, and make a chat completion call. The README shows the client is OpenAI compatible, so you can also use the OpenAI SDKs directly by changing the base URL. For deployment beyond local, the README links to guides for Docker, Node.js server, Cloudflare Workers, Replit, and AWS EC2 via a CloudFormation template. The EC2 quick launch is a one-click deploy button, which is useful for teams that want a managed instance without writing infrastructure code. The gateway console is available immediately at the /public/ path, so you get local logging without setting up a separate observability stack. That is a low-friction start, but the README does not mention how to persist logs or configure authentication for the console, which matters if you deploy it beyond your laptop.
The Guardrail Config Is Powerful but Finicky
The headline feature after routing is guardrails. The README gives one concrete example: an output guardrail that denies any response containing the word 'Apple', with a retry count of 5. The config uses a key called 'default.contains' with an operator set to 'none' and a list of words. The comment claims that with this config, the model would always respond with 'Bat' because the guardrail denies any reply containing 'Apple', and the retry config would retry 5 times before giving up. That mechanism is clever: instead of just blocking a response, the gateway retries the generation until the output passes the guardrail or the retry limit is hit. But the example also reveals a limitation. The guardrail only checks the final output text for exact word matches. It does not catch paraphrases, synonyms, or semantic violations. If you need to block harmful content that does not contain a specific banned word, this operator is the wrong tool. The README mentions 50+ AI guardrails in the project description, but the visible documentation shows only this one operator. You would have to dig into the docs to see what the other guardrails detect and how they are configured. The config structure is also rigid: you must know the exact key name like 'default.contains' and the valid operators. A typo in the operator name would likely fail silently or error at runtime, and the README does not describe any validation or error messages.
Latency and Footprint Claims Need Scrutiny
The README makes two bold performance claims: 'blazing fast (<1ms latency)' and 'tiny footprint (122kb)'. The 122kb likely refers to the size of the JavaScript bundle or the deployed artifact, but the README does not specify what that number measures. The <1ms latency claim is even harder to verify. A gateway that proxies a request to a remote LLM cannot add only 1ms of overhead in practice, because network round trips alone take longer. What the claim probably means is that the gateway's own processing time, excluding the upstream call, is under 1ms. That is plausible for a simple router, but the README does not provide a benchmark, a methodology, or a test environment. There is also a claim that the gateway processes over 10B tokens every day. That number appears without a source or a measurement period, so it should be treated as a marketing figure, not a technical fact. For an engineer evaluating this gateway, the real question is not whether the proxy is fast in isolation, but whether the added layer changes your p99 latency when you add retries, guardrails, and load balancing. The README gives no data on that.
Where It Is the Wrong Tool
Portkey AI Gateway is not for every scenario. If you only call one model from one provider, adding a gateway introduces a new service to deploy, monitor, and secure. The local run command is fine for a demo, but in production you need to handle scaling, persistence for logs, and access control for the console. The README does not explain how to secure the gateway endpoint. Anyone who can reach the port can send requests and use your configured provider keys. The gateway also assumes you are comfortable with the Portkey SDK or the OpenAI-compatible protocol. If your application uses a provider-specific feature that the gateway does not expose, you may hit a wall. The README mentions multi-modal capabilities and agentic workflow integrations, but it does not show how to pass image inputs or tool calls through the gateway. That gap means the gateway might be a poor fit for complex agent loops that rely on provider-specific tool schemas. The guardrail retry mechanism also has a hidden cost: if a model keeps producing blocked content, the gateway will retry up to 5 times, which multiplies your token spend. The README does not warn about that cost.
Alternatives: LiteLLM and Direct Provider SDKs
The most direct alternative is LiteLLM, another open-source proxy that also exposes an OpenAI-compatible API and routes to many providers. LiteLLM takes a different approach to configuration: it uses a config.yaml file where you define model names, provider keys, and fallback models. Portkey instead pushes configuration into the client via a Python dict, which is more dynamic but less declarative. If you want infrastructure-as-code and a central config file that your whole team reads, LiteLLM's model is cleaner. If you want to change routing rules from application code without redeploying a config file, Portkey's approach is more flexible. The other alternative is to skip the gateway entirely and use each provider's native SDK. That gives you full access to provider-specific features and the lowest possible latency, but you lose unified retries, fallbacks, and guardrails. For a team with only one or two models, that trade-off is often worth it. The README also mentions that Portkey has a hosted cloud version, but the open-source gateway is the focus here.
Maintenance, Licensing, and the 2.0 Pre-Release
The project is under the MIT license, which means you can use it commercially without paying royalties, and you can modify the source. That is a permissive license, but it comes with no warranty, so you are responsible for any bugs you hit. The repository shows recent releases, with v1.15.2 dated January 2026, so the project is actively maintained. However, the README contains a prominent note about Gateway 2.0 (Pre-Release), saying that Portkey's core enterprise gateway is merging into open-source with the 2.0 release. That is a significant signal. If you adopt the current 1.15.x line, you may need to migrate to 2.0 when it stabilizes. The README links to a separate branch for 2.0.0, but it does not describe what changes or breaks between 1.x and 2.0. An engineering team should read the changelog and the 2.0 branch before committing to a long-term deployment. The maintenance cost also includes keeping up with new releases, because the gateway handles security-sensitive tasks like forwarding API keys. The README does not state how security patches are announced, so you would need to watch the repository or the changelog.
Editorial conclusion
Adopt Portkey AI Gateway if you run multiple LLM providers and need a single OpenAI-compatible endpoint with retries, fallbacks, and content filtering. Skip it if you only call one model and want zero extra infrastructure, because the local footprint and config overhead outweigh the benefits. Before committing, verify that the guardrail operators you need (like 'default.contains') exist in the version you deploy, and test how the gateway behaves under your specific traffic pattern, since the README's <1ms latency claim is not backed by any benchmark in the repository.
Community notes