# aiproxy: one OpenAI compatible door in front of many AI providers

> AI Proxy is a Go gateway that accepts the OpenAI, Anthropic and Gemini protocols, converts between them, retries across channels by priority and error rate, and adds the things a shared AI backend needs: quotas per group, per group pricing, MCP servers, plugins and a management panel.

**labring/aiproxy** — AI Proxy is a high performance AI gateway using OpenAI / Claude / Gemini protocol as the entry point. It features intelligent error handling, multi-channel management, and comprehensive monitoring. With support for multiple models, rate limiting, and multi-tenant isolation.

- Repository: https://github.com/labring/aiproxy
- Stars: 564 · Forks: 117
- Language: Go
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/labring-aiproxy

## A gateway that speaks three protocols

AI Proxy is a high performance AI gateway that uses the OpenAI, Claude and Gemini protocols as its entry point, and the readme calls it a next-generation AI gateway with an OpenAI-compatible protocol. The code is Go, the licence is MIT, and the default branch is main.

The problem it solves is the one that appears the moment more than one provider or more than one team is involved. Every client has to know which upstream to call, which key to use, what to do when a provider is rate limited, and how the two of you are billed. A gateway puts one endpoint in front of all of that.

The protocol angle is what makes it low friction. Because the entry point accepts the OpenAI Chat Completions shape, the Anthropic Messages shape and the Gemini shape, an existing client usually keeps working and the conversion happens inside the proxy. The feature list calls this protocol conversion between OpenAI Chat Completions, Claude Messages, Gemini and the OpenAI Responses API, and it notes specifically that a Chat, Claude or Gemini request can be sent to a Responses API only model.

The audience is a platform team, not an individual developer. The readme describes the software as production-ready and as middleware for AI applications requiring reliability, scalability and advanced features, and the features it leads with are retry logic, priority based channel selection, load balancing across providers, monitoring and multi tenant isolation.

## The pipeline: auth, router, plugins, monitoring

The architecture diagram in the readme is the clearest description of how a request moves through the system.

Everything enters at the gateway, and the gateway hands the request to four subsystems. Authentication and authorization come first, then the intelligent router, then the plugin system, with monitoring and analytics running alongside.

The router is the part that talks to the outside world. It fans out to OpenAI, to Anthropic, to Azure OpenAI, and to other providers, so the upstream set is a configuration concern rather than something baked into the client. The plugin system sits beside it with four named plugins, the cache plugin, the web search plugin, the think split plugin and the stream fake plugin, and the monitoring subsystem feeds an alert system, an analytics dashboard and audit logs.

The MCP side is a third branch off the gateway, leading to MCP servers, and those servers are themselves split three ways: public MCP integrations, organisation level MCP servers, and embedded MCP servers with configuration templates. There is also an OpenAPI to MCP conversion described in the feature list, which turns an OpenAPI specification into MCP tools.

So the shape is a gateway with three outward directions, upstream model providers, MCP servers, and the monitoring and alerting surfaces, with plugins in the middle doing work on the request before it leaves. The management panel is the human face of the monitoring branch, and the readme shows a dashboard and a logs view.

## Retries, channel priority and the plugin layer

The request management features are the ones that make a gateway worth running, and the readme describes them at the level of intent rather than configuration.

The smart retry logic is described as intelligent retry strategies with automatic error recovery. Priority-based channel selection routes requests according to channel priority and to error rates, which is a different thing from a static ordering: a channel that starts failing stops being preferred even if it is nominally the first choice. Load balancing distributes traffic across multiple AI providers.

The plugins are where the unusual behaviour lives. The cache plugin provides high performance caching for identical requests with Redis or memory storage, and it is the one that needs infrastructure, since the compose file includes a Redis service. The web search plugin adds real time web search with support for Google, Bing and Arxiv. The think split plugin handles reasoning models by splitting content and automatically dealing with think tags, which is a real problem when a provider returns a chain of thought inside the response body. The stream fake plugin exists to avoid non-streaming request timeouts by transmitting internally as a stream.

That last one is a good illustration of what a gateway is for. A client that does not stream has a timeout problem that is not the client's fault and not the provider's, and the fix is to make the upstream leg look like a stream. That is a problem only a proxy in the middle can solve.

The advanced capabilities list adds multi-format support for text, image, audio and document, model mapping with flexible aliasing and routing, prompt caching with billing support, think mode for reasoning models with content splitting, and a built-in tokenizer with no external tiktoken dependency.

## Multi-tenancy as the organising idea

The multi-tenant section is what distinguishes this from a simple reverse proxy, and it is worth reading closely because it determines how the gateway is operated.

Organisation isolation is described as complete separation between different organisations. Access control is token-based with subnet restrictions, so a token can be limited to a network range as well as to a group. Resource quotas are RPM and TPM limits with usage quotas per group. And custom pricing means per-group model pricing and billing configuration.

That set of four decisions is the difference between a shared proxy and a platform. A team that just wants a fallback key does not need any of it, and a company running several departments or several customers on one set of provider credentials needs all of it, because otherwise one department's traffic consumes another's budget and one department sees another's data.

The monitoring side is built for the same audience. Real-time alerts cover balance warnings, error rates and anomalies, and the analytics cover request volume, error statistics, RPM and TPM metrics and cost analysis, with channel performance giving error rate analysis per channel. Detailed logging is described as complete request and response tracking with audit trails, and the feature toggle SAVE_ALL_LOG_DETAIL controls whether all request details are written.

The privacy implication is worth stating plainly. A gateway that logs every request and response body is storing every prompt and every completion, in a database, with a retention period you configure in hours. That is a deliberate feature for auditing and debugging, and it is also a data protection decision that the operator has to make on purpose.

## Running it: one container, or a compose stack

The quick start offers two routes, and the first one is a single container with a volume and an admin key.

```bash
# Quick start with default configuration
docker run -d \
  --name aiproxy \
  -p 3000:3000 \
  -v $(pwd)/aiproxy:/aiproxy \
  -e ADMIN_KEY=your-admin-key \
  ghcr.io/labring/aiproxy:latest
```

The nightly build uses the main tag instead of latest, so a development channel is a one word change. The compose route downloads the compose file and starts the services.

```bash
# Download docker-compose.yaml
curl -O https://raw.githubusercontent.com/labring/aiproxy/main/docker-compose.yaml

# Start services
docker-compose up -d
```

The compose stack is three services, and the dependency handling is explicit.

```yaml
    depends_on:
      pgsql:
        condition: service_healthy
      redis:
        condition: service_healthy
    ports:
      - "3000:3000/tcp"
    environment:
      - ADMIN_KEY=aiproxy
      - LOG_DETAIL_STORAGE_HOURS=1
      - TZ=Asia/Shanghai
      - SQL_DSN=postgres://postgres:aiproxy@pgsql:5432/aiproxy
      - REDIS=redis://redis
```

The application waits for both PostgreSQL and Redis to pass their health checks before it starts, and each service has its own health check, so a database that is not ready does not produce a crash loop.

The core settings are environment variables, and the defaults are the interesting part.

```bash
LISTEN=:3000                    # Server listen address
ADMIN_KEY=your-admin-key        # Admin API key
DISABLE_WEB_ROOT=true           # Redirect only `/` to GitHub, keep other web routes available
```

The disable web root option exists so that a deployment can point the root path at the project page while keeping the management panel and API routes reachable, which is what you want behind a company proxy.

## Where AI Proxy is the wrong tool

Four limits, and the first one is the simplest case.

A single developer with one provider and one key does not need a gateway. Every feature that makes this project valuable, the retries, the channel priority, the per group pricing, the organisation isolation, the MCP layer, the audit logs, is a feature for a shared backend, and each of them adds a component that can fail between the client and the model. The latency of the extra hop is real, and so is the operational surface.

The second is the data. This is a component that sees every prompt and every completion, and with SAVE_ALL_LOG_DETAIL enabled it writes the request and response details to the log database. The retention settings are in hours, LOG_STORAGE_HOURS for logs and LOG_DETAIL_STORAGE_HOURS for detail logs, and the compose example sets the detail retention to one hour. Even so, a team handling data subject to a contractual or regulatory constraint has to read the logging section of the configuration documentation before enabling that.

The third is the surface area. The plugin system, the MCP servers, the OpenAPI to MCP conversion, the think mode, the model mapping, the billing, the quotas: each of those is a configuration surface with its own failure modes, and the version is 0.7.x, so the surface is still moving. Versions 0.7.3, 0.7.4 and 0.7.5 were published in the last three weeks of the cadence, which is fast iteration on a project that has not reached 1.0.

The fourth is the dependency stack. The Dockerfile installs ffmpeg and enables it with an environment flag, which is what makes the multi-format media handling work, and it needs both a PostgreSQL and a Redis for the documented feature set. The single container quick start avoids that by pointing at defaults, and a team that wants the cache plugin and the audit trail is deploying three services.

## Conclusion

aiproxy is the right tool for a team running several model providers behind one endpoint, where you need retries across channels, per group quotas and pricing, a shared MCP layer, or an audit trail of every request, and the OpenAI compatible protocol means client code barely changes. It is the wrong tool for a single developer with one provider and one key, and it is the wrong tool if you cannot accept a component that sees every prompt and response in plaintext. Before adopting, read the body size and logging retention settings in the configuration documentation, decide whether request and response bodies are written to the log database, and confirm that the plugin and MCP surface is stable enough for the version you would deploy.

## FAQ

### What protocols does the aiproxy gateway accept?

It uses OpenAI, Claude and Gemini protocols as the entry point and performs protocol conversion between OpenAI Chat Completions, Claude Messages, Gemini and the OpenAI Responses API, so a request sent in one protocol can be routed to a model exposed only through another.

### How does aiproxy choose which provider to send a request to?

Through its intelligent router, using channel priority and error rates, with load balancing across multiple providers and automatic error recovery through smart retry strategies. The monitoring side reports error rate analysis and performance monitoring per channel.

### What does the aiproxy cache plugin need?

The cache plugin provides caching for identical requests with Redis or memory storage, so a full deployment needs Redis. The supplied docker-compose.yaml starts PostgreSQL, Redis and the gateway, and the gateway waits for both to pass their health checks before starting.

### Does aiproxy support MCP servers?

Yes, in three forms: public MCP integrations that are ready to use, organisation level MCP servers, and embedded MCP servers with configuration templates. There is also automatic conversion of OpenAPI specifications into MCP tools.

## Sources

- [Issues](https://github.com/labring/aiproxy/issues)
- [labring/aiproxy on GitHub](https://github.com/labring/aiproxy)
- [License: MIT](https://github.com/labring/aiproxy/blob/main/LICENSE)
- [README](https://github.com/labring/aiproxy/blob/main/README.md)
- [Releases](https://github.com/labring/aiproxy/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/labring-aiproxy