astaxie/TokenHub: a private AI gateway for enterprise token governance

The problem TokenHub targets: model access that outgrows raw provider keys

The README states the core problem directly: what happens after every team, application and workflow starts consuming more models and more tokens. The failure mode is familiar. A provider key is copied into a service, then into a notebook, then into a workflow tool, and nobody can say which project consumed which tokens or why the provider bill and the internal estimate disagree. TokenHub positions itself as the governance layer for that stage, not as another proxy. Its four stated pillars are model routing, permission management, token savings and provider reconciliation. The audience is explicit in the role table: users who create project-scoped keys and call models, team leaders who manage project spaces and cost attribution, and administrators who configure providers, routing, RBAC and audit. If your organization has one developer and one key, that structure is overhead. If you have several teams and a finance function asking questions about AI spend, it is the point of the product.

How the routing, key and attribution layers fit together

TokenHub sits between client applications and upstream providers. Clients speak the provider APIs they already use: the README lists OpenAI-compatible endpoints at /v1/chat/completions, /v1/responses and /v1/embeddings, Anthropic Messages at /v1/messages and /v1/messages/count_tokens, and image generation and editing at /v1/images/generations and /v1/images/edits. Upstream, providers connect through a Provider abstraction, with native adapters for OpenAI, Azure OpenAI, Anthropic, Gemini, DeepSeek, Qwen, Codex subscriptions and local models, plus a catalog of 150+ provider templates. The governance state lives in the middle. Project-scoped keys carry team ownership, member permissions, quotas and concurrency controls. Routing policies carry priority, weight, failover order, scenario-aware selection and route health diagnostics. Request logs and usage analytics are attributed to user, project, team, model and cost center, which is what makes the reconciliation claim possible rather than aspirational. The design decision worth noting is that access, routing and fallback are administrator-side policy, so client code does not change when a route changes. That is the architectural bet: centralize the decision, keep the client dumb.

Installing TokenHub and making a first governed call

The README describes a SQLite-first private deployment with native systemd and Docker Compose options, and the repository ships a start.sh at the top level plus a deploy/ directory. The README excerpt does not spell out a full install transcript, so the honest path is to open docs/deployment.md and follow the steps it gives for your target, then docs/postgresql-setup.md if you intend to run more than one instance. The repository does provide start.sh at the top level, which is where the README points readers for launching the stack. What you should see after that is the console with login, overview, API documentation, provider channels, model catalog, routing policies, usage analytics and system settings, which is the tour shown in the screenshots section of the README. The first real task is administrative: configure a provider channel, then a model catalog entry, then a routing policy. After that, a project key is issued in the user workspace. The README lists the OpenAI-compatible chat path as /v1/chat/completions, and that is the endpoint a client calls with the project key. The README does not state the default listening port or the exact request body, so confirm both in docs/deployment.md and the API documentation panel rather than assuming values. Once a call succeeds, it should appear in usage analytics attributed to the project that owns the key, which is the whole reason to route through TokenHub instead of calling the provider directly.

Where TokenHub is the wrong tool

TokenHub is a gateway, so it is a single point of failure for every governed call. The README describes route health diagnostics and failover order, which addresses upstream failure, but it does not document what happens to the gateway itself under load beyond the PostgreSQL guidance for multi-instance deployments. The README is also silent on rollback of routing policy changes, so an administrator who pushes a bad route has no documented undo path. The SQLite-first default is a real constraint: the README states that PostgreSQL supports multi-instance deployments, which implies the default single-file database is not the path for horizontally scaled frontends and backends. If you need that scale, you take on the PostgreSQL setup described in docs/postgresql-setup.md, including connection pool configuration. Two more boundaries matter. Token savings are described as partly future work, since caching-driven savings are listed as a future item, so do not adopt it expecting immediate cache-based cost reduction. And the project is young: the most recent release listed is v0.8.0 from 2026-09-09, so the API surface and policy formats should be expected to move between minor versions.

TokenHub compared with a plain OpenAI-compatible fan-out gateway

The README draws this contrast itself. Many open source AI gateways focus on provider fan-out: one endpoint that can call many upstreams. That solves connection, and TokenHub also does it through the same OpenAI-compatible and Anthropic-compatible routes. The difference is what surrounds the route. In a fan-out gateway, the credential is usually a provider key held by the operator, and attribution stops at the endpoint. In TokenHub, the credential is a project key with team ownership, quotas and concurrency limits, and the log record carries user, project, team, model and cost center. A generic OpenAI-compatible router such as OpenRouter is a hosted service that gives you one API across many models; TokenHub is software you run privately, and its stated purpose is to compare internal usage against provider bills so finance, platform and business teams can explain actual AI cost. That reconciliation goal is the sharpest difference. If you want to reach many models with minimal setup, a hosted router is less work. If you want the routing decision and the cost explanation to stay inside your infrastructure, TokenHub is the closer fit.

Maintenance cost, release cadence and the Apache-2.0 licence

The last push to the default branch was on 2026-09-15, and the release history shows v0.6.0 on 2026-08-17, v0.7.0 on 2026-08-25 and v0.8.0 on 2026-09-09. That is a fast minor-version cadence, which cuts both ways: fixes arrive quickly, and so do changes you have to absorb. Plan for upgrade work on the policy and provider configuration surfaces, since those are where the README concentrates functionality. The repository is not archived. Operationally, the cost of running TokenHub is the cost of running the console, the backend, and a database, plus the administrative time to keep provider channels, model catalog entries and routing policies current. The licence is Apache-2.0, which is a permissive licence and permits commercial use, modification and redistribution with the usual conditions around notices and attribution. That is a factual description of the licence, not legal advice; have your own counsel review obligations if you redistribute or embed it.

Editorial conclusion

Adopt TokenHub if you already have multiple teams and projects consuming models and you need per-project keys, routing policy and usage attribution that finance can reconcile against provider bills. Do not adopt it if you only need a single OpenAI-compatible fan-out endpoint for one developer, since the provider abstraction and the governance layer are the same install and you would carry the console, database and role model for nothing. Before committing, verify three things in the repository: whether the routing policy format in docs/administrator-guide.md covers your failover cases, whether your deployment target matches the SQLite-first path or requires the PostgreSQL setup described in docs/postgresql-setup.md, and whether the provider you depend on appears among the native adapters or only in the template catalog.

Frequently asked questions

Official sources

1. astaxie/TokenHub on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases