Budgets are counted after the answer, and the limiter fails open
Enterprise AI bastion host for secure AI API and MCP access, with unified proxying, RBAC, audit logs, rate limiting, and cost tracking across OpenAI, Anthropic, Gemini, and self-hosted LLMs.
At a glance
- What is it?
- ThinkWatch is a Rust gateway that puts a corporate identity provider, spending limits and an audit log between your users and every model provider. Its own documentation is where the interesting parts are: a budget can be overshot by one request, the rate limiter opens when Redis is down, and the production database URL ships with TLS disabled.
- Who is it for?
- Judgment: ThinkWatch is unusually candid about its failure modes, and reading the behaviour notes before deploying is worth more than any feature list. Two defaults deserve a decision rather than an acceptance: the rate limiter fails open when Redis is unavailable, and the generated production DATABASE_URL carries sslmode=disable.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Rust, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 14, 2026, and from our analysis. They are not legal advice.
Star history
- ThinkWatchProject/ThinkWatch817 ★ · +3
| Repository | 2026-09-11 | 2026-11-14 |
|---|---|---|
| ThinkWatchProject/ThinkWatch | 814 | 817 |
3 Stars in the last 64 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
Two ports, and only one of them is meant to be public
The gateway listens on 3000 and the console on 3001, and the README is explicit about which is which. The gateway is the only part clients need to reach; the console serves the management UI and the admin API and belongs behind a VPN or firewall. The diagram shows why the split exists: Claude Code, Cursor, a custom agent and a CI/CD pipeline all point at the gateway, which then fans out to OpenAI, Anthropic, Google Gemini and Azure OpenAI or AWS Bedrock, while an admin browser points only at the console. The gateway is also where format conversion happens, since OpenAI Chat Completions, OpenAI Responses, Anthropic Messages and Gemini requests are all served on that one port and translated to whatever the upstream speaks. Routing spreads traffic by weight, latency or health, and a circuit breaker removes failing upstreams from rotation. In development the frontend runs separately on 5173 and the setup wizard lives at that port:
# 1. Start infrastructure
make infra
# 2. Generate dev secrets + start backend (gateway :3000 + console :3001)
make dev-secrets # writes .env from .env.example with random secrets
make dev-backend
# 3. Start frontend dev server
cd web && pnpm install && pnpm dev
# 4. Complete the setup wizard at http://localhost:5173/setupThe wizard creates the first Super Admin account, sets the site name and issues a first API key; providers get added in the console afterwards.
A request-count limit is checked first, a budget is checked after
The limits and budgets section describes two different check points, and the difference is the operational risk. Request-count limits are checked before the request goes out, so they behave like a wall. Token limits and budgets are counted after the response comes back, which means a single request can push a user past their budget and still succeed; the refusal happens on the next call, not this one. Anyone building a hard ceiling for finance has to accept that the ceiling is soft by one request's worth of spend. The rest of the model is more conventional: sliding windows from one minute to one week can limit either requests or tokens, and daily, weekly or monthly budgets cap spending. Both attach to users, API keys or roles rather than to the account, and rate limits apply to MCP tool calls as well as to model requests, which is what keeps an agent loop from spending through the side door. Budget alerts fire once per period at 50, 80, 95 and 100 percent.
When Redis is unavailable, limits fail open
One line covers the outage case: if Redis is unavailable, limits fail open by default. The setting `security.rate_limit_fail_closed` flips that, so requests are refused instead. That default is a deliberate choice about which failure hurts more, and it is the one to think about before a deployment rather than during one. The rate limiter lives on Redis, which the development infrastructure brings up alongside Postgres, ClickHouse, Zitadel and RustFS, so an unavailable Redis is not a hypothetical: it is a container that has not started. The same reasoning applies to the rest of the stateful set. The ClickHouse audit trail is where request and response bodies land, along with user, parameters, response, latency and errors, and events can be forwarded to a SIEM over Syslog, Kafka through a REST proxy, or signed webhooks. PII redaction of request bodies before storage exists and is off by default.
Tool-call inspection starts in observe mode and stays there until you choose
The security guards have an unusual default. Tool-call inspection starts in observe mode: rules hit, hits are recorded, and nothing is cut off until enforce mode is chosen. Built-in rules can be switched off individually or re-graded, and custom rules can be added. The other guards have their own defaults too. Hidden-character detection defaults to warn and covers Unicode tag characters and bidirectional overrides, in the caller's messages and in tool results, not just in prompts. The content filter ships with rules for common prompt-injection phrases, each set to block, warn or log, and the PII patterns are editable in the console. The PII mechanism itself is a substitution rather than a filter: emails, phone numbers and card numbers are replaced with placeholders before the request goes upstream and restored in the answer, including answers that arrive as a stream. That design keeps the upstream seeing clean text while the model still sees a coherent conversation.
An unconnected user gets error -32050 and a link to fix it
The MCP identity model is the part most likely to surprise an implementer. A user connects their own GitHub, Notion, Linear, Slack or Atlassian account through OAuth or a personal token, so the upstream's own audit log shows the real person acting rather than a shared service identity. Tokens are encrypted at rest, tool lists are cached per user, and each tool can be granted per role and per API key. A user may connect several accounts, work and personal for instance, and each `tw-` key is pinned to one of them. Responses from servers that use per-user credentials are cached per user and account and never shared. The error path is the neat part: a user who has not connected an account still sees the tool list, and calling a tool returns JSON-RPC error `-32050` carrying the authorization URL, which compliant MCP clients can render. Adding a server takes only its URL, since the gateway discovers OAuth endpoints and registers itself when the upstream supports dynamic client registration, and the MCP Store ships 37 ready-made templates.
Secrets are generated from one template, and production Postgres runs without TLS
The environment template is a single source of truth for both development and production, and the file says not to write `.env` or `.env.production` by hand. Two generator modes exist: `bash deploy/generate-secrets.sh --dev` writes `.env`, and `--prod` writes `.env.production` plus a ClickHouse default user XML under deploy/clickhouse/users.d. Both fill secrets through `openssl rand` and choose hostnames to match the topology, localhost in dev and container names in prod, and the syntax supports `# dev:` and `# prod:` tagged lines plus `__SECRET_HEX_<N>__` placeholders substituted at generation time. Two values in that template deserve attention. The production DATABASE_URL points at the postgres container and ends in `?sslmode=disable`, so a generated production file starts with database traffic unencrypted. And DB_MAX_CONNECTIONS is 10, which is a small pool for a gateway whose hot path is database-adjacent. The JWT clock-skew tolerance is capped at 300 seconds, with a note that raising it lengthens every token's expiry window.
The manifest says BUSL-1.1 and the repository's license field is empty
The licensing situation needs reading rather than assuming. The workspace manifest declares `license = "BUSL-1.1"`, and the tree carries a LICENSE file plus LICENSING.md and a Chinese LICENSING.zh-CN.md, so the terms are present as text. The repository's own license metadata, though, comes back empty, which is why the field shows as unrecognised on the project page. The visible README text does not carry a license section at all, so a reader arriving from the repository page rather than the manifest gets nothing. That combination is a documentation gap rather than a contradiction, and it is worth closing because the terms differ by version: BUSL is not MIT, and a company reading only the badge would reach the wrong conclusion about redistribution. One more detail points the same way. The sponsorship line is a mailto whose subject is ThinkWatch Lite Sponsorship, so a Lite edition is referenced in the README without being described anywhere in the visible text.
Version 2.2.0 is both the newest tag and the workspace version
The release cadence is fast and the manifest keeps up. v2.0.0 landed on 24 September 2026, then v2.1.0 at 08:24 on 30 September and v2.2.0 at 16:32 the same day, eight hours apart, and the workspace package version is 2.2.0, matching the newest tag. The workspace has six members: crates/server, crates/gateway, crates/mcp-gateway, crates/auth, crates/common and crates/test-support, which matches the two-port design. Edition 2024 is used with `rust-version = 1.85` pinned as the minimum, deliberately set to the first stable compiler that ships that edition, so that older toolchains get a clear floor instead of confusing parse errors; the toolchain actually used is tracked separately in rust-toolchain.toml, currently stable. The release profile is tuned for the server binary: `lto = true` trims roughly 15 to 20 percent off the stripped binary and helps the gateway hot path, at the cost of a link step of about 30 seconds on Apple silicon and several minutes on x86 CI, with `codegen-units = 1` to unlock it and `strip = "symbols"` so operators load symbols from a separate debuginfo package. The dev profile keeps line tables only, because each integration test is a separate binary linking the whole workspace and full debug info filled about 150 GB of target/ in a day.
Editorial conclusion
Judgment: ThinkWatch is unusually candid about its failure modes, and reading the behaviour notes before deploying is worth more than any feature list. Two defaults deserve a decision rather than an acceptance: the rate limiter fails open when Redis is unavailable, and the generated production DATABASE_URL carries sslmode=disable. Both are configurable, which is the point, but neither is the kind of default that should be discovered during an incident. Before installing, generate secrets with the provided script rather than by hand, set security.rate_limit_fail_closed if your threat model needs limits to hold during an outage, and read LICENSING.md, since the manifest declares BUSL-1.1 while the repository's own license metadata is empty. Anyone planning to run this should also budget for ClickHouse, which is not optional in the infrastructure set.
Frequently asked questions
What is ThinkWatch?
A self-hosted AI API and MCP gateway for organisations, written in Rust. Every model request and MCP tool call passes through one gateway that authenticates against the organisation's identity provider, applies limits and budgets, runs security guards, prices the call and writes an audit entry, taking the role a bastion host plays for server access.
Which ports does ThinkWatch expose?
Two. The gateway on 3000 is the only one clients need to reach, serving OpenAI Chat Completions, OpenAI Responses, Anthropic Messages and Gemini on the same port. The console on 3001 serves the management UI and admin API and is meant to sit behind a VPN or firewall. The frontend dev server uses 5173.
How does ThinkWatch rate limit, and what happens when Redis is down?
Sliding windows from one minute to one week can limit requests or tokens and attach to users, API keys or roles, and MCP tool calls count against them too. If Redis is unavailable the limits fail open by default, and setting security.rate_limit_fail_closed makes requests get refused instead.
What does ThinkWatch record in its audit log?
Every model request and tool call is stored in ClickHouse with the user, parameters, response, latency and errors. Request bodies can be PII-redacted before storage, which is off by default, and events can be forwarded to a SIEM over Syslog, Kafka through a REST proxy, or signed webhooks.
What license is ThinkWatch released under?
The workspace manifest declares BUSL-1.1, and the tree carries LICENSE plus LICENSING.md and a Chinese LICENSING.zh-CN.md. The repository's own license metadata comes back empty, and the visible README text does not name a license.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/thinkwatchproject-thinkwatch)Star history chart for your README
[](https://hysenlabs.com/projects/thinkwatchproject-thinkwatch)Was this useful?