Helicone Review: An Open Source AI Gateway and LLM Observability Stack

What Helicone Actually Bundles

Helicone is not a single tool. It is a platform that fuses an AI gateway with observability, evaluation, and experiment tooling. The README positions it for AI engineers who need to monitor, evaluate, and experiment with LLM calls. The core promise is that you can change one line of code, the baseURL in your OpenAI client, and start logging every request. That baseURL points to ai-gateway.helicone.ai, which routes to over 100 models from providers like OpenAI, Anthropic, and Gemini. The same gateway handles automatic fallbacks and intelligent routing. This means Helicone competes in two crowded spaces at once: gateway providers like LiteLLM and observability tools like Langfuse. The integration is deliberately shallow for OpenAI users, but the platform also supports LangChain, Vercel AI SDK, and other frameworks through separate adapters.

The Five Service Architecture Behind the One Line Change

The apparent simplicity of changing a baseURL hides a distributed system. The README lists five services: a NextJS frontend, a Cloudflare Worker for proxy logging, a dedicated server called Jawn built on Express and Tsoa, a Supabase instance for application data and auth, and ClickHouse for analytics. Minio provides object storage for logs. When a request hits the gateway, the worker logs it, Jawn collects and serves the logs, and ClickHouse stores them for querying. This separation is common in observability platforms, but it raises the operational bar for self-hosting. You are not running one process. You are running a web app, a worker, a logging server, a database, an analytics database, and an object store. The README calls manual deployment not recommended, which is a strong signal that the architecture expects orchestration.

Getting Started: Cloud Gateway vs. Self-Hosted Docker

The quick start is genuinely short. You sign up, get an API key, and set the baseURL to https://ai-gateway.helicone.ai. The example shows an OpenAI client with a model field that can be gpt-4o-mini, claude-sonnet-4, or gemini-2.0-flash. That model switching is the gateway's routing feature. For self-hosting, the README points to a docker-compose file in the docker directory. The commands are concrete: clone the repo, cd into docker, copy .env.example to .env, then run ./helicone-compose.sh helicone up. That script is the supported path. The Helm chart exists for enterprise workloads, but access requires contacting enterprise@helicone.ai. That is a limitation for teams that prefer infrastructure as code without a sales conversation.

What You Can Observe: Traces, Sessions, Cost, and Playground

The observability features go beyond raw request logs. Helicone supports tracing for agents, chatbots, and document processing pipelines, with sessions to group related calls. Cost and latency tracking are built in, and the FAQ explains how cost is calculated. The Playground lets you test prompts against traces and sessions. Prompt management is versioned and tied to production data, and prompts can be deployed through the gateway without code changes. The README stresses that prompts remain under your control. There is also integration with PostHog for custom dashboards, which is a one-line export. This is a practical feature for teams that already use PostHog and do not want another dashboard. The fine-tuning story is indirect: Helicone partners with OpenPipe and Autonomi rather than offering its own fine-tuning pipeline.

The Open Source License and the Cloud Catch

Helicone is released under Apache-2.0, which is permissive for commercial use. The repository is not archived, and recent releases are dated August 2025, indicating active development. However, the hosted service is not free beyond a monthly tier of 10,000 requests. The README describes a generous free tier but requires adding credits for continued use. This creates a split identity: the code is open, but the default path is a managed service. Self-hosting avoids the credit system but forces you to operate ClickHouse and Supabase yourself. The license permits that, but the documentation does not hide the complexity. The Helm chart being gated behind an email request also means the production-grade path is not fully open, even if the source code is.

Real Limitations and Failure Modes

The biggest limitation is operational weight. If you self-host, you inherit five services and an analytics database. That is not a weekend project. The README says manual deployment is not recommended, which suggests the team knows the friction. A second limitation is the dependency on the Helicone cloud for the simplest integration. The one-line baseURL change points to their gateway, not your own infrastructure. If you want that same simplicity with self-hosting, you must configure your own gateway endpoint, and the README does not show that setup. A third issue is provider coverage. The README says 100+ models, but the actual list lives on a models page. You must verify that your specific provider, such as a niche local model server, is supported. The gateway is OpenAI-compatible, but not every provider is guaranteed.

Alternatives and How They Differ

The most direct alternative is LiteLLM, which is also an open source gateway that provides a unified API for many LLM providers. The key difference is focus: LiteLLM is primarily a proxy and routing layer, while Helicone pairs that gateway with a full observability frontend, tracing, and playground. If you already have a logging solution and only need routing, LiteLLM is lighter. On the observability side, Langfuse offers tracing and evaluation but does not position itself as an AI gateway. Helicone's bet is that you want both in one deployment. That bet is attractive for small teams, but it means you cannot pick the best gateway and the best tracer independently. The PostHog export is a nod to this, but it does not replace native analytics.

Maintenance and Upgrade Cost

The repository shows frequent releases, with three versions in August 2025 alone. That pace means self-hosters must keep up with upgrades to get fixes and new features. The docker-compose script is designed to simplify updates, but any change to ClickHouse schema or the Jawn server could require migration steps. The README does not describe a migration path, so you should check the changelog before upgrading. The Supabase dependency also means you are tied to that auth and database stack. If your organization standardizes on Postgres without Supabase, you will need to adapt. The license is permissive, but the operational cost is real. For a small team, the cloud free tier might be the pragmatic choice, and self-hosting is only worth it when data residency or cost control becomes a hard requirement.

Editorial conclusion

Adopt Helicone if you run production LLM workloads and want a single gateway for routing, tracing, and cost analytics, especially if you already use OpenAI-compatible SDKs. Skip it if you need only lightweight logging or have no tolerance for operating five services. Before adopting, verify the Helm chart access process, since it is not publicly downloadable, and confirm your provider list against the models page. Self-hosted users should test the docker-compose flow in a staging environment first, because the manual deployment path is explicitly discouraged and the worker and Jawn services add operational surface area.

Official sources

1. Helicone/helicone on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases