OpenLIT: OpenTelemetry Traces for LLM Apps and Coding Agents

The problem OpenLIT targets: agents that are not just LLM calls

A single production agent request can fan out into LLM calls, tool invocations, retrieval, memory reads, sub-agent handoffs, prompt templates and code changes. The README draws exactly this flow, with evaluation sitting between those steps and an output of cost, quality and errors. Standard application tracing does not capture any of it, because the interesting spans are created inside SDKs and agent frameworks rather than in your own handler code. OpenLIT's answer is to instrument those libraries for you and emit OpenTelemetry spans and metrics, so the agent's internal structure shows up as a trace tree instead of a log line saying the request took four seconds. The audience is teams running agents in production or close to it: platform engineers who already own an observability stack, and AI engineers who need to answer why a run cost what it cost. It is not aimed at someone who wants a hosted dashboard with a support contract, since the documented path is a local Docker Compose deployment.

How the instrumentation actually works

The mechanism is auto-instrumentation wrapped around OpenTelemetry. In Python, a single openlit.init() call is documented as instrumenting supported LLM providers, frameworks, vector databases and other AI infrastructure, then exporting traces and metrics over OTLP. The TypeScript package is installed the same way conceptually, via npm install openlit. You do not write spans by hand for the common cases, which is the whole point: the library patches the client libraries you already use. Telemetry leaves the process through the standard OTLP exporter, configured either by the OTEL_EXPORTER_OTLP_ENDPOINT environment variable or the otlp_endpoint argument to init. That is a meaningful design choice. Because the wire format is OpenTelemetry rather than a proprietary agent protocol, the collector endpoint is swappable, and the same spans could in principle land in any OTLP-compatible backend. The evaluation layer sits on top of that data: the README describes LLM-as-a-Judge evaluations with built-in types including hallucination, bias and toxicity, applied automatically to LLM and agent outputs. Cost tracking is derived from token usage and attributed across models, providers, users, sessions, agents and environments, with custom pricing available for custom and fine-tuned models. The README does not document how the judge model is selected or what it costs to run, which is a gap worth noting if evaluation volume matters to you.

Getting the stack and SDK running

The documented quickstart is three commands. Clone the repository, change into it, and run docker compose up -d. The dashboard then answers on http://127.0.0.1:3000. On the application side, pip install openlit for Python or npm install openlit for TypeScript, followed by import openlit and openlit.init(). To point the SDK at the local collector, either export OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:4318" or pass otlp_endpoint="http://127.0.0.1:4318" to init. Port 4318 is the standard OTLP HTTP port, so the stack is listening on the conventional endpoint rather than a custom one. The repository topics list ClickHouse, which suggests it is the storage layer behind the dashboard, but the README does not spell out the service topology inside the Compose file. If you are sizing a deployment, read docker-compose.yml directly rather than inferring from the README.

The coding-agent CLI is the most distinctive part

Coding agents are the newest and least standardised target here. OpenLIT ships a CLI with platform-specific installers: a curl piped to sh on macOS and Linux, and an iwr piped to iex on Windows. Configuration is one command, openlit configure --endpoint http://127.0.0.1:4318. Instrumentation is installed per vendor with openlit coding install --vendor=cursor, --vendor=claude-code or --vendor=codex, or all at once with --vendor=all. A verification step exists: openlit doctor. What gets captured, per the README, is the user prompt, LLM calls, tool calls broken out into file reads, file edits, shell commands and search, sub-agent activity, token usage, cost and code impact, all surfaced in a Coding Agents dashboard. The honest caveat is that this works by hooking into third-party agent tools whose internals are not stable. The presence of a doctor command implies the maintainers expect installations to break or partially attach. Run it after every vendor install and after every upgrade of Cursor, Claude Code or Codex, because a vendor-side change is the failure mode you cannot control from here.

Where OpenLIT is the wrong tool

The OpenTelemetry-native design has a cost. Auto-instrumentation only covers libraries the project has written patches for, and the README does not enumerate them, so a homegrown agent loop or an obscure framework will produce a trace with your own spans and nothing underneath. You would then be writing manual OpenTelemetry instrumentation, which is fine but removes the reason to pick OpenLIT over a generic tracing backend. The second constraint is operational. A self-hosted stack with a database behind it is something you run, back up, upgrade and monitor. For a small team without existing observability infrastructure, that is real work for a tool whose value scales with the number of agents in production. Third, evaluation is LLM-as-a-Judge, which means evaluating outputs consumes model calls and inherits the judge model's biases; the README lists hallucination, bias and toxicity as built-in types without describing the judge configuration, so treat evaluation scores as a signal to investigate rather than a ground truth. Finally, the README documents no retention policy, sampling configuration or access control for the dashboard, so if prompt and response content is sensitive, assume you are storing it in full until you find otherwise in the Compose configuration.

How it differs from a general tracing backend

The obvious comparison is running OpenTelemetry straight into a general-purpose backend such as Jaeger, Grafana Tempo or a hosted tracing service. Those tools accept the same OTLP data and will render your spans, and if all you want is a trace waterfall, they are simpler to operate because you are not standing up another stack. The difference is what happens after ingestion. A general backend knows nothing about tokens, model pricing or judge evaluations. OpenLIT adds a semantic layer over the same spans: cost attribution across models, providers, users, sessions, agents and environments, custom pricing for fine-tuned models, and automatic evaluation types. That layer is the product. The trade-off is that you are adopting a domain-specific UI and its storage alongside your existing tracing, and the README does not describe how the two would be unified. If your AI traces and your service traces need to sit in one view, verify that the OpenLIT dashboard can act as a Grafana data source or that the spans are equally readable in your existing backend before you commit to running both.

Maintenance, releases and licence

OpenLIT is Apache-2.0, which permits commercial use, modification and redistribution, and includes an explicit patent grant. It does not impose copyleft obligations on your application code, so instrumenting a proprietary service with the SDK does not require you to publish that service. That is the general shape of Apache-2.0; it is not legal advice, and if you are redistributing a modified version or embedding the dashboard in a product, read the LICENSE file and the NOTICE requirements yourself. On maintenance, the release cadence visible in the repository shows openlit 2.1.0 and 2.0.0 landing roughly two weeks apart, with a separate otel-gpu-collector package at 0.0.8. A major version bump from 2.0.0 to 2.1.0 in that window means you should read the release notes before upgrading rather than pinning and forgetting. The GPU collector being versioned independently is a hint that it moves on its own schedule and may be less settled than the core. The upgrade cost that matters most is not the Python package; it is the coding-agent integrations, which sit between your editor and the vendor's tool and can silently stop working when either side changes.

Editorial conclusion

Adopt OpenLIT if you already run an OTLP collector or want your AI traces to live in the same pipeline as the rest of your services, and if you can accept a self-hosted Docker Compose stack with ClickHouse behind it. Skip it if you need a hosted service with an SLA, or if your stack is not covered by the auto-instrumentation list. Before committing, run openlit doctor after openlit coding install to confirm the vendor hooks actually attached, and check the SDK's supported provider list against your own dependencies.

Official sources

1. License: Apache-2.0
2. openlit/openlit on GitHub
3. Project website
4. README
5. Releases