Monocle: OpenTelemetry Tracing for Python GenAI Apps, With Trace-Level Tests

The gap Monocle fills: agents that fail without leaving a step-by-step record

A multi-step agent that returns a wrong answer gives you almost nothing to work with. The prompt is in one place, the tool results in another, the retrieval query somewhere else, and the final response is the only artifact anyone kept. Monocle's stated purpose is to close that gap by treating a GenAI run as a trace rather than as a request and a response. The README frames it as a GenAI-specific observability layer built on OpenTelemetry, and the entities it names are agent runs, prompts, responses, tools and vector operations. The intended audience is spelled out in three groups: app developers who do not want to decorate every function with bespoke OpenTelemetry code, platform engineers who would rather wrap an application than ask product teams to refactor it, and enterprises and SREs who want traces that fit existing dashboards and CI pipelines. The third group is the interesting one, because it implies the output is meant to be consumed by something other than a human reading a log file. If your problem is that you cannot tell which tool call produced a bad answer, this is aimed at you. If your problem is that you need a hosted evaluation suite with graded datasets, it is not.

Metamodel plus adapters: where the spans actually come from

The mechanism has two parts and they are worth separating, because only one of them is doing the work. The first is a metamodel, a JSON file at src/monocle_apptrace/metamodel/spans/span_format.json that defines entities such as agents, prompts, responses, tools and vector operations and maps them to standardized span attributes. That file is the contract: it decides what a span for an LLM invocation looks like and which attribute carries which value. The second part is instrumentation adapters, one per framework, which create spans for agent runs, tool calls, LLM invocations and retrieval queries without the application author writing span code. The adapters are what make the metamodel matter. A metamodel with no adapter is a schema document; an adapter with no metamodel produces spans that differ between frameworks and cannot be queried uniformly. The README lists LangChain, LlamaIndex, Haystack, Google ADK, OpenAI and Anthropic as supported through first-class instrumentation, and says the repository contains framework-specific adapters for LangChain, LlamaIndex, Haystack and Google ADK. Note that the two lists are not identical: OpenAI and Anthropic appear in the prose list of supported providers but not in the repository contents list, which names only the four frameworks. That is a documentation inconsistency worth resolving against the source before you plan around it. Export is standard OTLP, so the destination is whatever collector or backend you already run, and the README mentions file, console and cloud storage as export targets.

Getting it running: one install, one call, and a directory of JSON

The quick start is genuinely short. Install with pip install monocle_apptrace. Then call setup_monocle_telemetry once, passing a workflow name: setup_monocle_telemetry(workflow_name="simple_math_app"). According to the README, that call wires up OpenTelemetry, configures the Monocle metamodel, and auto-instruments supported frameworks, so you are not creating spans by hand. By default traces land as JSON files under a local ./monocle directory, named monocle_trace_{workflow_name}_{trace_id}_{timestamp}.json. Each file holds an array of OpenTelemetry spans covering agent runs, tool calls and LLM interactions. The README suggests loading those files into any OTLP-compatible backend, or using the Okahu VS Code extension for a Gantt-style timeline. For attaching business context, the SDK exposes monocle_trace_scope as a context manager and monocle_trace_scope_method as a decorator, both imported from monocle_apptrace.instrumentation.common.instrumentor. A scope such as monocle_trace_scope("user_id", "user-123") applies the attribute to every span created inside the block, which is how you would filter traces by tenant or correlate a multi-step flow. The README also notes async equivalents named amonocle_trace_scope. The file-based default is the part to think about first: writing every trace to disk is fine on a laptop and is a different proposition in a container with a small writable layer, so the export configuration is something you will want to change rather than inherit.

monocle-test-tools: assertions on traces instead of on strings

The companion package monocle-test-tools lives in the test_tools directory of the same repository and is described as a testing and validation framework for AI agent tracing, built on top of pytest. The distinction it draws is the one that matters: you assert on the trace itself, meaning which agents were invoked, which tools were used, what the token costs were, and whether error states appeared, rather than asserting only on an input and output pair. That is a different kind of test from a string comparison against an expected answer. A trace assertion can fail because the agent skipped a retrieval step even when the final text happens to look right, which is exactly the class of regression that output-only tests miss. The README does not reproduce the assertion API in the material available here, so the concrete matcher names and their signatures are not something I can state. Treat the test_tools directory as the reference. The practical consequence is that adopting Monocle for testing means adopting it for tracing first, since the assertions read the spans the instrumentation produced.

Where Monocle is the wrong tool

Two boundaries are clear from the material. The first is language. This repository is the Python implementation, and the README says so directly. If your agent runs in TypeScript, Go or Java, nothing here instruments it, and the metamodel file will not help you on its own. The second is framework coverage. Auto-instrumentation is the value proposition, and it depends on an adapter existing for the framework you use. The README lists four frameworks in the repository contents and adds OpenAI and Anthropic in a separate list of supported providers, so a team on an unlisted framework is looking at writing spans manually, which is the work Monocle exists to remove. There is a third boundary that is less obvious. The README positions Monocle as something that plugs into your existing OpenTelemetry stack, which means it assumes you have one, or are willing to stand one up. A team with no collector and no backend gets a directory of JSON files and a file viewer, which is a real but limited payoff. And the default file export means trace volume translates directly into local disk writes until you reconfigure it. None of these are defects in the design; they are the edges of the design.

Compared with hand-written OpenTelemetry instrumentation

The honest alternative is not another tracing vendor. It is the OpenTelemetry SDK you already have, with spans you write yourself around your LLM calls, tool invocations and retrievals. That approach has no framework dependency, works in any language with an OTel SDK, and produces spans your backend already understands. What it costs is that you decide the attribute names, and so does every other team, which is how you end up with one service tagging a model name as model and another as llm.model and no dashboard that spans both. Monocle's answer is the metamodel: a curated file that fixes the attribute names for GenAI entities and adapters that populate them, so traces from LangChain and Haystack land in the same shape. The trade is a dependency and a schema you do not control, in exchange for consistency you would otherwise have to enforce yourself. If you have already standardized your own GenAI span attributes across services and have the discipline to keep them that way, Monocle is solving a problem you have solved. If you have not, the metamodel is the reason to look.

Maintenance, licensing and what to verify before you commit

Monocle is Apache-2.0 and is described as built under the Linux Foundation AI & Data umbrella, with the README referring to CNCF community involvement. Apache-2.0 is permissive and includes a patent grant, which matters for a component you embed in your own code, but I am not giving legal advice and your counsel should read the terms. On cadence, the release history shows v0.8.12, v0.8.13 and v0.8.14 landing within roughly ten days of each other in August 2026, and the repository's last push is dated 2026-09-10. That is a fast patch rhythm on a 0.x line, which cuts both ways: fixes arrive quickly, and the SDK surface can still move. Pinning a version in your requirements file is the practical response. The upgrade cost is mostly the metamodel, since span attribute names are the part your dashboards and test assertions depend on; a rename in span_format.json propagates into both. Before adopting, do three things against the repository rather than the README. Read src/monocle_apptrace/metamodel/spans/span_format.json and confirm the attributes you would filter on are present. Check the test_tools directory for the assertion API, since the README does not show it. And confirm your framework has an adapter in the source tree, because the README's two framework lists do not agree.

Editorial conclusion

Adopt Monocle if your app is Python, already speaks OTLP, and runs on one of the instrumented frameworks, because then setup_monocle_telemetry(workflow_name=...) is the whole integration and monocle-test-tools gives you assertions on tool calls and token costs rather than on strings. Do not adopt it if your stack is not Python or your framework is not in the adapter list, since the metamodel only pays off when something fills it in automatically. Before committing, read src/monocle_apptrace/metamodel/spans/span_format.json to confirm the span attributes cover the fields you filter on, and check the test_tools directory for whether the assertions you need exist.

Official sources

1. License: Apache-2.0
2. monocle2ai/monocle on GitHub
3. Project website
4. README
5. Releases