phoenix

What Phoenix does

Phoenix is an open source AI observability platform built for experimentation, evaluation, and troubleshooting of LLM applications. The README lists the capabilities it provides out of the box. Tracing records an application's runtime using OpenTelemetry based instrumentation, so you can see each step a request takes. Evaluation uses LLMs to benchmark application performance with response and retrieval evals. Datasets let you build versioned collections of examples for experimentation, evaluation, and fine-tuning. Experiments track and compare changes to prompts, models, and retrieval. A Playground helps optimize prompts, compare models, and replay traced LLM calls. Prompt Management adds version control, tagging, and experimentation on prompt changes. PXI, short for Phoenix Intelligence, is an AI engineering agent built into Phoenix that helps debug traces and iterate on prompts. A Remote MCP Server connects clients such as Claude Code and Cursor to the instance's /mcp endpoint to query traces, datasets, and experiments. Together these features cover the loop of building, measuring, and fixing an LLM app. The breadth is the point: rather than stitching a tracer, an eval harness, and a prompt editor from different projects, a team gets one workspace where each step feeds the next, and the data stays in the same place as the application evolves.

Working with frameworks

The README states Phoenix is vendor and language agnostic, with out of the box support for popular frameworks. The listed integrations include the OpenAI Agents SDK, the Claude Agent SDK, LangGraph, the Vercel AI SDK, Mastra, and CrewAI, among others reached through documentation links. This means a team can instrument an existing application without rewriting it for a specific vendor's format, as long as the framework emits OpenTelemetry compatible traces or has a documented bridge. The platform is distributed as a Python package on PyPI and also through conda-forge, and it ships a Docker image plus a Helm chart for deployment. Because it is open source, the code can be run locally for development or self hosted for production use. The breadth of framework support is the practical reason many teams pick Phoenix: they can start with one model provider and switch later without losing their tracing and evaluation history, since the data stays in their own Phoenix instance. The conda-forge and Docker paths also matter for teams that do not use pip directly, because they lower the barrier to a reproducible install. The Remote MCP server rounds this out by letting an agentic coding tool query the same traces the developer sees, which closes the gap between observing a failure and fixing it.

Installing and running

Phoenix is a Python package, and the README points to PyPI and conda-forge as install sources, with a Docker image and a Helm chart for container and Kubernetes deployments. The documentation at arize.com/docs/phoenix covers tracing setup, evaluation, datasets, experiments, the playground, and the remote MCP server. The project also maintains community channels on Slack and social platforms, and it publishes release versions through the package indexes. For a new user, the typical entry point is installing the package, instrumenting an application with OpenTelemetry, and opening the Phoenix UI to inspect traces as requests arrive. From there, evaluation and datasets let the same user turn ad hoc inspection into repeatable checks. The built in PXI agent and the MCP server extend this by letting an engineer ask questions about their own traces in natural language. The project is written in Python and developed in the open on GitHub at github.com/Arize-ai/phoenix, where issues and releases are tracked. The combination of a familiar Python install, standard OpenTelemetry instrumentation, and a UI that works locally means a developer can be looking at their first trace within minutes, then grow into evaluations and experiments as the application matures without changing the underlying observability layer they have already adopted.

Editorial conclusion

The project is written in Python and is developed in the open at github.com/Arize-ai/phoenix.