MCPJam inspector: a testing and evals platform for MCP server developers

What MCPJam inspector is for

MCP servers behave differently depending on the client reading them. The README states this directly: ChatGPT, Claude and Cursor each read a server differently, and MCPJam is built to catch those differences and show where a server breaks. The audience is narrow and specific: developers who ship MCP servers, plus the people testing MCP apps and ChatGPT apps built on the OpenAI Apps SDK. The repository topics list mcp-server, mcp-inspector, mcp-clients, oauth2 and evals, which matches that audience. If you are writing a server and want to know whether it works, this is the tool category. If you are consuming an MCP server rather than publishing one, most of the surface here (conformance checks, eval scoring, regression gating) is aimed past you.

How the inspector, playground and trace views fit together

The platform has three overlapping surfaces. The Playground is a cross-client chat interface that emulates UIs, tool calls and skills, with a Chrome DevTools-style widget emulator and Desktop, Tablet and Mobile viewports. The README says it supports the OpenAI Apps SDK and MCP app UIs, text tools, locale changes, CSP permissions, light and dark mode, hover and touch, and safe-area insets. Chat is a multi-server interface that runs on frontier models for free or with your own API key, and compares up to three models side by side while showing each server's token usage. Underneath both, the trace view records every tool call, agent step and JSON-RPC message in one timeline, plus window.openai messages. That trace layer is the actual product. The chat windows are a way to generate traffic; the timeline is what tells you why a call failed. Server Debugging covers the manual path: run tools, resources, resource templates, prompts and elicitation flows by hand with full JSON-RPC logs. The README calls this "every feature of the original inspector, and more", which positions MCPJam as a superset of the reference MCP inspector rather than a fork of it.

Installing MCPJam and running a first inspection

The fastest path needs no install at all. The README points at the hosted app at app.mcpjam.com for that. For HTTP, SSE and local STDIO servers, the README gives a single npx command. Run it from a terminal in the project you want to test:

npx @mcpjam/inspector@latest

That fetches the latest published version of the inspector package and starts it locally. The README does not print the URL or port the local server binds to, so read the terminal output after the command finishes. The repository is a private npm workspace named mcpjam-workspace, and its build script compiles the packages in sequence: @mcpjam/evaluators, @mcpjam/sdk, @mcpjam/cli, @mcpjam/vitest, @mcpjam/chat-ui, @mcpjam/widget-react, then @mcpjam/inspector. If you clone the repository instead of using npx, that ordering matters, because the inspector depends on the packages built before it. A partial build will not produce a working inspector.

npm run build:packages
npm run build:inspector

The first script builds the evaluators, SDK and CLI workspaces. The second builds the inspector alone. The root typecheck script runs the SDK typecheck before building it, then typechecks the CLI, vitest, design-system, chat-ui and widget-react workspaces and a fast typecheck on @mcpjam/mcp, so a clean checkout can be validated before you touch the UI. Once the inspector is running, the README's first real task is to point it at a server and open the trace view: invoke a tool to render its widget instantly, or drive the server with an LLM, and watch the JSON-RPC messages appear.

OAuth conformance and the version problem

The OAuth Debugger is the most concrete differentiator in the README. It visualizes OAuth and EMA requests step by step and runs guided conformance checks across protocol versions 2025-03-26, 2025-06-18, 2025-11-25 and the 2026-07-28 draft. It supports client pre-registration, Dynamic Client Registration and Client ID Metadata Documents. If your server sits behind an authorization flow, this is the part that will save you time, because OAuth failures in MCP are usually a mismatch between what the client expects and what the server implements, and a step-by-step view of the exchange is the only sane way to find it. The trade-off is scope: the README lists four protocol versions, and a server pinned to an older draft that is not in that list will not get a guided check. The README also does not describe what happens when a check fails, whether it produces a diff, a report file, or only a visual state. Treat the debugger as an interactive tool first and a compliance gate second until you confirm the output format.

Evals, the CLI and CI/CD gating

Evals are test cases with expected tool calls, run across LLMs, with accuracy metrics tracked over time. The CLI probes servers, runs doctor checks, exercises OAuth, and lists tools, resources and prompts from the terminal. The SDK lets you drive inspections programmatically, snapshot capabilities, and assert on tool and resource shapes from your own tests. CI/CD runs conformance, end-to-end tests, evals and OAuth checks on every pull request in GitHub Actions or any pipeline. Read together, these four form a progression: the CLI for a quick manual probe, the SDK for assertions inside your existing test suite, evals for behavioural scoring across models, and CI for blocking regressions. The weak point is that the README names the capabilities without showing the command syntax or the assertion API. You will need the docs at docs.mcpjam.com/cli/overview and docs.mcpjam.com/sdk before you can write a pipeline, and the README does not document rollback or how to quarantine a flaky eval, which matters once a check can fail a merge.

Where MCPJam is the wrong tool

MCPJam assumes a server worth testing repeatedly. For a throwaway local server you poke at once, the hosted app plus the npx command is more surface than you need, and a plain JSON-RPC client will get you further faster. The cross-client claim is also bounded by what the README lists: 16 client configurations and 170+ models. If your users are on a client that is not one of those configurations, an eval score tells you nothing about them. Skills carry a second constraint worth reading closely. The README states that local skills are read from your filesystem and never leave your machine, while a project can also carry hosted skills available on accounts where that is enabled. That is a real split in behaviour between local and hosted use, and the README does not spell out what the hosted path transmits. Teams with strict data handling rules should confirm that before enabling hosted skills. Finally, the repository metadata reports the licence as NOASSERTION while the README badge says Apache 2.0, and the README does not reconcile the two. Check the LICENSE file at the repository root rather than either badge.

Alternatives and how the approach differs

The obvious comparison is the reference MCP inspector, which the README itself invokes when it describes Server Debugging as "every feature of the original inspector, and more". The reference inspector is a single manual debugging surface: you connect, list capabilities, and invoke them. MCPJam keeps that manual path and adds the parts the reference tool does not cover, namely cross-client emulation, model-driven chat, eval scoring over time, an OAuth conformance walker and CI gating. The cost of that addition is a larger surface: a hosted app, a local npx package, a CLI, an SDK and a vitest integration, each with its own docs page. If your need is genuinely just "list the tools and call one", the reference inspector is smaller and has fewer moving parts. If you need to know whether a tool call succeeds under Claude but fails under Cursor, the reference inspector cannot answer that question at all, because it does not emulate multiple clients. The repository also contains slack-app and discord-app workspaces, which suggests the team is extending the same testing surface into chat platforms, though the README does not document what those do.

Maintenance, releases and what adoption costs

The last push to the repository was on 2026-09-10, and the most recent releases are v3.5.1, v3.5.0 and v3.4.1, all dated 2026-09-10. The repository is not archived. That is recent enough to treat the project as one that is being worked on, and the release cadence within a single day suggests small, frequent version bumps rather than long release cycles. The upgrade cost depends on which surface you adopt. If you only run the npx command, you track @mcpjam/inspector and the version string is in the command itself. If you build against the SDK or wire the CLI into CI, you are tracking @mcpjam/sdk and @mcpjam/cli as well, and the root build script shows those packages are built and versioned together with the inspector in one workspace. A breaking change in the SDK can therefore land alongside an inspector release. The repository carries a .changeset directory, which indicates the project uses changesets for versioning and changelog generation, so per-package release notes should exist. On licensing, the README badge links to Apache 2.0 and the repository metadata says NOASSERTION. The two do not agree, and this article cannot resolve which governs. Read the LICENSE file before you depend on the terms.

Editorial conclusion

Adopt MCPJam if you maintain an MCP server and need to see how different clients and models actually drive it, or if you want conformance and eval checks running on pull requests. Skip it if you only need a one-off manual poke at a local server, since the hosted app and the CLI overlap there and the README does not describe a lighter path. Before committing, verify the CLI and SDK package names on npm, confirm which OAuth protocol versions your server must support, and check the LICENSE file, because the repository metadata says NOASSERTION while the README badge says Apache 2.0.

Frequently asked questions

Official sources

1. Issues
2. MCPJam/inspector on GitHub
3. Project website
4. README
5. Releases