MCPJam Inspector: a cross-client test bench for MCP servers

The problem: one server, many clients, no shared test surface

An MCP server does not have one behaviour. The README states the premise directly: clients like ChatGPT, Claude, and Cursor all read your server differently. A tool that returns a well-formed result can still break a widget, an elicitation flow, or an OAuth handshake in one client while working in another. The usual workaround is to open each client by hand, click through the same tool call, and compare what happened from memory. That does not scale past a handful of tools, and it leaves nothing behind for the next person on the team.

MCPJam Inspector targets that gap. It is a testing and evaluation platform aimed at MCP server developers, with the stated goal of running evals on test cases, showing what is breaking, and gating regressions before deployment. The audience is narrow on purpose: people who maintain an MCP server, an MCP app, or a ChatGPT app and need to know how it behaves outside their own development client. If you are writing a client, not a server, most of this tool is pointed away from you.

What actually sits between your server and the model

The mechanism is a client-side harness rather than a proxy you deploy. MCPJam connects to your server and speaks the protocol itself, then renders what came back. In the playground, the README describes three views: Chat, Trace, and Raw. Trace puts every tool call, agent step, and JSON-RPC message on one timeline. Raw shows the underlying messages. That split matters, because the difference between a tool that was never called and a tool that was called and returned an error is invisible in a chat transcript but obvious in a message log.

The playground also emulates the host environment. The README lists support for OpenAI Apps SDK and MCP app UIs, text tools, and a Chrome DevTools-style widget emulator, with a viewport switch between Desktop, Tablet, and Mobile. It claims testing of locale changes, CSP permissions, light and dark mode, hover and touch, and safe-area insets. Those are browser-level concerns that a protocol-level inspector would never surface, and they are where a lot of app breakage lives.

For authorization, the OAuth debugger walks an exchange step by step and visualizes requests. The README states guided conformance checks across four spec versions (2025-03-26, 2025-06-18, 2025-11-25, and the 2026-07-28 draft), with support for client pre-registration, Dynamic Client Registration, and Client ID Metadata Documents. Testing a draft spec version is a real capability, not a marketing line: OAuth implementations break on exactly these version differences.

Evals are the other half. Test cases carry expected tool calls, run across models, and produce accuracy metrics over time. Chat supports comparing up to three models side by side with per-server token usage. The README does not specify the eval scoring algorithm, the storage format for results, or how baselines are pinned between runs, so treat the metric as a signal rather than a certified measurement until you read the docs.

Running it: npx first, hosted app second

The shortest path is the hosted app at app.mcpjam.com, which the README says needs no install. For HTTP and local STDIO servers, the local route is a single command:

npx @mcpjam/inspector@latest

That is the whole quick start as documented. There is no configuration file described in the README, no environment variable list, and no port or transport flag. If you need to point the inspector at a server behind a header-based auth scheme, or run it headless on a build agent, the README does not tell you how. The CLI page and the CI page under docs.mcpjam.com are where that would live, and this review cannot confirm their contents.

The CLI is described as probing servers, running doctor checks, exercising OAuth, and listing tools, resources, and prompts from the terminal. The SDK is described as programmatically driving inspections, snapshotting capabilities, and asserting on tool and resource shapes from your own tests. The CI/CD entry claims conformance, E2E, eval, and OAuth checks on every pull request in GitHub Actions or any pipeline. Exact command names, flags, and assertion APIs are not in the README, so any CI wiring has to start from the docs rather than from this description.

Two operational details are worth noting. Skills are read from your local filesystem and, per the README, never leave your machine, though a project can also carry hosted skills where that is enabled. Workspaces are shared server groups with real-time team sync. Both are team features, and both imply that the hosted product is the centre of gravity for anything collaborative.

Where the design costs you something

The first limitation is that the README does not describe a self-hosted deployment. The npm package runs the inspector locally, but evals, workspaces, hosted skills, and the free frontier-model chat are described as account-dependent surfaces. If your organisation cannot send server definitions or tool payloads to a third party, the local inspector is usable but the evaluation and collaboration features are not, unless the docs describe an on-premise path that the README omits.

The second is reproducibility. Evals run across LLMs and report accuracy over time. Model behaviour drifts, and the README does not state which model versions are pinned per run, how many samples a test case takes, or how a flaky tool call is treated. A single-pass eval against a non-deterministic model produces a number, not a guarantee. Anyone using these results as a release gate needs to know the sampling policy first.

The third is scope. The feature table is long: playground, chat, OAuth debugger, server debugging, skills, workspaces, evals, CLI, SDK, CI/CD. A tool that does all of these will be shallower in each than a dedicated one. If your only problem is that a tool's JSON schema is wrong, you do not need a widget emulator or an OAuth conformance suite. And if your server is a pure STDIO process with no widgets, no OAuth, and no UI, a large part of the surface is inert for you.

Compared with the reference inspector and with plain tests

The README positions MCPJam against the original MCP inspector: the server debugging section says it covers every feature of the original inspector, and more. The difference in approach is the axis of testing. The original inspector is a protocol console. You connect, you list capabilities, you invoke a tool, you read the JSON-RPC. It answers the question of whether the server speaks the protocol correctly. MCPJam keeps that console and adds a client-emulation layer on top, so the question becomes whether a specific host renders and drives your server correctly. The OAuth debugger and the widget emulator are the clearest expressions of that shift.

The other alternative is writing your own tests. The SDK exists precisely for this, and it is the honest comparison: a few assertions in your existing test suite that connect to the server and check tool and resource shapes will catch schema regressions with no account, no model spend, and no new UI. What that approach will not catch is a widget that renders in Desktop but overflows on Mobile, or an OAuth flow that passes against one spec version and fails against another. Those are the cases where the inspector's extra surface earns its place.

Maintenance, releases, and the licence question

The release cadence visible in the material is fast: v3.5.1, v3.5.0, and v3.4.1 all landed on 2026-09-10, and the last push to main is the same day. Three releases in one day is a signal about how the project is developed, not about its quality. It means you should expect to move versions often, and it means pinning a version matters more than usual if you wire the CLI into CI. A fast cadence also raises the odds that a documented flag changes between the docs you read and the binary you installed, so pin the npm version rather than tracking @latest in a pipeline.

On licensing, the material is contradictory and this is the one thing to resolve before adoption. The repository metadata reports NOASSERTION, which usually means GitHub could not match a licence file to a known identifier. The README badge says Apache 2.0 and links to opensource.org. Those two facts can coexist (a modified or non-standard licence file, or a licence file GitHub does not parse), but they can also mean the terms differ from stock Apache 2.0. Read the actual licence file in the repository root and, if the terms matter to your legal team, have them read it too. This is a factual discrepancy in the published material, not legal advice.

Upgrade cost beyond that is hard to estimate from the README alone. There is no migration guide, no changelog excerpt, and no statement about API stability for the SDK. If you build assertions on the SDK, treat its surface as moving until the docs say otherwise.

A concrete way to judge it in an afternoon

Pick the single client configuration that generates the most bug reports for your server. Connect it in the playground, invoke the tool that fails, and read the Trace view rather than the chat output. If the JSON-RPC log shows the call arriving with different arguments than you expected, the inspector has already paid for itself, because that is the class of bug a chat transcript hides. If the trace looks correct and the problem is elsewhere, you have learned that the tool is not the bottleneck.

Then take one OAuth-protected server and run it through the guided conformance checks against two spec versions, for example 2025-06-18 and 2025-11-25. A server that passes both is in good shape. A server that passes one is exactly the case this debugger was built for. Neither step requires an eval run, a workspace, or a team account, so you can evaluate the core before deciding whether the hosted features are worth the data-sharing trade.

Editorial conclusion

Adopt MCPJam Inspector if your MCP server ships to more than one client and you keep getting bug reports you cannot reproduce, because the playground, the OAuth debugger, and the cross-client eval runs are all aimed at that exact failure. Skip it if you want a small offline script that asserts tool schemas in CI with no hosted component and no model spend; the CLI and SDK exist, but the eval and chat surfaces assume accounts and model access. Before committing, verify the licence text in the repository root, because GitHub reports NOASSERTION while the README badge says Apache 2.0, and confirm whether the CI/CD checks you need run against the hosted app or entirely from your own runner.

Official sources

1. Issues
2. MCPJam/inspector on GitHub
3. Project website
4. README
5. Releases