Canary: a QA harness that turns Claude Code browser runs into Playwright scripts

The gap Canary fills between opaque agent runs and hand-written Playwright

Two workflows dominate browser QA right now, and both have a cost. An agent run is fast to start and impossible to audit: the model clicks something, the page changes, and nobody can say which selector it used. Hand-written Playwright is auditable and reproducible, but someone has to write and maintain every spec, and that person has to know the flow before the test exists.

Canary is built for the second half of that trade. According to the README, it reads your code diffs, identifies the affected UI flows, and tests them in real browser instances using Claude Code. The output is not a verdict from a model. It is a session directory containing a screen recording, console logs, network requests, HARs and Playwright traces, plus the Playwright script behind every step.

The audience named in the README is explicit: developers who currently maintain Playwright scripts by hand, QA engineers who click through flows to reproduce issues, and PMs or reviewers who want a self-contained report.html they can open without a build. The PM case is the interesting one, because it treats the recording as the deliverable and the script as the by-product. That ordering is the opposite of most test tooling, where the assertion is primary and the artifact is a log nobody reads.

How the QuickJS WASM sandbox and the session daemon fit together

The mechanism described in the README is a QuickJS WASM sandbox that exposes the full Playwright API. Scripts run inside that sandbox with the Playwright Page API available and, per the README, no Node and no host access. That is a deliberate boundary: the agent writes browser code, and that code cannot touch your filesystem or spawn processes.

Around the sandbox sits a session model. You start a session, then run one or more scripts against it with a step name. Each run is a step. When the session ends, the artifacts are written to a session directory, and the README gives the path as ~/.canary/sessions/<id>/report.html. Capture is on by default and each stream can be turned off individually with --no-trace, --no-video, --no-har or --no-console.

The part worth pausing on is the trace decoding. Playwright traces are a binary-ish archive, not a readable call list. The README states that Canary decodes the full trace into the exact calls the run made, naming goto, waitForSelector, evaluate and screenshot, with params and timing. That decoded call list is what becomes the reusable script. So the data flow is: agent script in, browser session out, trace decoded, script captured for replay. Replay costs no inference, which is the claim the README leans on hardest.

Installing Canary and recording a first session

The README gives two install paths. The direct one installs the CLI and the viewer globally, then runs a one-time setup that pulls Chromium and the runtime into ~/.canary. The README puts that download at roughly 150 MB.

npm i -g @usecanary/cli @usecanary/ui   # puts `canary` + `canary-viewer` on your PATH
canary install                          # one-time: Chromium + the runtime into ~/.canary (~150 MB)

There is also a guided wizard, which the README says offers to install the same pieces for you, including through npx if you would rather not install globally.

npm create canary@latest                # guided setup (Ink wizard)

To record a session, the README's example starts a named session, captures the id, runs two scripts as separate steps, then ends the session. The report lands in the session directory.

id=$(canary session start --name "checkout")
canary run ./open.js   --session "$id" --step open
canary run ./submit.js --session "$id" --step submit
canary session end "$id"                # -> ~/.canary/sessions/<id>/report.html

After that, canary-viewer browses every recorded session, and canary stop shuts the background daemon down. For a throwaway check with no recording, the README pipes a script into canary-browser instead, which prints the page title to stdout.

echo 'const p = await browser.getPage("main");
await p.goto("https://example.com");
console.log(await p.title());' | canary-browser

If you already have Chrome running with --remote-debugging-port=9222, canary-browser --connect attaches to it, which the README suggests for driving a browser that is already logged in. Note that the CLI and UI ship as two separate packages, @usecanary/cli and @usecanary/ui, and the viewer is a distinct binary from the runner.

Where Canary is the wrong tool

Canary keeps an agent in the discovery loop, and that is the first constraint. The captured Playwright script is replayable with no inference cost, but the flow only exists after an agent has driven it once. If your requirement is a test suite that a human writes, reviews and owns line by line, Canary adds a layer you did not ask for. Nothing in the README describes a mode where Canary generates a script without an agent run.

The sandbox is the second constraint, and it cuts both ways. No Node and no host access means a test that needs to read a fixture file, call a local helper, or shell out to seed a database cannot do so from inside the sandbox. The README presents this as a safety property. It is also a ceiling on what a single Canary script can express.

The third is environmental. The install pulls Chromium and a runtime into ~/.canary at roughly 150 MB, and a background daemon runs while you work. There is no retrieved release history for this repository, so the upgrade path is whatever the npm packages publish. The README does not document rollback, and it does not describe how a captured script behaves when the underlying UI changes; a decoded goto or waitForSelector is only as stable as the page it targets. Treat the first captured script as a draft you will edit, not a finished spec.

Canary against plain Playwright and against agent-only browser tools

The honest comparison is with Playwright itself, because Canary is not replacing the browser engine. Plain Playwright gives you a stable, documented API, a mature trace viewer and a CI story that has nothing to do with model inference. What it does not give you is discovery. Someone must know that the checkout flow breaks when a coupon field is empty, and must encode that as a test.

Canary inverts the order. The agent discovers the flow, and the trace decoder converts that discovery into Playwright calls. The difference in approach is where the human sits: in plain Playwright the human is upstream, writing the assertion; in Canary the human is downstream, reading report.html and deciding whether the captured script is worth keeping.

Against a generic agent browser tool, the difference is the artifact. An agent-only tool leaves you a transcript. Canary leaves you a script plus a trace plus a HAR, and the README's framing is that you should not have to choose between the two extremes. That is a fair description of the design, though it does mean Canary carries the maintenance burden of both halves: the agent side when discovery fails, and the Playwright side when the captured selectors drift.

Licence, repository layout and what an upgrade costs

The repository's package.json declares "license": "MIT", while the repository metadata reports the licence as NOASSERTION. Those two disagree, and the discrepancy is worth resolving with the maintainers before you depend on the project commercially. This is a description of what the files say, not legal advice.

The layout is a pnpm workspace driven by turbo, with packages for the browser, daemon, UI and CLI, plus apps, plugins, agents, commands, skills and rules directories, and a .claude-plugin and .cursor-plugin at the top level. The root package.json sets packageManager to pnpm@9.15.0 and engines.node to >=20. The Makefile is a thin wrapper over pnpm and turbo and documents that workspace targets go through turbo --filter so the topological build graph is respected, with the note that building the browser first builds the daemon it embeds. That embedded daemon is the thing that makes upgrades non-trivial: the browser package and the daemon package move together, so a partial upgrade is not a supported shape.

The last push to the default branch was on 2026-06-20. The repository is not archived. There are no retrieved releases, so version 0.4.4 in the root package.json is the only version marker available here. For an upgrade budget, plan on re-running canary install after a version bump, because the runtime in ~/.canary is installed separately from the npm packages.

Editorial conclusion

Adopt Canary if your team already runs Claude Code, Cursor or Codex against a web UI and wants recorded evidence plus a replayable Playwright script instead of hand-written E2E specs. Skip it if you need a deterministic CI suite with no agent in the loop, or if you cannot install Chromium and roughly 150 MB of runtime into ~/.canary. Before committing, verify three things on your own machine: that `canary install` completes, that a session ends with a report.html at ~/.canary/sessions/<id>/report.html, and that the captured script still passes when re-run without the agent. The README does not document rollback, so treat the captured script as the artifact you own, not the recording.

Frequently asked questions

Official sources

1. 0xnyn/canary on GitHub
2. Issues
3. README