web-platform-tests/wpt: the cross-browser conformance suite and how to run it locally

The interop gap WPT was built to close

Browser engines disagree. A specification can be unambiguous on paper and still produce three different behaviours in Chrome, Firefox, and Safari, and the disagreement usually surfaces in production rather than in review. The web-platform-tests Project exists to make those disagreements visible before they ship. It is a cross-browser test suite for the Web platform stack, covering specs from WHATWG, W3C, and others, and the README frames the goal plainly: tests written so they can run in all browsers give browser projects confidence that they are shipping software compatible with other implementations.

The audience follows from that. This is not a test framework for application teams. It is infrastructure for people who implement the platform: browser engineers, specification editors, and contributors who find an interoperability bug and want to pin it down. The README states that absolutely everyone is welcome to contribute test development, and that no test is too small, especially if it corresponds to an interoperability bug someone has noticed in a browser. That sentence is the clearest statement of who the project is for.

How the suite is laid out and served

The repository is organised by specification area rather than by test type. Top-level directories map to platform features: FileAPI, IndexedDB, WebCryptoAPI, content-security-policy, cookies, cors, and dozens more. A test lives next to the feature it exercises, which means a contributor fixing a cookie behaviour works inside the cookies directory rather than in a shared test tree.

Running those tests requires an HTTP server, because many platform features only behave correctly over a real origin with real headers. The wpt command provides that server, and the README lists wpt serve as the command for starting the wpt HTTP server. A separate command, wpt manifest, updates or generates a MANIFEST.json test manifest, which is the index the tooling uses to find tests. That manifest is a build artifact, not something you edit by hand.

The results side lives outside the repository. wpt.fyi is described in the README as an archive of test results collected from an array of web browsers on a regular basis, and wpt.live is a public deployment of the suite that anyone can run by visiting it from an Internet-enabled browser. The master branch is automatically synced to wpt.live and w3c-test.org. So there are two ways to consume WPT: run it yourself against a local build, or read the aggregated results on wpt.fyi.

Installing WPT and running a first test

The README's setup section is short: clone or otherwise get the repository from github.com/web-platform-tests/wpt. It adds a maintenance note that because branches are created and deleted frequently in this repo, stale branches should be pruned when fetching updates.

git clone https://github.com/web-platform-tests/wpt.git
cd wpt
git config core.autocrlf false

The core.autocrlf setting matters more than it looks. The README warns that git and your text editor must not automatically convert line endings, because that causes lint errors. Setting it in the working tree before you edit anything avoids a class of failures that are tedious to diagnose later.

Once the checkout is in place, the wpt command is the frontend to the tooling. The README lists the most useful subcommands: wpt serve starts the HTTP server, wpt run runs tests in a browser, wpt lint runs the lint against all tests, wpt manifest regenerates the manifest, wpt install installs the latest release of a browser or webdriver server locally, and wpt serve-wave starts the HTTP server together with the WAVE test runner. To lint a change before opening a pull request, the contributing section gives the exact invocation:

./wpt lint

On Windows there is one adjustment. The README states that wpt commands must be prefixed with python or the path to the python binary if python is not on your PATH:

python wpt [command]

The README points to the documentation website for the actual running instructions, specifically the system setup page for running tests locally. That page, not the README, is where dependency installation for each operating system is described. If you are starting from nothing, read it before running wpt run, because the browser and webdriver binaries have to be present first.

The contribution path and its AI policy

The workflow the README describes is conventional: fork the repository, create a topic branch with git checkout -b topic, make your changes, run ./wpt lint, commit and push to your fork, then open a pull request. The lint step is not optional in practice, since the project lints all tests and line-ending problems are a common cause of failure.

The AI policy is the part most likely to surprise a new contributor, and it is more specific than most projects manage. Using LLMs to help author tests is generally acceptable, but every change must be attributable to a human who understands it and can take responsibility for it. Using LLMs to participate in discussions is generally not acceptable. Any LLM-generated content in a discussion must either come from a bot account, or be contained inside a blockquote for short contributions of at most one paragraph, or inside a details element. The README links to a full Policy for LLM Use on the documentation site.

That asymmetry is deliberate and worth reading carefully. Authoring a test is a bounded artifact that a human can review and own. Participating in a discussion is a social act, and the policy treats unattributed machine text there as a problem rather than a convenience.

Where WPT is the wrong tool

WPT tests the platform, not your application. If you want to verify that your checkout flow works, that your API returns the right payload, or that a refactor did not break a component, this project has nothing for you. The tests are written against specifications, and the harness assumes it is driving a browser to check platform behaviour. Pointing it at a product is a category error.

A second limitation is environmental. Running the suite locally depends on having a browser and a webdriver server available, which is why wpt install exists as a separate command. The README does not document rollback for that install step, nor does it describe what happens when a downloaded browser build is incompatible with the tests you want to run. The system setup documentation is the place to check, and the README itself defers to it rather than restating it.

There is also a maintenance cost that is easy to underestimate. The repository is large, organised by specification area, and changes constantly. The README's own advice to prune stale branches with git pull --prune or git fetch -p && git merge is a small signal of how much branch churn the project generates. Cloning it once and returning months later without pruning is a poor experience. And because master is automatically synced to wpt.live and w3c-test.org, anything merged becomes publicly visible quickly, which raises the bar on getting a test right before it lands.

WPT against a browser-specific test suite

The obvious alternative is a vendor's own test suite. Chrome, Firefox, and WebKit each ship internal tests, and those tests are tuned to the engine they live in: they can reach into private APIs, assume the engine's own build system, and encode behaviour that is not yet specified. WPT cannot do any of that, and that constraint is the point. A test in WPT has to be runnable in every browser, which is exactly why browser projects use it to check compatibility with other implementations.

The practical difference shows up when a behaviour is disputed. A vendor suite can assert whatever that vendor currently does. A WPT test has to assert something all engines can agree on, or it fails everywhere and becomes a visible interop problem. That is a harder test to write and a more useful one to have. The trade-off is coverage of engine internals: if you are debugging a rendering pipeline or a JavaScript engine optimisation, WPT is the wrong layer and the vendor's own tests are the right one. Conversely, if your question is whether a feature behaves the same in two browsers, a vendor suite cannot answer it by construction.

Licence and the cost of keeping up

The repository carries a LICENSE.md at the top level, and the project's licence is reported as NOASSERTION by the tooling that classifies it. That means the licence could not be automatically matched to a standard identifier, not that no licence exists. If you intend to reuse test files or the harness in another product, read LICENSE.md directly rather than relying on a classifier, and check whether the per-directory licensing differs, since specification test suites sometimes carry more than one. This is a description of what the repository contains, not legal advice.

The upgrade cost is unusual for an open source project because there is no version to pin. Releases in the listing are named merge_pr_62847 and similar, which are merge artifacts rather than semantic versions. There is no changelog to read and no upgrade path to follow; you track master or you fall behind. For a browser vendor running WPT in CI, that is normal and expected. For anyone else, it means the cost of adoption includes committing to a moving target. The last push was on 2026-09-22, so the tree is current, but currency here is a property of the branch rather than of a release you can hold still.

Editorial conclusion

Adopt WPT if you are implementing a Web platform feature, fixing an interop bug, or need to know whether a behaviour holds across engines; the wpt run and wpt serve commands are the entry points. Do not adopt it as a general-purpose end-to-end test framework for your own application, because it tests specifications, not your product. Before committing, verify which browser and webdriver binaries wpt install can fetch for your platform, check the system setup page for your OS, and read the LLM policy if any part of your contribution is machine-assisted.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Releases
5. web-platform-tests/wpt on GitHub