Speedometer: the WebKit benchmark for browser responsiveness

A benchmark for responsiveness, not for speed

Speedometer is a benchmark for web browsers that measures web application responsiveness by timing simulated user interactions on various workloads.

The word responsiveness is doing the work here, and it is a deliberate contrast with the more familiar idea of raw speed. A browser can execute a script quickly and still feel slow, because what the user notices is how long the interface takes to respond to an interaction, which is a question about the whole engine, about scripting, about layout, about painting, and about the scheduler, rather than about one of them.

The stated primary goal is that the benchmark should reflect the real-world web as much as possible, and the readme spells out the consequence: when a browser improves its score on the benchmark, actual users should benefit. That is the bar the design has to clear, and it is a much harder bar than producing a number.

The repository is the open source home of the benchmark, it belongs to the WebKit organisation, and it is run in a browser rather than in a test harness, which is unusual for a benchmark and is the point of the third design rule. The licence field points at a file the hosting platform cannot classify, the manifest records the licence as SEE LICENSE IN LICENSE, and the version in the manifest is 4.0.0-alpha, with no GitHub releases attached.

The last push to the repository was on 2026-09-23, and the engines required are recent, Node 24 or newer and npm 8.19.3 or newer, which is a sign of a project that keeps its tooling current rather than one that pins an old environment.

Three rules that keep the benchmark honest

The readme lists what the benchmark should be, and the three items are rules for the maintainers rather than features for the user.

The first is that it should test end-to-end user journeys instead of testing specific features in a tight loop. Each test should exercise the full set of what is needed from the engine for a user to accomplish a task. That is a statement about what is being measured, because a tight loop around one feature can be optimised without the user noticing anything, while a journey that spans a list, a detail view and a form exercises the engine the way a page does.

The second is that it should evolve over time, adapting to the present web on a regular basis. The readme says this should be informed by current usage data and by consensus about which features are important for engines to optimise, to provide a consistent experience for users and site authors. So the benchmark is expected to change, and the way it changes is supposed to be argued about rather than decided by whoever writes the code.

The third is that it should be accessible to the public and useful to browser engineers. It should run in every modern browser by visiting a normal web page, and it should run relatively quickly while providing enough test coverage to be reflective of the real world.

Those three together are what make it a benchmark rather than a private scoreboard, and they also explain the structure of the repository, which carries a Governance document, a Development document and a Testing document next to the suites.

What a test actually measures

The measurement section is precise, and it is the part that decides whether two scores can be compared.

Each test can contain several steps, and those steps contribute to the duration of the test. The prepare step is unmeasured, and therefore does not contribute to the score. All the following test steps are measured, and they are summed into the test step time.

That distinction matters more than it looks. The prepare step is where the test sets up its state, and it is excluded precisely because setup time is not what a user experiences. If it were included, a test that spent its budget building a large structure before measuring would be penalised for a cost the real interaction never pays.

Each measured step is then divided into two phases. The sync phase is the time spent in synchronous JavaScript execution, and the async phase is the time spent in asynchronous JavaScript, the Promise and setTimeout style tasks that the synchronous step scheduled.

Splitting the step this way is what makes the number diagnostic rather than merely comparative. A step where the synchronous part dominates and the asynchronous part is small points at scripting and at the work done in the task itself. A step where the asynchronous part dominates points at the scheduler and at the work the engine defers. A browser that improves its score by shrinking the synchronous part has done something different from one that improves it by draining the asynchronous queue sooner, and the readme's timeline diagram exists to show that difference.

The measurement timeline is published as an image in the resources directory, so the shape of a run is documented rather than left implicit in the code.

The score is an inverse geometric mean

The scoring rule is stated in two sentences and it is the piece most people misread.

The final benchmark score is calculated based on the inverse of the geometric mean of all the tests, and the score is averaged over multiple iterations with the arithmetic mean.

The reason for the geometric mean is given as an invariant. By using the geometric mean to combine the test durations, relative improvements are favoured equally amongst all tests, even though they can have vastly different durations.

So a test that takes ten milliseconds and a test that takes ten seconds both count for the same when they improve by the same proportion, and a benchmark where one test dominates the total is not possible. That is a deliberate protection against a specific failure mode, the case where a suite gets slower on one heavy workload and the aggregate stays flat because the other workloads improved, or where an engine gains a large improvement on a cheap test that happens to be numerically large in the mean.

The two means are used for two different jobs. The geometric mean combines the tests, and the arithmetic mean averages the runs of the whole suite, so that a single slow iteration of noise does not move the result. The score is an inverse, so that less time is a better score, which is what makes the number comparable with the other things a browser team measures.

The published diagram for the score sits in the resources directory next to the measurement timeline, so the geometry of the aggregation is available to anyone who wants to check it.

Suites, experimental suites and a four browser harness

The repository layout tells you how the benchmark is organised and how it is checked.

There is a suites directory for the workloads themselves and a suites-experimental directory beside it, which is how a benchmark can add a workload that is not yet part of the score. That distinction is the practical form of the evolve over time rule, because a workload under discussion sits in the experimental directory until the consensus rule has been applied to it, and the score does not move while that is happening.

The tests directory holds the harness, and the manifest scripts describe it in detail. The unit tests run through a script that takes a browser name, with test:chrome, test:firefox, test:safari and test:edge, and the end-to-end tests have the same four variants under test-e2e. The default test script runs the Node unit tests with mocha, and the development script starts a local server from the tests directory, which is how you work on a suite without deploying anything:

node tests/server.mjs
node tests/run-end2end.mjs --browser=chrome

There is also a formatting script of its own, run through a Node file in the tests directory, with an all flag, and a separate package.json linter, which is the kind of check that matters in a repository that contains one package.json per suite. The prettier and eslint configurations are at the root, and there is an npm package lint that is separate from them.

The pages are static, index.html, about.html and instructions.html, with a netlify.toml, which matches the rule that the benchmark should run by visiting a normal web page.

Editorial conclusion

Speedometer fits a browser engineer who wants a single number that reflects what a user feels when a page is busy, because the suites are built as end to end journeys and the scoring is designed so that a test that takes a second and a test that takes a minute count the same when they improve by the same proportion. It is the wrong choice if you want to compare two products rather than two engines, because the workloads are deliberately drawn from the open web rather than from a neutral test corpus, and it is the wrong choice if you want a stable number to put in a report, because the benchmark is designed to change as the web changes and a 4.0 alpha is not a fixed point. Before you use it, read the score and measurement section rather than the number, and run the whole harness rather than a single suite, since the score is only meaningful as the combination.

Frequently asked questions

Official sources

1. Issues
2. README
3. WebKit/Speedometer on GitHub