Lighthouse CI: running Lighthouse on every commit and failing the build on regressions

What Lighthouse CI adds on top of plain Lighthouse

Lighthouse itself is a single audit run: you point it at a URL, it produces a report. That is useful once and awkward repeatedly. Lighthouse CI is the suite around it, and the README describes the goal plainly: making it easy to continuously run, save, retrieve and assert against Lighthouse results. The audience is teams that already have a CI pipeline and a build step, not people looking for a one-off score.

The value shows up in three places. First, running the audit automatically on every push instead of on demand. Second, assertions, so a drop in an accessibility or SEO score can fail the build instead of being noticed later. Third, history, so you can see whether a metric moved because of this commit or was already drifting.

The README also lists variance reduction as a feature: running Lighthouse many times rather than once. That matters because a single Lighthouse run on a shared CI machine is noisy, and an assertion threshold tuned against one run will either flap constantly or be so loose it catches nothing.

How autorun, assertions and the server fit together

The repository is a Lerna monorepo with a packages/ directory, and the root package.json builds only two of those packages: @lhci/server and @lhci/viewer. The command users interact with comes from @lhci/cli, which the README installs globally as @lhci/[email protected]. The root start script runs the CLI directly from source at ./packages/cli/src/cli.js, which is how the project develops itself.

The data flow is a pipeline. The CLI collects a URL or a static directory, runs Lighthouse against it one or more times, uploads the resulting reports to a storage target, then evaluates the configured assertions and exits non-zero if any fail. The storage target is either the temporary public storage the CLI uses by default or your own @lhci/server instance, which is the package that gives you the dashboard and the diff view shown in the README screenshots. The server is a separate deployable, not something autorun starts for you.

That split is the design decision worth noticing. Assertions work without any server at all, so a team that only wants a pass or fail gate can ignore the server entirely. The server earns its place when you want trend lines and per-resource diffs between two versions of the site, which the README lists as a feature.

Installing the CLI and getting a first run

The README's quick start targets GitHub Actions, and it is the shortest path to a working setup. Add this file to your repository as .github/workflows/ci.yml. The workflow checks out the code, installs Node 18, installs dependencies plus the CLI globally at the 0.15.x line, builds the project, then runs lhci autorun. Note that the build step comes before autorun, because autorun needs something to audit.

name: CI
on: [push]
jobs:
  lighthouseci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
      - run: npm install && npm install -g @lhci/[email protected]
      - run: npm run build
      - run: lhci autorun

For a local first run, install the CLI the same way and invoke autorun from the project root. The repository keeps its own .lighthouserc.js at the top level, and the root package.json has a start:server script that passes --config=./packages/cli/test/fixtures/lighthouserc.js, which confirms the config file is the expected entry point.

npm install -g @lhci/[email protected]
lhci autorun

What you should see is autorun discovering your build output or dev server, running the audit, and printing assertion results. If it cannot work out where the site is, it stops and asks for configuration rather than guessing. The README points to docs/getting-started.md for other providers and setups, and to docs/configuration.md for the config surface.

Where Lighthouse CI is the wrong tool

The most common failure mode is environmental, not configurational. Lighthouse scores depend heavily on the machine and the network conditions of the run. A CI runner auditing a development server that has not been warmed up, or a preview URL behind an authentication wall, will produce numbers that vary between commits for reasons that have nothing to do with your code. The README's answer to this is running the audit many times to reduce variance, but repetition costs CI minutes and does not fix a fundamentally unstable target.

A second limit is scope. Lighthouse CI asserts on what Lighthouse measures: performance metrics, accessibility, SEO, offline support and best practices. It does not test your application's behaviour, and it is not a replacement for functional tests. A page can score well and still be broken.

The third case is the server. If your team has no appetite for running and maintaining another service, the @lhci/server package is dead weight. The README's architecture and server documents describe it as a separate component, and the repository's own test:docker script exists specifically to test the server in a container, which tells you it is infrastructure you own rather than a hosted product. There is no homepage listed for the project, so there is no vendor-hosted option to fall back on.

Lighthouse CI compared with the community GitHub Action

The README's related projects section names treosh/lighthouse-ci-action, described there as running Lighthouse CI on every PR with GitHub Actions and requiring no infrastructure. The difference is where the CLI lives. With the official path, you install @lhci/cli in your workflow and call lhci autorun yourself, which means you control the Node version, the build step and any flags. With the community action, that wiring is packaged, so a minimal workflow is shorter but you are one layer removed from the CLI's configuration surface.

Both end up running the same underlying CLI. The choice is about how much of the pipeline you want visible in your own workflow file. If you need a custom build step, a specific Node version, or a matrix of configurations, the explicit install is easier to reason about. If you want a score comment on a pull request with almost no YAML, the action is the shorter route. The README also links adevinta/actions-lighthouseci-compare, which compares the current commit against its ancestor and emits a Markdown table, a job the CLI's own diff view handles inside the server UI instead.

Release cadence, licence and upgrade cost

The last push to the repository was on 2026-03-27, and the repository is not archived. The most recent tagged release is v0.15.1 from 2025-06-26, following v0.15.0 on 2025-06-09 and v0.14.0 on 2024-06-20. That spacing is worth reading carefully: a year passed between v0.14.0 and v0.15.0, so the project does not ship on a tight cadence, and the README's own install command pins to the 0.15.x line rather than a floating latest.

The repository includes a docs/version-policy.md document, which is the place to check before you pin a version in a workflow. The README does not summarise that policy, so treat the pinning decision as something you verify in the document rather than assume.

The licence is Apache-2.0, which permits commercial use and modification with the usual conditions around notices and patent grant. This is a summary of the identifier, not legal advice; if you are redistributing the packages or running the server as a service, have your own counsel read the terms.

Upgrade cost is mostly configuration drift. Because the CLI is installed globally in CI at a pinned range, moving to a new minor line means re-checking your assertion thresholds, since scores and metric definitions come from Lighthouse itself, which versions independently of this suite.

What to check before you make it a required check

The repository's own root scripts are a useful signal about how much surface area exists. The test script runs typecheck, lint and unit tests, and test:quick deliberately skips cli.test.js, which suggests the CLI integration test is the slow one. There is a separate test:docker script for the server. That is a fair amount of machinery for a tool you are about to make block merges.

Start with assertions in a non-blocking mode so you can see what your real scores are across a week of commits, then tighten. The README's feature list includes setting and keeping performance budgets on scripts and images, which is a different kind of assertion from a score threshold and behaves differently under variance. Budgets on resource sizes are far more stable than a performance score on a shared runner, so if you want a check that does not flap, start there.

Finally, decide the storage question early. If you never stand up @lhci/server, your reports live in temporary public storage and your history is limited to what the CI provider keeps. That is fine for a gate. It is not fine if you want to answer whether a metric regressed this month or last.

Editorial conclusion

Adopt Lighthouse CI if your team already runs a Node build in CI and you want performance, accessibility and SEO scores to fail a build rather than sit in a dashboard nobody opens. Skip it if you cannot give the runner a stable, production-like URL, because a Lighthouse run against a cold local dev server produces numbers you cannot compare across commits. Before rolling it out, verify three things in your own repository: that lhci autorun discovers the build output you expect, that your chosen assertion thresholds match the scores you actually get on the first run, and whether you need the @lhci/server package for history or can live with the temporary public storage the CLI uses by default.

Frequently asked questions

Official sources

1. GoogleChrome/lighthouse-ci on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases