Pa11y CI turns accessibility checks into a command with exit codes

One command, three ways of choosing what to test

Pa11y CI is a test runner built using Pa11y and designed for continuous integration environments, and the entry point is deliberately small. You pass one or more URLs.

pa11y-ci https://example.com https://example.com/about

You can point it at a sitemap instead, which is the mode that matters at scale, since the argument takes a sitemap URL and the tool walks it.

pa11y-ci --sitemap https://example.com/sitemap.xml

Or you can point it at local files, using relative paths, absolute paths or glob patterns, and the README asks that glob patterns be quoted so the tool expands them rather than the shell doing it. A single page is passed as a plain relative path, and a quoted recursive pattern is handed to the tool to expand. The same runner is available as a JavaScript module, which is what you want if the check belongs inside a larger test script rather than in a shell step.

const pa11yCi = require('pa11y-ci');

pa11yCi(['https://example.com', 'https://example.com/about'], {}).then((results) => {
    // Use the results
});

The framing is careful about scope. The README states that automated accessibility checks complement manual testing and testing with assistive technologies, and that passing these checks does not guarantee the site is accessible or complies with accessibility standards. That sentence belongs at the front of any evaluation, because the failure mode of tools like this is a team treating a green build as a compliance certificate.

Exit codes do the failing for you

The exit code contract is the part that makes this usable in a pipeline, and it is specified precisely. Exit code 0 means all pages passed, or the total number of reported issues stayed below the command line threshold. Exit code 1 means the tool itself failed for a technical reason, such as an invalid configuration or a sitemap that could not be loaded. Exit code 2 means at least one page failed and the issue count reached or exceeded the threshold. The default threshold is zero, so by default any accessibility error or failed page test produces exit code 2, and only accessibility errors are reported unless you turn on the includeWarnings and includeNotices Pa11y options in your configuration. A threshold can be set for the whole run.

pa11y-ci https://example.com https://example.com/about --threshold 5

There is a subtlety worth knowing, and the README explains it in a separate paragraph. A threshold can also be set in the configuration, either in the defaults or in the options of an individual URL, and those per page thresholds work differently from the command line one. A page passes when its own issue count is at or below its threshold, and its issues do not count towards the command line threshold. The practical effect is that a pipeline can tolerate a small number of known issues on a specific page without weakening the run as a whole.

Configuration is discovered in a fixed order

Pa11y CI looks in the current working directory for a configuration file and uses the first one that exists, in this order: .pa11yci in JSON form, .pa11yci.cjs, .pa11yci.js, and .pa11yci.json. A minimal configuration lists the URLs to visit.

{
    "urls": [
        "https://pa11y.org/",
        "https://pa11y.org/contributing"
    ]
}

Entries in that array can be URLs or local file paths, relative or absolute, though glob patterns are only expanded when they come from the command line rather than from the configuration. A different configuration file can be pointed at with the config option, either JSON or JavaScript. Inside it, options are split by scope, and the README is explicit that most Pa11y options configure individual page tests and belong in a defaults property, where individual URL objects can override them, while the options that control the whole run must be set in defaults and cannot be overridden per URL. Those whole-run options are concurrency, the reporters, the browser launch configuration, and the incognito context switch. The defaults property can also carry a set of Pa11y configurations applied to every page test, for example a viewport.

{
    "defaults": {
        "viewport": {
            "width": 320,
            "height": 480
        }
    },
    "urls": [
        "https://pa11y.org/",
        "https://pa11y.org/contributing"
    ]
}

One browser for the whole run shapes what you can vary

The implementation detail that most affects a large run is that Pa11y CI launches one shared browser, which is why the browser launch configuration cannot vary by page. The same constraint shows up in the concurrency setting, which is the maximum number of page tests run in parallel and defaults to one. A default of one is a deliberate choice for a tool that drives a real browser, since each page test needs a page of its own and a shared context can leak state between them, but it means a sitemap of several hundred URLs will be slow by default and you will want to raise concurrency on a machine with memory to spare. The incognito option is the answer to the state leakage problem. When it is enabled, each URL gets a fresh isolated incognito browser context that is closed once that URL has been tested, so cookies and local storage are not shared and a login performed while testing one URL does not carry over to another. When it is disabled, all URLs share the browser default context, including its cookies and storage.

JSON output for machines, reporters for humans

The output story has two branches, and the distinction matters when you wire the tool into a dashboard. The json flag prints the full report as JSON to standard output instead of the usual console output, which is what you pipe into a file or a parser.

pa11y-ci https://example.com --json > results.json

If you want JSON written to a file while keeping the console output, the README points you at the JSON reporter instead, which is part of the reporter option. The reporter option itself accepts either an npm module or a path to a local file, so a team can write its own reporter to post results wherever they need, and reporters are a whole-run option, so they are set in the defaults of the configuration. The console output is what a human reads in a pull request check, and the JSON is what the automation consumes, and the tool supports both from the same run.

Version 5 moved the toolchain forward and left a migration guide

The 5.0.0 release, published on 2026-10-05, is the current version, and the manifest shows what it demands. The engines field requires Node 22.13 or newer, or 24 and above, and the documentation says a supported LTS version of Node is required and points to a support and migration section for which versions each major release covers. The dependency set is correspondingly current, with Pa11y 10, Puppeteer 25.9, commander 14 and globby 7, and the tooling around it uses c8 for coverage, mocha with sinon and quibble for the unit and integration tests, and eslint 9 with the shared config. The repository carries a MIGRATION.md alongside the changelog and the contributing guide, which is the first place to look when upgrading a pipeline from an earlier major, and a dedicated bin and lib directory, with the command line entry point in bin and the module entry point in lib. The licence is LGPL-3.0-only, the last push was on 2026-10-19, and the release history shows steady work, with 4.1.0 in March 2026, 4.1.1 in May and the 5.0.0 in October.

Editorial conclusion

Pa11y CI suits a team that wants accessibility regressions caught on every build rather than audited once a year, since the exit code contract, the threshold and the JSON output are exactly what a pipeline needs, and the sitemap mode covers a whole site without hand listing pages. It is not a compliance audit, and the README says so in the first paragraph, since passing the automated checks does not mean the site is accessible or conforms to a standard. Verify three things before you wire it into a build that blocks merges: which threshold you will start with, because a default of zero will fail a build immediately on an existing site, which Pa11y options you enable, since only errors are reported by default and warnings and notices are opt in, and whether the shared browser model works for your pages, since one browser instance is launched for the whole run and the launch configuration cannot vary per page. The licence is LGPL-3.0-only and the newest release is 5.0.0 from 2026-10-05.

Frequently asked questions

Official sources

1. License: LGPL-3.0
2. pa11y/pa11y-ci on GitHub
3. Project website
4. README
5. Releases