diff-cover: the coverage report that only counts the lines you changed

The metric is diff coverage, not coverage

Diff coverage is the percentage of new or modified lines that are covered by tests. The project puts the point bluntly: if you touch a line of code, that line should be covered, and code coverage is every developer's responsibility.

That reframing is the entire value proposition. A project-wide coverage number drifts downward as the codebase grows and tells a reviewer nothing about the change in front of them. A diff-scoped number tells you whether the ten lines you just wrote are exercised by a test, which is a question you can answer and act on before approving a pull request.

The tool itself is deliberately simple in mechanism. The diff-cover command line tool compares an XML coverage report with the output of git diff, then reports coverage information for the lines in the diff. The result is described as a code quality metric used during code reviews.

The same idea extends to style and correctness checks. The tool also finds diff lines that have violations according to tools such as pycodestyle, pyflakes, flake8 or pylint, and the companion command diff-quality exposes that as a separate report. The two commands are declared as separate console scripts in pyproject.toml, diff-cover pointing at diff_cover.diff_cover_tool:main and diff-quality at diff_cover.diff_quality_tool:main, so they can be used independently.

Two requirements that decide whether it applies to you

diff-cover states two conditions up front, and both are hard requirements rather than preferences.

The first is that you are using git for version control. That is not an incidental detail, because the tool works by comparing a coverage report with the output of git diff, and the diff can be produced either from a branch comparison or from a file you supply. The project is also explicitly designed to be extended, and the README points contributors at how to add support for other version control systems, which confirms that git is the built-in case and everything else is work you would have to do.

The second is that your test runner generates coverage reports in Cobertura, Clover or JaCoCo XML format, or LCov format. The listed producers cover a wide range of stacks: Cobertura, Clover and JaCoCo for Java, coverage.py for Python, JSCover for JavaScript, and lcov for C and C++. So the tool is language-agnostic in practice, but it is report-format-bound, and if your coverage tool emits something else there is no adapter for it out of the box.

The Python package metadata reflects that scope. It is classified as Development Status 5, Production/Stable, operating system independent, and requires Python 3.10 or newer, with classifiers covering Python 3.10 through 3.14. Its runtime dependencies are small and unremarkable: Pygments for colouring the output, Jinja2 for the report templates, pluggy for the plugin hook system, and chardet for encoding detection.

A first run against a branch

The installation is a single pip command for the released version:

pip install diff-cover

For a development checkout, the project documents a uv based flow that creates a .venv with the project installed:

git clone https://github.com/Bachmann1234/diff-cover.git
cd diff-cover
uv sync

The commands then run through uv run, for example uv run diff-cover --help, or you activate the environment the usual way with source .venv/bin/activate.

The usage sequence has three steps. You set the current working directory to a git repository, you run your test suite under coverage to generate an XML report, and you run diff-cover against it. With pytest-cov that report step is:

pytest --cov --cov-report=xml

which produces a coverage.xml file in the working directory. The README notes that if you use a different coverage generator you will need different commands to produce the XML.

The final step is the report itself:

diff-cover coverage.xml

By default this compares the current branch to origin/main and prints the diff coverage report to the console. The same command produces HTML, JSON or Markdown output through the format flag:

diff-cover coverage.xml --format html:report.html
diff-cover coverage.xml --format json:report.json
diff-cover coverage.xml --format markdown:report.md
diff-cover coverage.xml --format html:report.html,markdown:report.md

That last line shows that formats can be combined in one invocation, which is what you want in a pipeline that posts an HTML artefact and a Markdown summary to the pull request at the same time.

Choosing the comparison branch and supplying your own diff

The default comparison target is origin/main, which is right for a trunk-based workflow and wrong for anything else. The compare branch is set explicitly with:

diff-cover coverage.xml --compare-branch=origin/release

That flag is the setting most likely to need attention when you adopt the tool, because a repository whose main branch is called master, or whose default branch lives on a different remote, will otherwise produce a diff against the wrong baseline and a coverage number that looks wrong for no clear reason.

There is a second mode for when a branch comparison is not what you want. You may provide a file containing the output of git diff instead of a branch name. The documentation walks through the case of two branches, main and feature, where commits A, B and C were made in that order on the feature branch. To see all changes between feature and main you generate a diff file with:

git diff main..feature > diff.txt

and to see the changes between the feature branch and commit A specifically, you use the same command with a different range. The diff file is then passed to diff-cover instead of a branch name, which is what makes the tool usable in a pipeline where the diff was already computed, for example by a CI system that publishes it as an artefact.

The same applies to combining coverage from several suites. Passing multiple coverage reports counts a line as covered if it is covered in any of them:

diff-cover coverage1.xml coverage2.xml

The number of reports is arbitrary, which matters for a monorepo or a project where unit and integration tests are instrumented separately.

diff-quality and the case for feeding it an existing report

The quality command runs a linter over the diff and reports the violations it finds on changed lines. The checker is named with the violations flag:

diff-quality --violations=<tool>

The supported tools are pycodestyle, pyflakes, flake8, pylint, checkstyle, checkstylexml, ruff.check and clang. The README says more checkers can and should be supported, and points at a section on adding diff-quality support for a new quality checker.

One exclusion is documented explicitly. There is no way to run findbugs from diff-quality, because it operates over the generated Java bytecode and should be integrated into the build framework instead. That is a sensible boundary rather than a gap in the implementation, but it does mean a Java project running findbugs still needs to wire that check in separately.

The more interesting decision is whether to let diff-quality run the checker or to hand it a report you generated yourself. The documentation describes the second path and explains why it is more efficient, since letting diff-quality re-run pycodestyle, pyflakes, flake8, pylint, checkstyle or checkstylexml repeats work the CI pipeline has probably already done. For pylint the report format differs by version, with the parseable format needed for pylint below 1.0 and the message template option for 1.0 and above:

pylint --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" > pylint_report.txt

and then the report is passed in:

diff-quality --violations=pylint pylint_report.txt

The same works for pycodestyle and for several reports at once, for example diff-quality --violations=pylint report_1.txt report_2.txt. Extra checker options go through the options flag, such as excluding migrations directories and asking for statistics.

Reports are produced with the same format flag as coverage, including the combined html:report.html,markdown:report.md form.

Where diff-cover is the wrong choice

The scope of the metric is the limitation to name first. Diff coverage deliberately ignores unchanged lines, so a project with 40 percent overall coverage and 100 percent diff coverage passes every check while most of the code remains untested. That is the intended behaviour and it is a defensible review standard, but it is not a substitute for tracking overall coverage over time, and adopting one without the other can create a false sense of coverage.

The git requirement is the second constraint. A team on Mercurial or another system has no diff to compare against, and the README positions other version control systems as an extension opportunity rather than a shipped feature, so the work is yours.

The report format requirement is the third. If your coverage tool emits a format outside Cobertura, Clover, JaCoCo and LCov, there is no path in. That covers most mainstream runners, but a project with a homegrown instrumentation layer will find the tool does not apply.

The comparison against a straightforward alternative is worth making. A CI step that checks the patch coverage percentage from a coverage service's own diff endpoint does the same job without a new dependency, and the diff-quality half of diff-cover overlaps with what a linter configured to fail on changed lines does in most CI systems. What diff-cover offers instead is a single local command that works the same way across languages and needs no service account, plus the option to run a checker in-process when you do not already have its report.

One deployment note for the plugin system. The dependency on pluggy is what allows extra checkers to be registered, and the project points contributors at documentation for adding a new one, so a team with an internal linter can wire it in as a plugin rather than by pre-generating a report.

Editorial conclusion

diff-cover fits a team that wants a coverage rule applied to pull requests, since it turns a project-wide percentage nobody can act on into a per-line standard for the code that just changed. It does not fit a workflow built on mercurial or another version control system, and it does not replace an overall coverage number for legacy code. Before wiring it into a pipeline, confirm your test runner can emit a Cobertura, Clover, JaCoCo or LCov report, set the compare branch explicitly with --compare-branch if your default branch is not origin/main, and decide whether you want the tool to run the linter itself or to consume a report you generated elsewhere, since the second path is both faster and easier to reproduce.

Frequently asked questions

Official sources

1. Bachmann1234/diff_cover on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases