pydriller: mining git repositories with Python

What the framework is for

The scope statement is short and the topic list explains the context.

pydriller is a Python framework that helps developers in analysing Git repositories, and with it you can easily extract information about commits, developers, modified files, diffs and source code. The keywords on the repository include mining software repositories and the abbreviation for it, and the project description calls it a framework for MSR.

Mining software repositories is a research field with a specific shape. The question is not how a repository works but how it changed, and the data you need is the join of several things at once: who wrote a commit, when, which files it touched, what the diff looked like, and what the code looked like before and after. Any one of those is easy to get with the Git command line, and the join is where a shell script starts to become a program.

That is what the framework is for. The five items in the list, commits, developers, modified files, diffs and source code, are the joins. A study of how often a particular file is changed together with another, or how a particular author's changes differ in size from everyone else's, or how a project's test coverage evolved, becomes a traversal plus a few lines of Python.

The framing that the authors use for it is worth repeating, because it explains the design. The long description in the packaging metadata says you can extract information from any Git repository and quickly export CSV files. CSV is the lingua franca of this kind of research, and a framework whose output is a table rather than a graph is a framework aimed at statistics.

The licence is Apache 2.0, and the project received funding from the European Union's Horizon 2020 research and innovation programme under a Marie Sklodowska-Curie grant, which is a hint about the audience it was built for.

The traversal is the API

The quick usage example is the whole interface in ten lines, and it is worth reading closely for what it implies.

for commit in Repository('https://github.com/ishepard/pydriller').traverse_commits():

You construct a Repository with a URL, ask it to traverse the commits, and get an iterator of commit objects. Nothing is loaded eagerly, which matters for a repository you do not have locally, since the example points at a GitHub URL rather than a directory.

Each commit exposes the hash, the message and the author, and the modified files are reached through a property that yields file objects with their filename. So the shape of the code is: iterate commits, and for each commit look at what changed.

The interesting design decision is that the repository is constructed with a URL. For a research tool that needs to work across hundreds of repositories, cloning each one by hand is not an option, so the library takes a remote and works with it directly, using GitPython underneath.

That choice has a cost worth knowing about. For a study over many repositories, the sensible pattern is to clone once and traverse locally, and the documentation is where to check whether the same interface accepts a local path, which the example's use of a URL does not settle.

The readme also points to the documentation for more usage examples, and there is a video walkthrough, which suggests the API has a few more layers than the example shows, in particular the filters that decide which commits you get.

Four dependencies, and one of them is a complexity analyser

The runtime dependency list is short enough to read in one glance.

gitpython
pytz
types-pytz
lizard

GitPython is the obvious one, since it is the interface to the Git data. pytz handles time zones, and the separate types package for it is there so that the type checker is satisfied.

The interesting entry is lizard, which is a code complexity analyser. It appears in the runtime requirements rather than in the development ones, which means the library computes complexity metrics as part of its normal work, not as an optional extra.

That is a meaningful signal about the intended use. Complexity metrics per file, per commit, are exactly the kind of quantitative signal that mining software repositories studies look for, and having a complexity analyser as a hard dependency means the framework can produce them without the researcher wiring up a separate tool.

It also means the library is not lightweight in the sense of few dependencies, but in the sense of few and well chosen. Four packages, one of which does something you would otherwise have to do yourself.

The packaging reflects the same care. The setup script reads the requirements from the files rather than duplicating them, pulls the version string out of the package's own init file so there is one place to change it, uses the standard package discovery with the tests excluded, and ships the requirements files as data files so they are available after installation.

The test fixtures live in a zip archive

The contribution guide contains the most unusual requirement in the whole readme, and it is worth understanding before you send a pull request.

The author states plainly that pull requests without tests are not accepted, and that the test suite is run against many small repositories that were manually created to test pydriller, packaged in a zip file. You unzip the archive, pick one of the repositories, and use it to test your feature, or create a new one if none is suitable.

Then comes the catch, and the readme highlights it in bold. If you create a new test repository, you must upload a new zip file that includes your new repository, otherwise the tests will fail.

So the test fixtures are not generated at test time, they are a committed binary artefact, and the test suite depends on the exact contents of that artefact. That is an unusual arrangement, and it has an obvious consequence: the repository grows by megabytes every time someone adds a case, and a pull request that adds a fixture is a pull request that adds a binary blob.

It also has a virtue, which is determinism. A manually created repository with a known history is exactly what a test for a commit analysis tool needs, because you know what the answer should be. Generating fixtures at test time would test the generator as much as the library.

The practical advice for a contributor is to reuse an existing fixture wherever you can, and to read the existing tests for inspiration, which the readme recommends before you write anything new.

Type checking, linting and a CodeQL step too big to run locally

The quality gate is described in detail, and it is the part of the readme that tells you what the maintainer values.

The source code is annotated for type checking following the standard annotation syntax, and the mypy type checker runs on every pull request, so code that does not type check will not pass that build step. The linter is Flake8, also executed on every pull request, and code that does not conform to its style rules will not pass either. The local invocations are given, along with a make target for each.

mypy --ignore-missing-imports pydriller/ tests/

pytest --cov-report term --cov=pydriller tests/

The Makefile is a short summary of the same thing, with a checkall target that runs the type check, the lint and the coverage run in sequence.

The CodeQL entry is the one with a candid explanation. The CI chain executes a set of CodeQL queries about Python quality and security, but a local CodeQL setup is quite big, more than 1.7 gigabytes, and requires multiple setup and configuration steps, so it is omitted from the local instructions and from the Makefile.

That is a good way to handle it. A static analysis step that is impractical to run locally is a step that contributors will not run, and a step nobody runs locally is a step that only bites the people who did not set it up. Leaving it out of the local tooling and saying why is better than pretending.

The rest of the repository is conventional: a coverage configuration, a Flake8 configuration, a readthedocs configuration, a pytest configuration, a manifest file, a Makefile and separate requirement files for the runtime, the tests and the development tooling.

Maintenance, citation and the state of the field

Two closing details.

The first is the release cadence. Version 2.10 came out in July 2026, 2.11 in early September and 2.12 on 2026-09-11, with the last push on 2026-09-26 and the repository not archived. A sequence of numbered releases with a release roughly every month or two is what you want from a research library, because the shape of the Git data it exposes changes over time and a pinned version tells you what you got.

The packaging declares a minimum Python of 3.5, which is worth noting as a curiosity: the library still supports an interpreter long past its own end of life, which is a deliberate kindness to researchers running on institutional machines, and also a sign that the supported surface is wider than the actively tested one.

The second is the citation. The readme gives a BibTeX entry for the ESEC/FSE 2018 paper, with the authors, the DOI, the ISBN, the page range and the ACM proceedings details. For a research tool that is not a requirement, it is a courtesy that tells you the project is aimed at published work, and the citation is a route to the design decisions if you want to understand why the object model looks the way it does.

The other research pointer is external, an exercise on a computer networking course site about using packetdrill for teaching TCP, which is a nice reminder that tools in this space tend to end up in teaching material as often as in research, and that readability of the output format has value beyond the project that produced it.

Editorial conclusion

pydriller suits a researcher or an engineer who needs commit level data across many repositories, because it removes the Git plumbing that usually dominates such a script and gives a documented object model instead. It is a poor fit if you only need to read one repository's log, and a poor fit if you need the raw diff bytes rather than a parsed model, since the library is built around interpretation. Before you build on it, read the documentation for the traversal and filter options, and if you contribute, read the contribution notes carefully, because the test fixtures are shipped as a zip archive that your pull request has to update if you add a repository.

Frequently asked questions

Official sources

1. ishepard/pydriller on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases