Anserini turns Lucene into a reproducible IR research baseline

Reproducibility is the product, not a feature

Most search libraries are built to be used. Anserini is built to be trusted, and the difference shows in the project's own framing: it is a toolkit for reproducible information retrieval research, and by building on Lucene the effort aims to bridge the gap between academic information retrieval research and the practice of building real world search applications.

The word reproducible is doing a lot of work there, and the project's history explains it. Anserini grew out of a 2016 reproducibility study of various open source retrieval engines, published at ECIR 2016, and the project states plainly that its effort aims to be the opposite of a well known alternative approach to IR evaluation. The academic write-ups referenced in the documentation describe Anserini as enabling the use of Lucene for IR research, and as providing reproducible ranking baselines using Lucene.

So the audience is a research group, a university lab, or an engineer who has to answer a question of the form does system A beat system B on collection C. To answer that honestly you need the same collection, the same relevance judgements, the same index configuration and the same evaluation procedure. Anserini's contribution is to package those decisions so that a run can be repeated, which is a much narrower goal than a general search engine and a much harder one to satisfy.

The practical consequence for a user is that the value of the project is concentrated in what surrounds the code: the prebuilt indexes, the evaluation data, the reproduction recipes, and the tooling that keeps a run tied to a specific version of everything. The Java code is the easy part to get, and the parts that determine whether your numbers mean anything are the ones that need attention.

Two setup paths, and the project ships instructions for agents

Installation is offered in two forms, and the documentation is explicit that they are different products.

The quick path is the fatjar. Anserini is packaged in a self-contained fatjar, described as the simplest way to get started, and the instruction is to curl the fatjar. The detailed steps live in a separate installation page for the fatjar. The full path is to clone the repository and set up a complete dev environment, with its own installation page. A third option is the Python route: most Anserini features are exposed through the Pyserini Python interface, and the documentation suggests starting there if you are more comfortable with Python.

Unusually for a research toolkit, the repository ships its own setup guidance addressed to coding agents, as instructions you can hand over directly. The fatjar route is expressed as a single instruction:

read the skill at https://github.com/castorini/anserini/tree/master/.agents/skills/install-anserini-fatjar to download the fatjar for me; no need to install the skill

And the full development environment has its own:

read the skill at https://github.com/castorini/anserini/tree/master/.agents/skills/install-anserini-dev-env to create a dev environment for me; no need to install the skill

The rationale the project gives is worth noting, because it explains the design. A generic request to install Anserini leaves an agent to decide between a full clone and a fatjar fetch, and the skills make that choice explicit while making the process more token efficient. The instruction to not install the skill locally is there because the skill is needed only once.

Beyond installation there are skills for day to day work. The anserini-cli skill covers command line usage after setup, for prebuilt index registry, topics registry, search and REST workflows, and the anserini-reproduction skill is for reproducibility experiments. So the repository treats a search index, the query topics, and the search itself as a workflow that can be scripted, which is a fair description of how a research group actually uses the tool.

Reproducing from prebuilt indexes versus from raw collections

Two separate reference documents describe how to reproduce experimental results, and the difference between them is the difference between minutes and days.

One path is reproduction from prebuilt indexes, and the other is reproduction from document collections. In the first case you take an index that someone else already built, which means the run is fast and the variable you are studying is the ranking system rather than the indexing pipeline. In the second case you start from the raw document collection and build the index yourself, which costs a great deal more time and machine time but is the only way to vary the indexing configuration.

The choice decides what your experiment can claim. If your contribution is a new ranking model or a new fusion strategy, an existing prebuilt index is sufficient and much cheaper. If your contribution is in how documents are indexed, the collection path is unavoidable, and the run is only meaningful if the index build is itself reproducible, which is precisely what the toolkit is for.

The repository layout supports the second mode. The root holds collections/, indexes/, runs/ and logs/ directories alongside src/, bin/ and pom.xml, which is the shape of a working tree where collections are fetched, indexes are built, run files are written and logs are kept. The presence of these as ordinary directories in the repository root, rather than only as documentation, tells you that building an index locally and keeping the run artefacts is the expected workflow rather than an exception.

Two more documentation entry points sit alongside these: a start here page that the project calls the beginning of the onboarding path, and an additional documentation page for users, with a note that coding agents should skip both the user installation and additional documentation sections and use the skills instead.

The tools/ submodule removal broke fixed path consumers

The sharpest edge in this project right now is a restructuring, and it is documented with unusual candour in the project README.

Anserini previously included a submodule checkout at tools/. That was removed in pull request anserini#3382 to eliminate an external dependency. The consequences are spelled out. At commit 43add83, dated 2026-08-09, the directory that used to be tools/ had its topics-and-qrels/ area refactored into separate topics/ and qrels/ directories, and at the same time the companion repository was renamed from anserini-tools to eval, with the change recorded as eval#118.

The stated effect is that this breaks consumers that depend on fetching a stable topics-and-qrels/ path on master. The README is unusually blunt about the fix: the most obvious solution, adding symlinks, will not work, because raw.githubusercontent.com URLs do not automatically redirect. It then quotes an assessment that either every downstream consumer has to be fixed or the data has to be copied, and concludes that there is no good solution.

The stabilisation point is the next commit. Commit 9bfc04b, dated 2026-08-11, was the first commit that pinned a specific commit of the external data, which is what ensures stability, and the associated pull request is anserini#3369. The README states that any state of the repository before that commit is likely broken.

For anyone building against this repository today, that sentence is the operational rule. Clone at or after 9bfc04b, or use a release artefact, and treat any reference to a topics-and-qrels/ path as something to update to the topics/ and qrels/ split.

Maven coordinates, the Lucene version, and a project without tags

The build is Maven, with a pom.xml at the repository root, and the project publishes to Maven Central under the io.anserini namespace, with a badge that tracks the published version. Continuous integration runs through a maven.yml workflow, and code coverage is tracked with codecov against the master branch.

The Lucene version is pinned and visible as a badge, currently v10.5.0, with a link to the Apache archive for that release. For a reproducibility project, that pin is not incidental detail but the kind of thing the whole design turns on. A baseline result is only meaningful relative to the engine version that produced it, so the Lucene release is part of the experimental condition rather than an implementation detail.

The licence is Apache 2.0, stated on the badge and in a LICENSE.txt file at the repository root. Apache 2.0 is permissive and includes an explicit patent grant, which suits a research toolkit that is likely to be embedded in academic or commercial systems, and the paperwork obligation is the standard notice requirement. Read LICENSE.txt for the terms themselves rather than relying on the badge.

One more fact shapes the upgrade story. The repository has no GitHub releases at all, so there is no tag to pin, and the version you get from Maven Central is the only published artefact number. The repository is not archived, and the last push was on 2026-10-14, so the code is moving; the absence of releases means you track the Maven Central version rather than the repository history when you want a fixed point.

Editorial conclusion

Anserini suits a research group that needs to rerun published retrieval baselines and compare systems on identical collections, and it suits an engineer who wants a Lucene based search system with a documented, reproducible evaluation path. It is the wrong choice if you need a stable data path, because the evaluation data moved from a topics-and-qrels directory to separate topics and qrels directories at commit 43add83 on 2026-08-09 and any state of the repository before the pinned commit 9bfc04b on 2026-08-11 is likely broken. Before adopting, clone at or after that pinned commit, decide between the fatjar and the full dev environment, and check which index the prebuilt index registry covers before you plan a reproduction.

Frequently asked questions

Official sources

1. castorini/anserini on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README