ACL Anthology: the data and pipeline behind a field's paper index

Three things in one repository

The README states plainly what the repository contains, and the three parts explain why it exists at all.

First, the metadata for all papers, authors and venues, held in the data/ directory, which is what the website at aclanthology.org is generated from. Second, the code and instructions for generating that website, which is why the repository has a hugo/ directory, a Makefile and a justfile. Third, a Python package for accessing the metadata, in the python/ directory, also published on PyPI under the name acl-anthology.

That third part is the one that matters most for reuse. The metadata is not locked inside a static site, it is available as structured data that a script can query, and the README points to a separate README for the Python package for the details. The root pyproject.toml names the project aclanthology.org at version 2026.2.1, and the workspace configuration ties the root to the python/ member, so the root project and the published package are one codebase rather than two things that have to be kept in step.

The licence is Apache-2.0, with a LICENSE file at the root of the repository, and the project is not archived, with the last push on 2026-10-21.

Building the site locally

The prerequisites are specific and two of them are the kind of detail that saves an evening. You need uv, the Python package manager, Hugo 0.160.0 or higher in the extended version, Dart Sass because Hugo needs it, and bibutils for creating non-BibTeX citation formats.

The extended Hugo requirement is called out in the README with emphasis, and it is the kind of detail that is easy to miss and hard to diagnose. bibutils is described as not strictly required to build the website, but without it you have to invoke the build steps manually as laid out in the detailed README.

Cloning and building is short.

$ git clone https://github.com/acl-org/acl-anthology

With the requirements in place, the README says building the website should be as simple as running make from the directory the repository was cloned into.

$ make

The fully generated site ends up in build/anthology, and a local webserver can be launched with make serve, which serves it at http://localhost:8000. The README also mentions running hugo server in the hugo/ directory, noting that it rebuilds the site and therefore takes about a minute to start.

The detailed README, referenced as README_detailed.md, documents the individual steps the top-level target performs, and that is the document to read when a step fails.

The memory cost of the Hugo step

The README is unusually precise about the resources involved, and the precision is worth preserving because the number is alarming on its own.

Building the site is described as quite resource-intensive, and the last step, invoking Hugo, uses about 18 GB of system memory. The build takes about ten minutes on a laptop with an SSD.

The README then does something that many projects do not: it immediately qualifies the figure. It notes that this does not mean you need that amount of RAM, and that the website builds fine on a laptop with 8 GB of RAM, with the system possibly slowing down due to swapping. The 18 GB figure is the maximum RAM usage reported when running hugo --minify --stepAnalysis.

So the practical guidance is that an 8 GB machine works, but will thrash during the Hugo step, and a 10 minute build on an SSD is the expected wait. Anyone planning to build this in CI should size the runner accordingly.

The Makefile gives a way to shortcut part of the pipeline for debugging, since it says the bibtex related targets can be disabled by setting NOBIB=true, for example make -j4 NOBIB=true, and the justfile has a serve recipe that passes NOBIB='true' as well. That is a practical shortcut for anyone iterating on the site rather than producing a full build.

The Makefile and justfile as the interface

There are two task runners in the repository, which is unusual enough to be worth explaining.

The Makefile is the traditional interface and the one the README documents. It starts with the Apache 2.0 header and a set of instructions, including that a command running python should always be invoked as uv run python and that dependencies are added with uv add in the root folder. The Makefile also handles the mirror prefix, exporting ANTHOLOGY_PREFIX ?= https://aclanthology.org, and it enforces a constraint: if the prefix ends in a slash, the build errors out. The reason is given in a comment, that a Perl regex below would fail. The prefix is then split into a host and a directory with sed, because Hugo works out the host and base directory on its own.

The justfile is the newer interface, and its recipes are thin wrappers. The build recipe calls make site. The serve recipe calls make with NOBIB set and then runs hugo server in the build directory. There is a check recipe that calls make check, and an upgrade-hugo-version recipe that rewrites the Hugo version across the GitHub workflow files and the READMEs with sed.

The justfile also has a grobid recipe, which starts a reusable GROBID service for PDF extraction through Docker, with a configurable image, port and platform and a startup timeout, and it fails with a clear message if the Docker daemon is not running. That is a separate concern from the site build, and it is the piece that extracts content from PDF papers, which the metadata pipeline needs for abstract and reference data.

Mirroring the Anthology and its cost

The README is unusually discouraging about mirroring, and the reason given is infrastructure protection rather than licensing.

Creating a mirror is slow and stresses the ACL Anthology infrastructure, because on initial setup you have to download every single file from the official webserver. The README says this can take up to eight hours no matter how fast your connection is, and asks readers not to play around with it just for fun.

The configuration is two environment variables. ANTHOLOGY_PREFIX is the http prefix the mirror will be reachable under, for example https://example.com/my-awesome-mirror, and the README notes there is no slash at the end. ANTHOLOGYFILES is the directory under which papers and attachments will reside on the webserver, which needs to be readable by the webserver but should not be a subdirectory of the mirror directory itself.

With those set, make creates the pages and make mirror mirrors all additional files into the build/anthology-files directory. If a mirror already exists, only the missing files are downloaded, so a resumed mirror is cheap. There is a make mirror-no-attachments target for mirroring the papers without every attachment.

The deployment step is rsync, with the build/website/ directory synced to the webserver, or build/website/FOO if the mirror is served in a subdirectory, and build/anthology-files rsync-ed to the ANTHOLOGYFILES directory.

Where the ACL Anthology is the wrong tool

The first limitation is that the site is an index, not a library of papers. The metadata describes the papers and the site hosts their files as well. A reader looking for a single packaged corpus of PDFs should look elsewhere, and the eight-hour mirror download is a mirror of files, not a redistribution of the articles themselves.

The second is the build weight. Ten minutes and a memory spike that tops out near 18 GB makes this a heavy thing to run on a laptop or a small CI runner, and the people most likely to want to query the metadata locally are the same people with the least spare memory.

The third is that the Python package and the website are two views of one dataset, and the package README is a separate document. A user who only wants the metadata never needs Hugo at all, and a user who only wants the website never needs the package, but the repository presents both and a newcomer has to work out which half they need.

The fourth is the mirror guidance. The README is candid that a mirror stresses the shared infrastructure and can take up to eight hours, which makes it a last resort rather than a normal option, and the no-slash rule on ANTHOLOGY_PREFIX is a sharp edge that a Perl regex failure will teach the hard way.

The fifth is the scope of the dependency list. The root pyproject.toml pulls in lxml, msgspec, nltk, openreview-py, pybtex, pypdf, reportlab, texsoup, gitpython and jsonschema, and requires Python 3.11 or newer. Some of that is needed only for the full site build, and the version floors are exact, such as iso-639 at 0.4.5.

The ACL Anthology compared with Semantic Scholar and with a plain BibTeX dump

Semantic Scholar is the closest thing to a competitor in the same category, and the difference in approach is between a curated, hand-checked index and a computed graph.

The Anthology's index is built around conference proceedings. A paper belongs to a venue, a volume, a year, with an editorial board and a specific set of reviewers, and the metadata reflects that structure because the Anthology is the record of the publication process itself. Semantic Scholar builds a citation graph and computes influence measures over it, which gives it coverage beyond the ACL venues, including preprints and other publishers, and gives it ranking signals that the Anthology does not attempt.

For a bibliometric study restricted to the field, the Anthology's structure is more precise, because a venue volume is a real unit of analysis and the Anthology treats it as one. For finding the most cited work on a topic across all of computer science, Semantic Scholar is the tool that answers the question.

The third comparison is with a plain BibTeX dump, which is what many researchers actually use. A BibTeX file is a list, and it is portable, and it works with any reference manager. What it lacks is the graph: no author identity resolution across papers, no venue structure, no separate tables you can join against. The Anthology's data/ directory is that same information normalised, and the Python package exists to let a script do the joining.

For a literature review tool, that normalisation is the difference between a citation list and a dataset.

Licence, automation and the deployment path

The licence is Apache-2.0, with the terms in the LICENSE file at the root of the repository, and the Makefile carries the same Apache 2.0 header with the copyright attributed to Arne Köhn for the 2019 to 2021 work.

Deployment is described as being handled by a GitHub Actions action that performs the deployment directly from GitHub. To use it, a repository setting is required, the PUBLISH_SSH_KEY secret in standard pem format for authentication without a passphrase, added through the settings interface. GitHub then builds and deploys the current master whenever the master branch changes, and the deployment is done through the upload target in the Makefile.

The repository carries the standard set of development infrastructure for a project of this size: a.pre-commit-config.yaml, a.python-version file, a justfile, a.ackrc, a.codecov.yml, an AGENTS.md, a CLAUDE.md, a uv.lock for reproducible dependency resolution, a google-cse/ directory, a bin/ directory of command line tools, a tests/ directory and a docs/ directory.

The last point worth making is about the dependency on external services. The dependency list includes openreview-py, which is how the pipeline talks to OpenReview, and that is where the submission metadata for many recent papers comes from. An external platform changing its API is a maintenance event for this pipeline, and the presence of that dependency is the kind of thing a maintainer of a mirror or a fork needs to watch.

Editorial conclusion

ACL Anthology suits a researcher who needs a citable, structured index of the field's papers and their authors rather than a scraped copy of a website, and a team that wants to build bibliometric tooling on top of a maintained metadata source. It is the wrong tool for bulk reading, since the site is a metadata index rather than a reading interface, and the wrong choice for a team needing a mirror for anything other than serious use, because the README says a first mirror can take up to eight hours and stresses the Anthology infrastructure. Before building, read the detailed README, because the basic make target hides the individual steps that the detailed document explains, and note that Python 3.11 or newer is required.

Frequently asked questions

Official sources

1. acl-org/acl-anthology on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README