ASReview LAB v3: Active Learning Screening for Systematic Reviews

The screening bottleneck ASReview LAB targets

Systematic review screening is a workload problem, not a modelling problem. A search across several databases returns thousands of records, and the reviewer reads titles and abstracts one at a time, marking each as relevant or not. In a conventional workflow the ordering is arbitrary (database order, or whatever the reference manager produced), so the relevant records are spread evenly through the pile and the reviewer cannot stop early with any confidence. ASReview LAB changes the ordering. The README describes it as an open-source machine learning tool for "efficient, transparent, and interactive screening of large textual datasets", used for systematic reviews, meta-analyses, and other systematic text screening. The intended user is a review team, often in health, social science or evidence synthesis, that has a corpus large enough for prioritisation to matter and a protocol that allows a machine-assisted order of reading. The project is developed in the open under an Apache-2.0 licence and carries the Utrecht University topic tag, with a documented scientific basis in Nature Machine Intelligence. The claim is about reading order and reviewer time, not about automating the relevance judgement.

The active learning loop and where the labels go

The mechanism is a loop between the reviewer and a classifier. You import a dataset, create a project, optionally supply prior knowledge (records you already know are relevant or not relevant), and then screen. Each decision you make becomes a training label. The model retrains on the accumulated labels and re-scores the unlabelled records, and the interface presents the next record according to that score. The README states the design position plainly: "Humans are the oracle". The reviewer decides, the model only proposes an order. Two consequences follow. First, the quality of the prioritisation depends on the labels you actually produce, so a reviewer who is inconsistent in the first hundred decisions is training the model on noise. Second, the model is only as good as the text it sees; the README points to pre-configured ELAS models and to building your own with custom components, which implies the feature extraction and classifier are configurable rather than fixed. The repository topics list active-learning, deep-learning, language-model and neural-network, so the model surface is wider than a single default. The documentation linked from the README (the models page under asreview.readthedocs.io) is the place to confirm which combination you are running, because the default matters for reproducibility.

Getting it running: pip, project creation, export

The README gives the installation path directly. Python 3.10 or later is required, then:

pip install asreview

Upgrading is the same command with the flag:

pip install --upgrade asreview

The README points to a separate installation guide for Docker and advanced setups, and to a live demo at asreview.app if you want to see the interface before installing anything. The documented workflow has six steps: import data (CSV, RIS, XLSX and similar formats), create a project, select prior knowledge, start screening, monitor progress on the dashboard, and export the labelled dataset or the project file. Two operational details are worth noting. Prior knowledge is optional, which means a cold start is possible but the first records you see will be poorly ordered until the model has something to learn from. And the dashboard is presented as the place where you "decide when to stop", which is a human judgement the software does not make for you. There is no config key in the README for a stopping threshold, so any stopping rule has to come from your protocol or from a simulation, not from a default.

Simulation as the way to justify a stopping point

The feature that separates ASReview LAB from a generic screening tool is the simulation toolkit, which the README lists as a way to "assess model performance on fully labeled datasets". The logic is straightforward: if you have a dataset where every record has already been labelled, you can run the active learning loop against it and observe how many records the model needed to surface before it found the relevant ones. That gives you an empirical basis for a stopping rule before you spend reviewer time. It also gives you a way to compare models on your own corpus rather than trusting a default. The constraint is that you need a fully labelled dataset of comparable size and topic, which many teams do not have at the start of a new review. A common workaround is to run the simulation on a previous, completed review in the same field. The README does not describe a built-in library of such datasets, so sourcing one is your problem. Treat the simulation as a planning instrument, not as a guarantee: performance on a prior review does not transfer automatically to a different search strategy or a different inclusion criterion.

Duplicate hiding in v3 and what it changes

The README's v3 notes list automatic duplicate hiding as a headline change: records with duplicate titles and texts are hidden during screening, and you can choose to include them again when you export. This is a real workflow change rather than a cosmetic one. In a multi-database search, the same paper often appears several times, and a reviewer who labels one copy as relevant and later meets an identical copy has to recognise it and decide consistently. Hiding removes that repetition, but it also means the screening count you see is not the count you imported. If your PRISMA-style flow diagram reports records screened, you need to know whether that number includes the hidden duplicates, and the export option is where you reconcile the two. The same release adds editable tags managed from the Collection screen, which the README frames as support for data extraction and classification. Tags are a lightweight way to mark why a record was kept, but they are not a substitute for a structured extraction form, and the README does not present them as one.

Where ASReview LAB is the wrong tool

The active learning loop needs a corpus large enough that prioritisation saves more time than it costs. Below a few hundred records, the setup, the prior knowledge step and the model's cold start can easily consume more reviewer time than simply reading everything in order. The tool also screens titles and abstracts; it does not perform full-text assessment or data extraction, so a review that is already past the screening stage gains nothing from it. There is a methodological constraint too. A protocol that requires two independent reviewers on every record, with a measured agreement statistic, is not served by a single reviewer working through a model-generated order, because the order itself depends on that reviewer's decisions. The README's "Humans are the oracle" framing is honest about this: the human is inside the loop, not outside it. Finally, the prioritisation is only as good as the text. Records with sparse abstracts, or datasets where the relevant material is concentrated in a language the model handles poorly, will produce a weak ranking, and the README offers no language coverage claim to rely on. In those cases the dashboard will show progress without showing that the progress is well directed.

How it differs from a reference manager with a screening add-on

The obvious comparison is a reference manager such as Zotero or EndNote combined with a screening extension. Those tools are built around storage, deduplication and metadata, and their screening views present records in a fixed order that the reviewer controls manually. The difference in approach is where the ordering decision lives. In a reference manager, the reviewer decides the order, usually once, before reading. In ASReview LAB, the model decides the order continuously, and the reviewer's decisions change the order as the session proceeds. That is the whole point of the active learning loop, and it is also why the tool needs prior knowledge or a warm-up period to be useful. A second comparison is a general-purpose annotation or text classification pipeline built on scikit-learn or a transformer library. Those give you full control over features and classifiers, and ASReview LAB's custom component support moves in that direction, but you would be building the interface, the label persistence, the export format and the simulation harness yourself. ASReview LAB ships those as part of the package, which is the trade: less control over the stack, more of the review workflow already assembled.

Maintenance, versions and the Apache-2.0 terms

The repository is active, not archived, and the release cadence visible in the supplied material is quick: v3.0.6, v3.0.7 and v3.0.8 landed within roughly three weeks in June 2026, with the last push to main in September 2026. For a review team, that cadence is a reproducibility question rather than a stability one. A methods section that says "ASReview LAB was used" is weaker than one that names the version, and pinning is the practical answer: install a specific release rather than tracking the latest, and record it alongside the model and feature extractor you selected. The upgrade command is a single pip invocation, so the cost of moving versions is low, but the cost of re-running a screening because the model default changed between versions is not. On licensing, the software is Apache-2.0, which permits commercial and academic use and modification, and the README states that the ASReview team "accepts no responsibility or liability for the use of the ASReview tool". That is a disclaimer, not a warranty. If your review is going into a regulatory submission or a clinical guideline, the liability position and the validation evidence you need are questions for your own institution, not something the licence answers. The README also states that no usage or user data is collected, which matters if your corpus is sensitive.

Editorial conclusion

Adopt ASReview LAB if your screening set runs into the thousands of records, you can supply prior knowledge or seed labels, and you have a stopping rule you are willing to defend in the methods section. Do not adopt it for a few hundred records, for full-text extraction, or if your protocol requires every record to be read by two independent reviewers; the tool prioritises, it does not replace that. Before committing, verify three things on your own data: that your file imports with the expected field names, that a simulation run with your chosen model and feature extractor reaches the recall you need at the workload you can afford, and that the version you pin is the one your methods section will name.

Official sources

1. asreview/asreview on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases