neuml/annotateai: LLM paper annotation that highlights sections instead of summarizing them

The problem annotateai targets is reading, not summarizing

Most LLM paper tools produce text about a paper. You paste an arXiv link, you get a paragraph, you read the paragraph instead of the paper. annotateai takes the opposite position. The README states the project focuses on "providing human readers with context as they read," and the output is a marked-up PDF rather than a generated essay. That distinction matters because a summary collapses a 30-page paper into something you cannot check against the source. Highlighting keeps you inside the document. The audience is narrow and specific: researchers, clinicians and engineers who already read medical and scientific papers in volume and want the machine to point at the passage about, say, hallucinations or retrieval, not to tell them what the paper concludes. The README says the tool works on any PDF but "works especially well for medical and scientific papers," and it points at arXiv, PubMed, bioRxiv and medRxiv as sources it handles. If your reading is mostly contracts, slide decks or scanned forms, nothing in the documentation suggests this is aimed at you.

How the annotation pipeline actually runs

The mechanism is a five-stage pipeline, and the README lists it plainly. The tool reads the paper, finds the title and important key concepts, walks each page looking for the sections that best emphasize those concepts, reads each selected section and builds a concise short topic for it, then annotates the paper and highlights those sections. So the concepts are extracted first, then used as a retrieval target against the page text, then compressed into a label, then rendered. This is why the keywords parameter exists and why it changes results so much: supplying keywords replaces the automatic concept extraction with your own retrieval target, which the README describes as useful when you have a large batch of papers and want a specific set of concepts identified for a review. Underneath, the project is backed by txtai, and it accepts any txtai-supported LLM. That means the model is a plug, not a fixed dependency, and the annotation quality moves with whatever model you plug in. The architecture image in the repository shows the flow, but the README does not document the scoring function that decides which page section "best" emphasizes a concept, nor the thresholds. Treat the selection step as opaque unless you read the source under src/.

Install annotateai and annotate a first paper

Installation is a single pip command. The README recommends a Python virtual environment, and Python 3.10 or newer is required.

pip install annotateai

The library installs with nltk, tqdm, txtai and txtmarker as dependencies, according to setup.py. The README also documents installing straight from GitHub when you want unreleased features:

pip install git+https://github.com/neuml/annotateai

Before you can annotate anything you have to choose a model, because the path to the LLM is the primary input parameter. The README calls Qwen/Qwen3-4B-Instruct-2507 the "lightweight but powerful default model." Constructing an Annotate instance with that model and passing an arXiv PDF URL is the whole first run:

from annotateai import Annotate

annotate = Annotate("Qwen/Qwen3-4B-Instruct-2507")
annotate("https://arxiv.org/pdf/2005.11401")

What you should see is a progress bar while the pipeline runs, then an annotated PDF with the selected sections highlighted. The README shows this exact call against the RAG paper as its worked example. If you want to steer the highlighting, pass keywords instead of letting the model pick concepts, and disable the progress bar when running in a batch:

annotate("https://arxiv.org/pdf/2005.11401", keywords=["hallucinations", "llm"])

The README also documents the progress parameter separately, as a call with progress=False.

annotate("https://arxiv.org/pdf/2005.11401", progress=False)

For a browser interface rather than a script, the project publishes a Docker image. The README gives this as the default launch, mapping port 8501:

docker run -d --gpus=all -it -p 8501:8501 neuml/annotateai

The same command accepts LLM, MAXLENGTH and n_ctx as environment variables, which the README demonstrates with a Gemma GGUF model. The application code lives in the app folder of the repository.

The model choice is the real cost, and the documentation is thin on it

annotateai does not ship a model. Every example in the README requires either a local download or an API account. The default Qwen3-4B model is small enough to be called lightweight, but the Docker command assumes --gpus=all, so the packaged web application expects a GPU. If you go the API route, the README lists gpt-5.5, claude-opus-4-6-20260205 and gemini/gemini-3-pro-preview as valid LLM strings, and those calls cost money per paper. There is a middle path through Ollama or a llama.cpp GGUF file from the Hugging Face Hub, which the README shows, and llama.cpp support requires running pip install llama-cpp-python first. The MAXLENGTH and n_ctx environment variables exist because context windows are finite. A long paper has to be chunked, and the README does not state how chunk boundaries interact with the section-selection step. That is the sharpest limitation here: the tool's core judgement, which passage best emphasizes a concept, depends on a model whose context you are truncating, and the documentation does not describe what happens to a passage that falls across a boundary. The README also does not document rollback, retry behaviour or what the pipeline does when a PDF fails to parse. For a batch job over hundreds of papers, that silence is the thing to test before you trust it.

Where annotateai is the wrong tool

If you need reproducible, auditable output, this is not it. Two runs with the same paper and the same automatic concept extraction can differ, because the concepts come from a generative model and the selection step is model-dependent. A systematic review that has to document its screening method cannot rest on highlights that shift between runs. Supplying the keywords parameter removes one source of variance but not the selection step underneath it. The second wrong-tool case is scale without hardware. The README's own Docker invocation asks for all GPUs, and the API examples bill per call, so an annotate-everything workflow on a laptop CPU is not a configuration the documentation supports. Third, if what you actually want is a chat interface over a paper, annotateai deliberately does not build one. It produces a marked-up document. Researchers who want to ask follow-up questions of a paper are better served by a retrieval-augmented chat stack, and the README's own example paper is about exactly that.

How it differs from a RAG-over-papers stack

The obvious alternative is a retrieval-augmented generation pipeline built on txtai, LlamaIndex or LangChain, where you embed a corpus of papers, retrieve chunks at query time and generate an answer. The difference is the direction of the interaction. A RAG stack is query-driven: you ask, it retrieves, it answers, and the output is text you have to trust. annotateai is document-driven: you hand it a paper, it finds the concept-bearing passages and marks them, and the output is the paper itself with highlights you can read in context. There is overlap in the machinery, since annotateai is backed by txtai and uses vector search internally, but the deliverable is inverted. A RAG stack is the right choice for a literature question that spans many papers. annotateai is the right choice for one paper you are about to read and want pre-marked. Note also that annotateai is not an annotation platform in the data-labeling sense despite the name; it does not manage label sets, annotator agreement or training data export.

Maintenance, licence and upgrade cost

The repository is not archived, and the last push was on 2026-05-05. Releases are sparse: v0.1.1 in December 2024, v0.2.0 in December 2024, v0.3.0 in December 2025, while setup.py declares version 0.4.0, so the working tree sits ahead of the published releases. Expect to track master rather than wait for tags if you want current behaviour. The licence is Apache-2.0, declared in setup.py and present as a LICENSE file at the repository root. Apache-2.0 permits commercial use and modification and includes a patent grant, but it also requires that you preserve notices and state changes; if you fork the pipeline into a product, read the licence text rather than relying on this summary. The dependency surface is small (nltk, tqdm, txtai, txtmarker), but txtai is the versioned constraint at >=9.2.0 and it is the component that carries the LLM backends, so an upgrade of txtai is the upgrade that can break your model string. Pin it.

Editorial conclusion

Adopt annotateai if you read a steady stream of arXiv, PubMed, bioRxiv or medRxiv PDFs and want concept-level highlighting rather than another chat summary. Skip it if you have no local GPU or no LLM API budget, or if you need deterministic, auditable output. Before committing, verify that your chosen txtai LLM backend loads on your hardware and check the LLM, MAXLENGTH and n_ctx environment variables against the app folder source, since the README does not document rollback or failure handling.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. neuml/annotateai on GitHub
4. README
5. Releases