mLLMCelltype: Multi-LLM Consensus Annotation for scRNA-seq
Cell type annotation for single-cell RNA-seq using multi-LLM consensus
At a glance
- What is it?
- mLLMCelltype asks several large language models to label the same cluster markers and combines their answers into one call with an uncertainty score. The published benchmark in Communications Biology reports 77.2% mean accuracy across 49 datasets, but the annotation is still a text judgement, not a statistical one.
- Who is it for?
- Adopt mLLMCelltype if you already have ranked marker genes per cluster and want a second opinion on labels you would otherwise assign by hand, especially for tissue with no clean reference atlas. Do not adopt it if your analysis must be reproducible without network calls, if you cannot send marker gene lists to a third-party API, or if you need a per-cell probability rather than a per-cluster label.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 9 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The annotation bottleneck mLLMCelltype targets
Cluster labelling is the step in a single-cell workflow where computation hands off to judgement. Scanpy or Seurat will happily produce twenty clusters and a ranked list of marker genes per cluster, and then someone has to decide that cluster 7 is a resident macrophage and not a monocyte. Reference-based tools such as SingleR or CellTypist answer that question by similarity to a labelled atlas, which means the answer is only as good as the atlas coverage for the tissue in front of you. mLLMCelltype takes the opposite position: it sends the marker gene list to several large language models, collects their label suggestions, and reports a consensus. The README states the method requires no reference dataset and no pre-training, so the intended user is someone working on a tissue or condition where no suitable reference exists, or someone who wants a sanity check against the atlas-based label they already have. The documentation also notes tolerance to imperfect marker gene lists, which matters because marker lists derived from a Wilcoxon test on a small cluster are frequently noisy.
How the consensus is actually assembled
The pipeline is text-in, text-out, wrapped in a voting layer. Marker genes per cluster go out to each configured provider (the README lists OpenAI GPT-5.5, Anthropic Claude Opus 4.7 and Sonnet 4.6, Google Gemini 3, X.AI Grok 4.3, DeepSeek V4, Alibaba Qwen 3.6, Z.AI GLM 5.1, MiniMax M2.7, Stepfun 3.5, and OpenRouter as a routing layer). Each model returns a proposed cell type plus reasoning. The framework then runs what the README calls iterative discussion, where models evaluate evidence across multiple rounds before the final labels are combined. Two metrics come out of that combination: Consensus Proportion, which is the share of models agreeing on a label, and Shannon Entropy, which measures how spread out the proposals are. Those two numbers are the actual value proposition. A cluster where every model says hepatocyte and a cluster where four models give four different answers can both be labelled, but only the second one should send you back to the marker list. The README also describes hierarchical annotation with consistency checks, so a coarse label set and a finer label set can be reconciled rather than treated as independent runs.
Installation and the optional dependency trap
The project ships two maintained packages from one repository: R under R/ and Python under python/, with web/ for the hosted application and paper/ for the published analysis code. The README notes that large paper datasets and generated results stay local under paper/data/ and paper/results/ and are excluded from Git, so cloning the repository does not give you the benchmark data. For R, the documented path is install.packages("mLLMCelltype") from CRAN, or devtools::install_github("cafferychen777/mLLMCelltype", subdir = "R") for the development version. For Python, pip install mllmcelltype, or pip install git+https://github.com/cafferychen777/mLLMCelltype.git#subdirectory=python. The part that catches people is the modular provider design. Provider SDKs are optional extras: pip install "mllmcelltype[openai]" for OpenAI models, "mllmcelltype[anthropic]" for Claude, "mllmcelltype[gemini]" for Google, or "mllmcelltype[all]" for everything. The README calls out the specific failure this causes, an ImportError such as cannot import name 'genai' from 'google', which means the provider package was never installed. If you install the base package and then configure a Gemini model, the error surfaces at call time rather than install time. A Colab notebook is linked for a no-install trial, and a browser interface runs at mllmcelltype.com.
What the 77.2% figure does and does not cover
The README reports that across 49 benchmark datasets, mLLMCelltype reached 77.2% mean accuracy, against 61.5% for the best single-LLM baseline, a 15.7 percentage point gap. That comparison is the honest one to quote, because it isolates the consensus layer: same task, same class of model, multiple models versus one. It is not a comparison against reference-based annotation, and the README does not present one. So the number tells you that averaging several LLMs beats picking one LLM. It does not tell you that mLLMCelltype beats a well-matched atlas, and for tissues with good reference coverage that is the comparison that decides your tooling. The other thing to read carefully is that 77.2% is a mean over 49 datasets. A mean hides the distribution. Fine-grained immune subsets and rare populations are where marker-based labelling is hardest for any method, and the README does not break the accuracy down by tissue or by label granularity. Treat the headline as evidence that the consensus mechanism works, not as a per-dataset expectation for your experiment.
The failure modes that matter in practice
Three constraints are visible in the material. First, every annotation is a network call to a third-party API. Marker gene lists for your clusters leave your machine and reach OpenAI, Anthropic, Google, or whichever providers you configure. For human subject data under a data use agreement, that is a policy question before it is a technical one, and the README does not describe a local-model option. Second, the output is a label per cluster with an uncertainty score, not a per-cell posterior. A cluster that is 85% one cell type and 15% another gets a single name, and the Consensus Proportion will not flag that, because it measures model agreement, not biological purity. You still need to subcluster. Third, results are not deterministic in the way a deterministic algorithm is. The same markers sent to the same model set on two days can produce different labels if a provider updates a model behind an unchanged API name. The README documents the reasoning trail for transparency, which helps you audit a single run, but it does not claim run-to-run reproducibility. If your analysis needs to be byte-identical when re-executed a year later, this is the wrong layer to depend on.
Where it sits against CellTypist and SingleR
The nearest alternatives are reference-based classifiers. CellTypist and SingleR both work by projecting your query cells onto a labelled reference and assigning the closest match, which makes them fast, free of API costs, and fully reproducible, since the reference is a fixed file. Their weakness is the mirror image of mLLMCelltype's strength: if your tissue is not represented in the reference, the classifier will still return a label, and it will be wrong in a way that looks confident. mLLMCelltype has no reference to be missing and can reason about a marker combination it has never seen in a training set, at the cost of a network round trip, a per-call fee, and a label you cannot reproduce from a stored model file. In practice these are complementary rather than competing. Running CellTypist first and then checking the clusters where its confidence is low against mLLMCelltype's Consensus Proportion uses each tool where it is strongest. The README positions the package as reference-free, and that framing is accurate, but reference-free does not mean assumption-free: it means the prior knowledge lives in the language model's training data rather than in an atlas file you can inspect.
Maintenance load, licence and what to verify first
The repository is active, with v2.0.8 released in August 2026 and v2.0.7 the month before, and the project is MIT licensed, which permits commercial and academic use with the usual requirement to retain the copyright notice. The maintenance cost is not in the package, it is in the provider surface. Ten-plus LLM providers means ten-plus SDKs that can break, model names that get deprecated, and API keys to rotate. The README's own changelog pointer is R/NEWS.md, and the release cadence suggests provider support is the main thing being maintained. Budget for pinning your provider SDK versions and for the possibility that a model you configured is retired between runs. On licence, MIT covers the code; it does not cover what the LLM providers do with the marker gene text you submit, and their terms are separate agreements you accept by configuring an API key. That is a question for your institution, not for the repository. The concrete first check on your own data: annotate one dataset, then look at the clusters with the lowest Consensus Proportion and highest Shannon Entropy, and confirm that those are the clusters you already found ambiguous by eye. If the uncertainty metrics do not line up with your own doubts, the consensus layer is not adding information for your tissue.
Editorial conclusion
Adopt mLLMCelltype if you already have ranked marker genes per cluster and want a second opinion on labels you would otherwise assign by hand, especially for tissue with no clean reference atlas. Do not adopt it if your analysis must be reproducible without network calls, if you cannot send marker gene lists to a third-party API, or if you need a per-cell probability rather than a per-cluster label. Before trusting it on your data, run one dataset twice with the same model set and check whether the Consensus Proportion and Shannon Entropy values reported by the package are stable across runs, and confirm which provider SDKs your install actually pulled in with pip show mllmcelltype or packageVersion("mLLMCelltype") in R.
Community notes