Count Anything: text-guided object counting across six visual domains

Counting by pointing, not by density maps

Object counting has long been split into two camps: specialized counters per category, and density-map regressors that produce a blurry heatmap whose integral is the count. Count Anything, the model behind a paper released on arXiv in May 2026, takes a third road. Given an image and a natural-language query, it returns a set of points, one per counted instance, so the count is the cardinality of visible, checkable marks rather than the sum of a heatmap.

That formulation does two jobs at once, which the README states cleanly: it unifies category-conditioned counting with spatial localization, and it gives every prediction an interpretable grounding. When the model claims forty-two cells, you can see which forty-two. The audience is researchers and practitioners who need counts across changing categories, crowds, cells, crops, cars, without retraining a specialist model for each.

CLOC: the benchmark behind the model

Training and evaluation run on CLOC, the Cross-domain Large-scale Object Counting dataset the paper introduces, reorganizing diverse public sources into one benchmark. The scale is real: about 220 thousand images, 619 categories and 15 million object instances, spread across six domains that few benchmarks attempt together: general scenes, remote sensing, histopathology, cellular microscopy, agriculture and microbiology.

The data engineering is a quiet contribution. Heterogeneous annotations, boxes, points, polygons, masks, rotated boxes, label maps, are all converted into counting points with optional boxes, and point-centric supervision ensures every valid instance is supervised by a point while boxes are used only where reliable. The team also maintains the benchmark itself: CLOC-v1.1, released 2026-07-02, came from a manual re-review that removed or re-annotated noisy samples, which is the kind of unglamorous work counting benchmarks rarely receive.

Two counters and a fusion rule

The architecture pairs two specialists with complementary failure modes. The Region-level Sparse Counter anchors large, sparse objects, the ones a region proposal can latch onto. The Pixel-level Dense Counter predicts dense points for small, crowded, weakly bounded targets, exactly where sparse approaches collapse.

The combination rule is named Complementary Count Fusion, and its defining property is that it is parameter-free: RSC and PDC outputs merge in a way that suppresses duplicate counts while preserving each counter's strengths, with no learned fusion layer to overfit. The design reads as an engineering answer to a real tension in counting, precision on big things versus recall on swarms, resolved by running both and trusting neither alone. All of this initializes from SAM3, Meta's segmentation model, whose pretrained weights serve as the training starting point.

Running it: conda, one checkpoint, one caveat

Environment setup follows the README verbatim:

conda create -n countanything python=3.12 -y
conda activate countanything
pip install -r requirements.txt

The dependency list is deliberately minimal, with a note that if pip's resolver picks the wrong CUDA build for PyTorch, you should install the matching PyTorch and torchvision for your local CUDA yourself. For inference, validation or test-only reproduction, one file suffices: download count_anything.pt from the project's Hugging Face page and place it at checkpoints/count_anything.pt, which the validation and test configurations load directly.

Training or fine-tuning adds the caveat. The model initializes from official SAM3 weights, and licence and redistribution restrictions prevent this repository from shipping them: you download sam3.pt from the official SAM3 Hugging Face page yourself, place it at pretrained/sam3.pt, and mind the README's warning that it must be SAM3, not SAM3.1.

What the results claim

The evaluation story centers on CLOC, and the recent news entries frame the competitive position. The 2026-07-20 update reports evaluation of LocateAnything-3B, a recent vision-language grounding model from NVIDIA, on CLOC-v1.1, with Count Anything holding a substantial lead in counting performance on the full test set and across all six domains. Earlier results on the original CLOC are shown against the paper's comparison table.

One presentation detail matters for anyone verifying claims: the main results tables ship as images in the README rather than text or machine-readable numbers, so exact figures live in the arXiv paper, a PDF of which is bundled under assets/paper. The qualitative visualizations page shows the point predictions across domains. The honest workflow is to read the paper for numbers, then reproduce on CLOC-v1.1 using the repository's scripts, which is evidently how the authors expect the claims to be checked.

Research code, in the good and bad senses

The repository is a research artifact, and it behaves like one on both axes. The good: shell scripts named train.sh, val.sh and test.sh wrap the entry points, configuration lives in config, the CLOC preparation has its own guide including a distributable augmented-image archive, and an Apache-2.0 licence covers the code. The no-code path exists too, a Hugging Face Spaces demo runs the model in a browser, which is the fastest way to test whether text-guided counting fits your images at all.

The research-code caveats: no releases or tags to pin, a checkpoint plus weight downloads as the setup contract, and no test suite visible in the tree. The last push was on 2026-08-12. None of this obstructs the intended workflow, reproduce the paper, fine-tune on your domain, but teams hoping for a pip-installable production counter will need to build that packaging themselves or wait for it.

Against specialist counters and plain detection

The alternatives frame the contribution precisely. Specialist counting models, the density-map tradition built per dataset and per category, remain strong inside their domain and are the wrong tool the moment your target category changes, which is the gap text-guided counting addresses. Open-world counting methods, the baselines the paper compares against, share the flexibility goal and lose to Count Anything on CLOC per the README's own account.

Plain object detection deserves its due as well: when your targets are large, distinct and covered by existing detectors, running a detector and counting boxes is simpler and gives you boxes rather than points. Count Anything earns its complexity in the regimes detection handles poorly, dense microscopy fields, aerial crowds, agricultural clusters, where individual instances are small, numerous, and better represented by points than boxes. Choose by object size and category volatility, not by which approach is more fashionable.

Editorial conclusion

Count Anything fits researchers and engineers who need counting across shifting categories and domains, want instance-grounded points they can audit, and are comfortable with a research codebase, conda environments and self-downloaded SAM3 weights for training. Skip it if your objects are large and well-detected already, if you need a supported production package, or if your verification standard requires machine-readable benchmark tables, these ship as images and a paper. Start with the Hugging Face demo on ten of your own images; if the points land where your eyes do, take the conda setup seriously next.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Mengqi-Lei/count-anything on GitHub
4. README