segment-anything's README now opens by sending you to SAM 2

The first section of the README is about a different model

Before describing SAM at all, the README announces a successor and tells you to go and look at it. The heading is Latest updates, the subject is Segment Anything Model 2, and three links follow: the code in a separate segment-anything-2 repository, a hosted demo, and the paper on arXiv at 2408.00714. What SAM 2 changes is stated plainly enough to matter. It extends SAM to video by treating an image as a video with a single frame, uses a transformer architecture with streaming memory for real-time video processing, and is trained through a model-in-the-loop data engine that improves both the model and the data from user interaction, which produced the SA-V dataset described as the largest video segmentation dataset to date. So the question this repository now answers is narrower than it did in 2023, and the answer is in the README's own opening lines.

Two entry points, and the difference is who supplies the prompt

The model produces high quality object masks from input prompts such as points or boxes, and the two ways in are distinguished by where those prompts come from. The prompted path builds a SamPredictor, calls set_image once to embed the image, then passes prompts to predict, which returns masks along with two other values the example discards. The automatic path builds a SamAutomaticMaskGenerator and calls generate on the image, which produces masks for everything in the frame without you naming anything. That second mode is the one that changes what the model is for, because it turns segmentation from a tool you aim into a description of a whole scene. The example code is deliberately short, four calls for the prompted version, which says something about how little glue the library asks for.

amg.py is the path that needs no Python at all

For a one-off over a directory of images, the command line script avoids writing any code:

python scripts/amg.py --checkpoint <path/to/checkpoint> --model-type <model_type> --input <image_or_folder> --output <path/to/output>

Four arguments, and the input takes either a single image or a folder, so the same command covers a file and a corpus. There is a matching script for the other half of the story:

python scripts/export_onnx_model.py --checkpoint <path/to/checkpoint> --model-type <model_type> --output <path/to/output>

Both take the same checkpoint and model type, which is the detail that makes the pair usable together. The notebooks directory is the documented route for the remaining cases, with one on prompted prediction and one on automatic generation, and it needs jupyter installed, which is one of the optional dependencies rather than a core one.

Three checkpoints sit behind one registry key

Model selection is a string lookup, and the three options differ by backbone size rather than by task. The key default and the key vit_h both resolve to the ViT-H model, so the short alias is not a fourth option but a synonym for the largest one. vit_l resolves to ViT-L and vit_b to ViT-B, each with its own download link, and all three are constructed the same way through sam_model_registry with a checkpoint path. For a reader the practical consequence is that the default is the heaviest model, and there is no guidance in the repository about the accuracy difference between the three or the memory each needs. If you are running SAM on a constrained GPU, choosing vit_b over the default is the single decision that changes your ability to run it at all, and the documentation does not tell you what that costs in mask quality.

setup.py declares no dependencies, so pip brings you nothing useful

The installation section tells you the code needs python>=3.8 with pytorch>=1.7 and torchvision>=0.8, and points you at PyTorch's own instructions for both, with CUDA support strongly recommended. Then the install command is this:

pip install git+https://github.com/facebookresearch/segment-anything.git

The reason for the separate PyTorch step is visible in setup.py, where install_requires is an empty list. Extras exist, with all covering matplotlib, pycocotools, opencv-python, onnx and onnxruntime, and dev covering flake8, isort, black and mypy, but neither pulls the deep learning stack. So a clean environment that runs the one-line install will then fail at import, and the versions in the README are floors from 2020 rather than tested pins. The two-year gap between those floors and the current ecosystem is the practical risk here, and nothing in the repository has been updated to narrow it.

ONNX export ships the decoder and leaves the backbone behind

The export path is narrower than exporting the model, and understanding why explains the browser demo. It is specifically the lightweight mask decoder that is exported to ONNX, so the result runs anywhere that supports an ONNX runtime, which is what the hosted demo does. The backbone does not travel with it. The example notebook exists to combine image preprocessing through SAM's backbone with mask prediction from the ONNX model, which means the browser path still needs the PyTorch side for the embedding step, and the note recommends the latest stable PyTorch for the export itself. The demo folder is a one page React application that runs the exported model in a browser with multithreading, with its own README and a TypeScript and Tailwind build, so it is a worked reference rather than a drop-in component.

The code is Apache-2.0 and the dataset is not

Two licences apply and they do not cover the same thing. The repository itself is Apache-2.0, with a LICENSE file at the root and a Meta Platforms copyright header in setup.py, so the code and the published checkpoints are straightforward to use. The training data is separate. The SA-1B dataset is linked to a separate overview page and a separate downloads page, and downloading it means agreeing that you have read and accepted the terms of the SA-1B Dataset Research License. The scale of what sits behind that research licence is the 11 million images and 1.1 billion masks the model was trained on. If you are only running inference with the published weights, the dataset terms do not touch you; if you are retraining or fine-tuning on it, they are the thing to read, and they are not the Apache-2.0 grant.

No releases, and the last push was 2024-09-18

The repository has no GitHub releases, so there is no version history to consult and nothing to pin beyond the commit, even though setup.py carries a version of 1.0. The last push to the repository was on 2024-09-18, which is about two years old and far too long ago to describe the project as one still receiving work. That is not a criticism so much as a fact about how to use this code: it is a finished reference implementation rather than a library expecting patches, and the pointer at the top of the README is the signal that development moved elsewhere. The project structure supports that reading, with a segment_anything package, scripts, notebooks, a demo app, a linter.sh and a setup.cfg, which is a small and tidy tree rather than a large one. If you need a maintained image segmenter, the successor repository is where to look first.

Editorial conclusion

Adopt this repository if you need image segmentation with point or box prompts today and want the reference implementation with its published checkpoints. Do not start a new project on it without reading the first section, because the project itself directs you to SAM 2 for anything involving video, and the code here has not moved since 2024-09-18. Verify first what your installer actually pulls, since setup.py declares no dependencies and will not bring PyTorch with it, and settle the data licence separately from the code licence if you plan to train rather than use the published masks.

Frequently asked questions

Official sources

1. Official README
2. Project repository