ISAT_with_segment_anything: a PyQt5 labeling tool that puts SAM, SAM2 and SAM3 behind one canvas

The gap ISAT-SAM fills: masks are slow to draw by hand and slow to review

Segmentation datasets are the expensive kind. A bounding box takes a second to draw; a pixel-accurate mask around a leaf, a lesion or a car panel can take a minute or more, and the reviewer has to check every vertex. Promptable segmentation models changed the economics of that task, but they arrive as libraries and notebooks, not as something an annotator can sit in front of for eight hours.

ISAT-SAM is the layer between the two. The README describes it as "An Interactive Semi-Automatic Annotation Tool Based on Segment Anything", and the repository topics list annotation-tool, labeling, sam, sam2 and sam3. The intended user is a computer vision engineer or a small annotation team that already knows which checkpoint it wants and needs a canvas, a class list, and a file format at the end. The example directory ships convert, images, instance, semantic and videos folders, which tells you the project expects work in both the instance and semantic directions, and in video as well as stills.

How the SAM prompt loop works inside the application

The mechanism is prompt-and-refine. A model produces a candidate mask from a click, a box or, in the newer versions, a text or visual prompt; the annotator accepts it, or adds another point to push the mask toward the correct boundary. The interactive part is the correction loop, not the model. This is why the tool can support several backbones without changing the interface: SAM, SAM2, SAM3, sam-hq, MobileSAM and EdgeSAM all answer the same kind of prompt.

The release notes mark the version history clearly. Version 1.5.0 added SAM3 support together with text prompts, and the README shows separate demonstrations for a single category and for multiple categories. Version 1.5.2 added visual prompts based on SAM3. Version 1.4.0 added the plugin system, and the two official examples are small by design: the README states that the YOLO-based auto-annotation plugin is implemented in about 240 lines of code and the mask export plugin in about 160. That line count is the real architectural claim. If a plugin that changes how masks leave the tool fits in a couple of hundred lines, the extension surface is narrow and legible rather than a framework you have to learn.

The desktop shell is PyQt5, and setup.py lists pyqt5, opencv_python_headless, torch>=2.3.0, torchvision, pycocotools, shapely, hydra-core>=1.3.2 and timm among the required packages. The presence of pycocotools and shapely points at polygon and mask handling as first-class concerns rather than an export afterthought.

Installing ISAT-SAM and running a first annotation

The README gives a conda environment as recommended but optional, then a single pip install and a single launch command. The conda step pins Python 3.8, which is the version the README uses in its example; setup.py states python_requires=">=3.8".

Create and activate the environment first:

conda create -n isat_env python=3.8
conda activate isat_env

Then install the package from PyPI. The distribution name is isat-sam, which is not the same as the repository name, so this is the string to search for:

pip install isat-sam

Launch the application with the console entry point of the same name:

isat-sam

A window should open. From there the workflow the README demonstrates is: load images, define your categories, then prompt the model on the canvas and correct the mask. What you should see is a mask that appears after the first prompt and changes as you add points. The README does not document a headless mode, a CLI annotation path, or a server mode, so treat this as a desktop application rather than something you script from a pipeline.

Where ISAT-SAM is the wrong tool

The dependency list is the first constraint. torch>=2.3.0 plus torchvision plus the SAM-family checkpoints means the install is heavy and the runtime wants a GPU. On a CPU-only laptop the interactive loop that makes the tool worth using degrades into waiting, and at that point a manual polygon tool is faster. The README does not publish hardware requirements or a minimum VRAM figure, so you are testing that yourself.

The second constraint is scope. This is a segmentation annotator. If your dataset is bounding boxes, keypoints or classification labels, the promptable-mask machinery is dead weight and a general labeling tool will be lighter. The third is collaboration. Nothing in the README describes user accounts, task assignment, review states or a hosted backend; the application is a local PyQt5 program, and the plugin examples are about annotation and export, not about multi-user coordination.

There is also a licensing detail worth reading carefully. setup.py declares license="Apache2.0", while the repository metadata reports the licence as NOASSERTION, meaning GitHub could not map the LICENSE file to a known identifier. Those two statements do not agree, and the LICENSE file is the one that governs. Read it before you ship anything built on this. The same caution applies to the SAM checkpoints themselves, which carry their own terms separate from this tool.

How it compares with X-AnyLabeling and the other SAM front ends

The obvious alternative in the same space is X-AnyLabeling, and the difference is focus rather than quality. X-AnyLabeling is a general labeling application that also carries SAM-family models; ISAT-SAM is a segmentation tool that carries a wider set of segmentation backbones. The README names SAM, SAM2, SAM3, sam-hq, MobileSAM and EdgeSAM, and the topics include text-prompt and video-segmentation. If your work is masks, video and prompt types, the narrower tool has more of the thing you need. If your work mixes boxes, polygons and classification in one project, the general tool saves you from running two applications.

A second comparison point is the plugin model. ISAT-SAM's extension story is deliberately small: the README's two official plugins are roughly 240 and 160 lines. That is attractive if you want to add a YOLO pre-annotation pass or a custom mask export without reading a large codebase. It is less attractive if you need a broad ecosystem of ready-made integrations, because the plugin system only arrived in version 1.4.0 and the catalogue is two packages.

Maintenance, upgrade cost and what the version numbers imply

The last push to the default branch was on 2026-09-02, and the most recent release in the repository metadata is v1.5.2 from 2025-12-31, preceded by v1.5.1 on 2025-12-24 and v1.5.0 on 2025-12-17. Three releases inside a fortnight followed by a quieter stretch in the release feed is a pattern worth noting when you plan an upgrade: feature work lands in bursts rather than continuously.

The upgrade cost is dominated by two moving parts. The first is torch. Because the package requires torch>=2.3.0 and the SAM-family backbones are version-sensitive, a torch bump can break checkpoint loading in a way that has nothing to do with ISAT-SAM's own code. Pin torch in your environment rather than letting pip resolve it. The second is the plugin API. The plugin system is recent, and the README does not state a stability guarantee for it, so a plugin written against 1.4.0 may need attention after a minor release. Pin the isat-sam version in the environment you hand to annotators, and test the plugin against the new version before you upgrade the team.

On licensing, setup.py says Apache2.0 and the repository metadata says NOASSERTION. That discrepancy is a fact you should resolve by reading the LICENSE file, not by assuming either label. This article is not legal advice.

Editorial conclusion

Adopt ISAT-SAM if your labeling work is segmentation-heavy, your annotators work on machines with a GPU or are willing to accept CPU-speed inference, and you want SAM-family models behind a desktop canvas rather than a web service. Do not adopt it if you need a browser-based multi-annotator platform with review queues, or if your labels are boxes and keypoints rather than masks. Before rolling it out, verify three things on your own hardware: that the SAM3 and SAM2 checkpoints you intend to use load inside the installed torch version, that the plugin API in the version you pin still matches the plugin you plan to write, and that the mask export format your training pipeline consumes is the one the plugin produces.

Frequently asked questions

Official sources

1. Issues
2. README
3. Releases
4. yatengLG/ISAT_with_segment_anything on GitHub