AnyLabeling: AI-assisted image labeling with YOLO and Segment Anything

What AnyLabeling solves, and who ends up using it

Hand-drawing polygons for a segmentation dataset is slow, and the first pass is usually the one that hurts most. AnyLabeling targets that first pass. The README describes it as "AnyLabeling = LabelImg + Labelme + Improved UI + Auto-labeling", and the feature list backs that up: polygon, rectangle, circle, line and point annotation, plus auto-labeling driven by YOLOv8 for object detection and by the Segment Anything family for segmentation.

The intended user is an individual annotator or a small team working on a desktop machine. The tool is a PyQt6 application, so it runs where you can open a window: Linux, Windows, or Apple Silicon macOS. The feature list includes text detection, recognition and KIE labeling, which points at document and receipt-style datasets as well as natural images. The interface ships in English, Vietnamese and Chinese.

It is not a labeling service and it is not a web platform. There is no server component in the README, no account system, and no reviewer queue. If your workflow assumes several people labeling the same dataset from browsers, this is the wrong shape of tool, and the section on limitations below explains why.

How auto-labeling actually runs: ONNX Runtime, model weights, and prompts

The pipeline is local inference. AnyLabeling depends on onnxruntime and onnx, and the supported model table lists SAM ViT-B/ViT-L/ViT-H, MobileSAM, SAM 2 Hiera-Tiny through Large, SAM 2.1 in the same sizes, SAM 3 ViT-H, and YOLOv8n through YOLOv8x. Each model entry declares its prompt types. The original SAM, MobileSAM, SAM 2 and SAM 2.1 accept point and rectangle prompts. SAM 3 ViT-H accepts text, point and rectangle, and the table describes it as open-vocabulary, where "text drives detection".

That distinction matters when you plan a project. With point or rectangle prompts, a human still clicks on the object, and the model turns that click into a mask. With SAM 3 text prompts, the class name itself is the input, which changes the labeling loop from click-per-object to type-a-phrase-per-class.

The README states that required model weights are downloaded automatically on first use, and the dependency list includes huggingface_hub, which is the likely download path. That means the first run of a given model is a network operation, not an offline one. The v0.4.43 release notes mention fixes for recovery after model download or load failures and for skipping invalid images during SAM preload, which tells you both failure modes are real enough to have been patched.

Execution provider selection is handled by the ANYLABELING_DEVICE environment variable. The README says AnyLabeling automatically selects CUDA for Linux and Windows GPU builds and CoreML for the macOS GPU build, with CPU fallback for unsupported model operations. On Windows PowerShell the override is written as $env:ANYLABELING_DEVICE = "DIRECTML".

Installing AnyLabeling from PyPI and labeling your first image

There are two install paths. The Download page and GitHub Releases carry prebuilt artifacts per platform: AnyLabeling-Linux-CPU-x64 and AnyLabeling-Linux-GPU-x64, AnyLabeling-Windows-CPU-x64.exe and AnyLabeling-Windows-GPU-x64.exe, and AnyLabeling-macOS-CPU.zip and AnyLabeling-macOS-GPU.zip. The README warns that on macOS you must preserve the archive's symlinks while extracting, and points at docs/macos_folder_mode.md.

The source path requires Python 3.11 or newer, with 3.12 recommended, and the README suggests Miniconda or Anaconda. Create the environment first:

conda create -n anylabeling python=3.12
conda activate anylabeling

On macOS only, the README says to install PyQt6 through Conda rather than pip, because the PyQt6 dependency in pyproject.toml is marked platform_system != 'Darwin':

conda install -c conda-forge pyqt=6

Then install the package. The plain distribution is the CPU build:

pip install anylabeling

For NVIDIA CUDA inference on Linux or Windows, the README says to use a fresh environment and the GPU distribution instead:

pip install anylabeling-gpu

Apple Silicon users get ONNX Runtime CoreML plus native CoreML SAM2 models through the macos extra, with the device override set explicitly:

pip install "anylabeling[macos]"
export ANYLABELING_DEVICE=COREML

Once installed, the README's final step is a single command:

anylab

That launches the GUI. What you should see is the annotation window described in the README, where you load images or a folder, draw shapes, and pick a model for auto-labeling. The first time you select a model, expect the weights download described above before anything is labeled.

GPU, DirectML and NPU environments are where installs go wrong

The README is explicit that the GPU distribution bundles pip-managed CUDA 12 and cuDNN runtime libraries, so a compatible NVIDIA driver is enough and a system CUDA toolkit is not required. The pyproject.toml pins onnxruntime-gpu to >=1.20.0,<1.27 with a comment explaining the upper bound: ORT 1.27 and later require CUDA 13, and the pin keeps broad driver compatibility on CUDA 12. If you already have a CUDA 13 stack and want a newer ONNX Runtime, that pin is the constraint you will hit.

NPU setups are stricter. The README states that NPU execution requires the matching vendor ONNX Runtime package in a fresh, dedicated environment, and that you must replace the default onnxruntime package because ONNX Runtime allows only one variant per environment. Intel Core Ultra systems use onnxruntime-openvino with ANYLABELING_DEVICE=INTEL_NPU; Qualcomm Snapdragon Windows ARM64 systems use onnxruntime-qnn with ANYLABELING_DEVICE=QUALCOMM_NPU. The README also notes that Qualcomm HTP models generally need QDQ quantization and that support still depends on the operator coverage of the chosen model.

Supported ANYLABELING_DEVICE values listed in the README are CUDA, COREML, DIRECTML, ROCM, MIGRAPHX, OPENVINO, TENSORRT, CANN, QNN, VITISAI and WEBGPU, with NPU aliases NPU, INTEL_NPU, QUALCOMM_NPU, AMD_NPU and ASCEND_NPU. The breadth of that list is a good sign for portability, but each provider is a separate dependency resolution problem. The README's advice to use a fresh environment per provider is not boilerplate; it is the only way the single-variant rule holds.

Where AnyLabeling is the wrong tool

The licence is the first hard boundary. pyproject.toml declares GPLv3, and the repository carries a GPL-3.0 LICENSE. For an internal labeling tool that is usually fine. If you intend to link AnyLabeling into a proprietary product or ship a modified binary under different terms, GPL-3.0 is a constraint you need to evaluate with your own counsel. Nothing in the README offers a commercial exception.

The second boundary is collaboration. Nothing in the README describes a server, a shared project database, or multi-user review. Every annotation session is a local desktop session writing label files. The v0.4.43 release notes mention that label files are closed reliably after saving and loading, and that grouped shapes are persisted with undo support, which describes single-user file handling, not concurrent editing. A team that needs browser access, role-based review, and a central store should look at a web-based platform instead.

The third boundary is offline and air-gapped work. Because model weights download on first use, a machine with no network cannot run a model it has not already fetched. The README does not document a supported procedure for pre-seeding the weight cache, so if that is your environment, verify it before you plan around it.

Finally, model coverage is not unlimited. The supported table lists YOLOv8 and the SAM family. A custom architecture you trained yourself is not covered by that table, and the README does not document a custom-model registration workflow.

AnyLabeling compared with CVAT

The comparison people search for is AnyLabeling against CVAT, and the difference is architectural rather than a feature checklist. CVAT is a web application: annotators work in a browser against a server, which is what makes multi-person projects, task assignment and review practical. AnyLabeling is a PyQt6 desktop program launched with anylab, with no server in the picture.

That split produces the trade-offs you would expect. AnyLabeling wins on setup cost for one person: install a package or run a prebuilt binary, and inference happens on your own GPU through ONNX Runtime with CUDA, CoreML, DirectML, OpenVINO or a vendor NPU provider. There is no infrastructure to run. CVAT wins when the bottleneck is coordination rather than drawing speed, because a desktop file-per-session model does not give you a review queue.

On the AI side, AnyLabeling's auto-labeling is the point of the tool: YOLOv8 for detection, SAM, MobileSAM, SAM 2, SAM 2.1 and SAM 3 ViT-H for segmentation, with SAM 3 adding text-prompted open-vocabulary segmentation. If your interest is model-assisted pre-labeling on a workstation, that combination is the reason to pick it. If your interest is a shared annotation operation with audit trails, the desktop model is the reason to look elsewhere.

Maintenance, releases and upgrade cost

The repository is not archived, and the last push was on 2026-08-30. Release cadence is fast: v0.4.40, v0.4.42 and v0.4.43 all landed on 2026-08-29 and 2026-08-30, with v0.4.40 marked superseded. The v0.4.43 notes list recovery after model download or load failures, skipping invalid images during SAM preload, persistence of grouped shapes with undo support, and reliable closing of label files after saving and loading.

The release workflow is described as testing Python 3.11 through 3.13 on Linux, Windows and macOS, then building and launch-smoke-testing all six CPU and accelerated artifacts on their native runners. That is meaningful coverage for a project of this type, and it explains why the supported Python versions in pyproject.toml and the CI matrix line up.

Upgrade cost is mostly dependency resolution, not data migration. Label files are the durable artifact, and the release notes describe fixes to how they are written and closed rather than format changes. The real friction is environment churn: the onnxruntime-gpu pin below 1.27, the one-ONNX-Runtime-variant-per-environment rule, and the separate extras for gpu and macos mean an upgrade can force you to rebuild the environment rather than run pip install -U in place. Budget for that if you are pinned to a specific CUDA or NPU stack.

On licensing, GPL-3.0 applies to the project as distributed. The README says nothing about model weight licences, which are separate artifacts downloaded at runtime, so treat that as something to check yourself rather than something the README settles.

Editorial conclusion

AnyLabeling fits teams that want a local desktop annotator with YOLOv8 and SAM auto-labeling and are comfortable with GPL-3.0 and pip-managed ONNX Runtime environments. It is not the right choice if you need browser-based multi-annotator collaboration, or if you cannot ship GPL-3.0 code with your product. Before committing, install the exact distribution you plan to use (anylabeling, anylabeling-gpu, or anylabeling[macos]), confirm the model weights download and load on your hardware, and check that your target export format is one the tool actually writes.

Frequently asked questions

Official sources

1. License: GPL-3.0
2. Project website
3. README
4. Releases
5. vietanhdev/anylabeling on GitHub