MONAI Label: AI-Assisted Annotation for Medical Images, Wired Into 3D Slicer and OHIF

The gap MONAI Label fills between model developers and annotators

The README states the project aims to fill the gap between developers creating new annotation applications and the end users who want to benefit from them. That is a fair description of the actual problem. A segmentation model exists, an annotator sits in front of a viewer, and nothing connects the two. MONAI Label is that connector: a server that exposes labeling apps as a service, and clients that plug into viewers clinicians already use.

The audience is split in two. On one side are researchers and developers who package a model as a MONAI Label app and want it reachable over HTTP rather than embedded in a desktop tool. On the other are clinicians, technologists and annotators who never touch the server code, only the viewer panel. The project supports radiology through 3DSlicer, MITK and OHIF, pathology through Digital Slide Archive, QuPath and CVAT, and endoscopy through CVAT. If your annotation workflow lives outside those viewers, the value proposition shrinks considerably.

How the server, the apps and the viewer clients fit together

MONAI Label is described in the README as a server-client system. The server is a FastAPI application, which is visible in requirements.txt: fastapi, uvicorn, pydantic, python-multipart and httpx are all pinned there. The labeling logic lives in apps under sample-apps/, and setup.py walks that directory with recursive_files to ship the app files as package data, alongside plugins/ and monailabel/logging.json.

Data flow is request-response. A viewer client asks the server for a model's output on a study, the app runs inference and returns a label, and the annotator edits it. The learning half is the part that distinguishes it from a plain inference endpoint: the README says the tool enables adaptation of AI to the task at hand by continuously learning from user interactions and data, and it lists an automated active learning workflow for endoscopy using CVAT.

Model coverage comes through the monaibundle app, which the README points to for supported models including whole body segmentation, whole brain segmentation, lung nodule detection and tumor segmentation. The supported matrix also names Segmentation, DeepGrow, DeepEdit and SAM2 (2D/3D) for radiology, and DeepEdit, NuClick, Segmentation, Classification and SAM2 (2D) for pathology. Note that SAM-2 is pulled from a git commit in requirements.txt and only for python_version >= '3.10', so the dependency graph is not uniform across Python versions.

Installing MONAI Label and running a first annotation session

The README lays out the getting-started path as five steps: installation, sample applications, supported viewers, data preparation, then starting the server and annotating. The package is on PyPI, so the install is a pip command.

pip install monailabel

After that, the README's step 2 is to pick a sample application. The repository ships them under sample-apps/, and setup.py installs them as package data, so they are available from the installed package rather than only from a git checkout. The monaibundle app is the one the README names for Model Zoo models.

Step 3 is the viewer. For radiology the README links a 3DSlicer plugin under plugins/slicer and an OHIF plugin under plugins/ohif. The Dockerfile shows how the OHIF viewer is built into the server image: a Node stage copies plugins/ohifv3, runs ./build.sh, and the release directory is copied to monailabel/endpoints/static/ohif. That means when you run the container, OHIF is served by the same process.

DOCKER_BUILDKIT=1 docker build -t projectmonai/monailabel:latest .

That command is the one given in the Dockerfile header comment, which also notes you can pass --build-arg FINAL_IMAGE=... to change the base. The default FINAL_IMAGE is ubuntu:22.04 and the default BUILD_IMAGE is python:3.10. The Dockerfile installs git, curl, openslide-tools, python3 and python3-pip in the final stage, so OpenSlide-based pathology readers work out of the box.

Step 4 is data preparation and step 5 is starting the server and annotating. The README does not print a full server invocation in the excerpt available, so the reliable next move is the tutorial series it links under Project-MONAI/tutorials/tree/main/monailabel, which the README describes as notebook-like tutorials for application and viewer workflows. Go there rather than guessing at flags.

Where MONAI Label is the wrong tool

The server-client split is the main constraint. If your annotators work in a browser on locked-down machines, you are running a FastAPI service with uvicorn and a GPU somewhere, plus a viewer plugin that has to be installed on each workstation. A team that only needs rectangles drawn on 2D photos will find the whole stack heavier than the task.

Version discipline is a second issue. requirements.txt pins exact versions for uvicorn (0.29.0), pydantic (2.7.0), fastapi (0.110.2), pydicom (3.0.1) and many others, and it carries a comment that scipy and scikit-learn latest packages are missing on python 3.8. That pinning is deliberate but it means MONAI Label does not float with your existing environment; installing it into a shared virtualenv with other FastAPI services is likely to fight over pydantic and httpx versions.

There is also a platform caveat in the dependency list. numpymaxflow is marked python_version < "3.13", so on Python 3.13 that package is skipped, and SAM-2 is only installed for python_version >= '3.10'. The intersection where every optional piece is present is narrower than the Python version range suggests. The README itself is honest about the supported matrix, saying those are only the fields, modalities and viewers the project has explicitly tested and that this does not mean another dataset or file type will not work.

How MONAI Label differs from plain nnU-Net or a hosted annotation service

The closest comparison is nnU-Net plus a viewer plugin. nnU-Net trains and infers; it does not serve results to a viewer over HTTP, and it has no concept of an annotator correcting a mask and that correction feeding back into the model. MONAI Label's active learning loop, which the README describes as continuously learning from user interactions and data, is the difference in approach. You are not choosing a better segmentation architecture, you are choosing a delivery and feedback mechanism around one.

Against a hosted annotation platform, the difference runs the other way. A hosted service gives you a browser, storage and a workforce, but your imaging data leaves your network. MONAI Label runs locally, which the README states explicitly: it is an ecosystem that can run locally on a machine with single or multiple GPUs, and server and client can sit on the same or different machines. For institutions where DICOM cannot leave the premises, that locality is the deciding factor, and the README notes PACS connectivity via DICOMWeb as the route in.

The cost of that choice is operational. You own the GPU, the server process and the viewer plugin rollout. MONAI Label gives you the pieces, not the managed service.

Maintenance, releases and what the Apache-2.0 licence does not cover

The last push to the main branch was on 2026-07-29, so the repository is being worked on. The release cadence is uneven, though. The recent releases listed are 0.8.5 (2023-10-18), 0.8.4 (2024-10-17) and 0.8.3 (2024-07-23), which means the version numbers and dates do not run in a clean order and the gap between 0.8.4 and the current main branch is not covered by a tagged release. If you need a stable artifact, pin an explicit version from PyPI rather than tracking main, because requirements.txt pins its own dependency set and an unpinned install will drift.

Upgrade cost concentrates in two places: the FastAPI and pydantic pins, and the model dependencies. The SAM-2 dependency is a git URL at a specific commit, so it is not resolved from PyPI and will need network access to GitHub during install.

The licence is Apache-2.0 for the repository, which permits commercial use and modification with the usual notice and patent terms. That licence covers the code, not the model weights. The README points at the MONAI Model Zoo through the monaibundle app, and the licence of any individual bundle is set by whoever published it, not by this repository. Check the terms of the specific weights you intend to serve before clinical evaluation. Nothing here is legal advice; the LICENSE file in the repository is the authoritative text.

Editorial conclusion

Adopt MONAI Label if your annotation work happens in 3D Slicer, OHIF, QuPath, DSA or CVAT and you want a model to pre-fill masks while you correct them, because the sample apps and the monaibundle app cover radiology, pathology and endoscopy without you writing a server. Do not adopt it if you need a pure web annotation tool with no local server, or if you cannot host GPUs for training. Before you commit, confirm which sample app matches your modality, check the licence and provenance of the specific Model Zoo weights you intend to serve, and pin the version in requirements.txt rather than tracking main.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project-MONAI/MONAILabel on GitHub
3. Project website
4. README
5. Releases