# label-studio-ml-backend: turning your model into a Label Studio service

> The Label Studio ML backend is an SDK for wrapping machine learning code into a web server that a running Label Studio instance can call to automate labeling. The repository ships a directory of worked examples, from a BERT classifier to a RAG agent, each with a flag saying whether it supports pre-annotation, interactive mode and training.

**HumanSignal/label-studio-ml-backend** — Configs and boilerplates for Label Studio's Machine Learning backend

- Repository: https://github.com/HumanSignal/label-studio-ml-backend
- Stars: 1,107 · Forks: 491
- Language: Python
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/humansignal-label-studio-ml-backend

## An SDK whose output is a web server

The Label Studio ML backend is an SDK that lets you wrap your machine learning code and turn it into a web server. That server can then be connected to a running Label Studio instance to automate labeling tasks.

The design inverts the usual shape of a machine learning library. Instead of importing a model and calling it from your own code, you implement the model inside this SDK's conventions and the SDK runs it as a service, and Label Studio calls it over HTTP. Label Studio owns the task queue, the annotator interface and the project configuration; your server owns the model.

The documentation is direct about when you do not need this. If you just need to load static pre-annotated data into Label Studio, running an ML backend might be overkill, and the alternative is to import pre-annotated data through the predictions import. That is a useful piece of scoping, because the ML backend brings a service boundary, a container and a network hop with it, and if your task is one-shot annotation of an existing dataset, none of that is needed.

The package is named label_studio_ml and is distributed on PyPI, with a console script entry point label-studio-ml that points at label_studio_ml.server:main. The repository's licence field says Apache-2.0, and the setup.py classifiers list the MIT licence, which is a small inconsistency worth noting if you are doing a compliance review. The primary language is Python, and the last push was on 2026-09-15.

## The quickstart is a clone, a cd, and docker-compose up

Getting a model running takes three commands. The documented flow is to use docker-compose to run the ML backend server:

```bash
git clone https://github.com/HumanSignal/label-studio-ml-backend.git
cd label-studio-ml-backend/label_studio_ml/examples/{MODEL_NAME}
docker-compose up
```

The third line is the important one, because you substitute the model name. The examples live under label_studio_ml/examples/ inside the package, and each one is a self-contained directory with its own compose file and its own dependencies. So the repository is really a gallery of ready-to-run services rather than a single library you configure.

The service comes up at http://localhost:9090, and that is the address Label Studio then points at.

There is a second configuration step that the documentation calls out separately, and it is the one most people miss. In most cases you need to set the LABEL_STUDIO_URL and LABEL_STUDIO_API_KEY environment variables to allow the ML backend access to the media data in Label Studio. Without them the backend cannot fetch the images or text it is supposed to annotate, and the failure looks like an empty prediction rather than an authentication problem. The documentation points at the section on allowing the ML backend to access Label Studio data for the details.

For local development without containers, the Makefile is short. The install target is `pip install -r requirements.txt` and the test target is `pytest tests`.

## The examples table is a capability matrix

The model table in the README is the most useful part of the documentation, because each row carries three capability flags rather than a vague description.

The Pre-annotation column tells you whether the model can be used for pre-annotation, which is where pre-annotated data appears when you open the labeling page or after running predictions for a batch of data. The Interactive mode column tells you whether the model can be used for interactive labeling, which means you see interactive predictions as you perform actions on the labeling page. The Training column tells you whether the model can be used for training, which means the model state is updated based on the submitted annotations.

Those three are genuinely different capabilities and the matrix is not uniform. The bert_classifier example supports pre-annotation and training but not interactive mode. The gliner example is the opposite way round, interactive mode and training but no pre-annotation. The grounding_dino and grounding_sam examples support interactive mode only. The easyocr and flair examples support pre-annotation only. The langchain_search_agent is the only row in the table with all three ticked.

A separate column records whether the model handles arbitrary labels or a set of labels. The docling example produces set labels for layout categories, and the easyocr example produces set labels for characters, while the rest are arbitrary. That distinction matters for a task designer, because an arbitrary-label model can propose a label the schema did not define.

The Required parameters column lists the environment variables each example needs. Most are None, but docling needs DOCLING_SERVICE_URL, DOCLING_SERVE_API_KEY and LABEL_STUDIO_URL for uploads, and the langchain_search_agent needs OPENAI_API_KEY, GOOGLE_CSE_ID and GOOGLE_API_KEY.

## The example gallery covers four kinds of model

Reading across the model names gives you a good sense of what the SDK is designed to accommodate, because the range is wider than typical model serving.

The text models are the most straightforward group. bert_classifier does text classification with Hugging Face transformers, huggingface_ner does named entity recognition with the token classification task, and flair does NER with the flair library. These are the examples where the SDK adds the least, because the model is a callable function and the interesting part is the Label Studio protocol around it.

The vision group is where the diversity shows. easyocr is automated OCR with EasyOCR, grounding_dino is object detection with prompts, and grounding_sam combines GroundingDINO with SAM 2 for object detection with segmentation. The docling example is different again, using the IBM Docling SaaS service through a DoclingServiceClient to produce layout as canonical rectanglelabels and polygonlabels regions. That one is not a local model at all, it is a client for a hosted service, which tells you the SDK does not assume the inference happens in your container.

The language model group has three members. huggingface_llm does inference with Hugging Face text generation, llm_interactive does prompt engineering with OpenAI and Azure LLMs, and langchain_search_agent is a retrieval-augmented pipeline with Google Search and LangChain.

And there is one deliberately trivial example, interactive_substring_matching, which is described as simple keyword search. It is there as a reference implementation, and it is genuinely useful as one, because it is the shortest possible thing that still speaks the protocol correctly.

The concrete libraries the examples depend on are also listed in requirements.txt, which is short: Flask above 2.3, colorama, requests, semver, and the label-studio-sdk pulled directly from its GitHub repository rather than from PyPI. The SDK dependency is the piece that matters most, because it is the client your server uses to talk back to Label Studio.

## A single Flask service, a single console entry point

The packaging is conventional Python, and looking at the setup.py tells you what the runtime is made of.

The console script entry point is label-studio-ml=label_studio_ml.server:main, so the server is started through a module inside the package rather than through a separate script file. The package metadata is read from the label_studio_ml module itself, with setup.py printing the package name and version at build time, which means the version lives in one place.

The requirements are deliberately minimal. Flask above 2.3 is the web framework, and everything else is supporting: colorama for coloured console output on Windows, requests for outbound HTTP, semver for version handling, and the label-studio-sdk. Python 3.8 or newer is required, and the classifiers list the package as OS independent and as a Python 3 project.

The SDK is installed from a git URL rather than from PyPI, with a commented-out pin to label-studio-sdk==1.0.2 left in the file. That is worth a second look. A git dependency without a pinned ref means the installed SDK tracks the tip of that repository, so a backend installed today and one installed next month can behave differently, and a release build of your backend is not reproducible. Whether that is acceptable depends on how you deploy.

The repository layout is a normal Python project: label_studio_ml/ for the package, tests/ for the test suite, deploy/ for deployment material, a MANIFEST.in for package data, a requirements.txt, a setup.py, a Makefile, and a codecov.yml. There is also a .rules/ directory, which is a non-standard entry and suggests project-specific conventions kept outside the standard tooling.

## Where this approach stops being the right one

The boundary the documentation draws first is the one already mentioned, and it is the sharpest: for static pre-annotated data, use the predictions import instead of a backend. A service, a container and an HTTP hop are a real cost for a task that is a one-time import.

The second boundary is the training column. Of the models in the table, only a handful support training, meaning the model state is updated from the annotations people submit. That is a fundamentally different requirement from serving predictions, and it implies the model has to hold state that survives between requests and knows how to absorb a feedback format it did not produce. A model that is a thin wrapper around a hosted API, such as the docling example, cannot do it at all, and the table says so rather than leaving you to find out.

The third boundary is operational. Every example runs as a separate service that Label Studio has to reach, which means in a real deployment you are running a model container per model, with its own GPU or its own API keys, alongside Label Studio itself. The examples ship as docker-compose files, which is convenient for a trial and a starting point for production, but the compose files are examples rather than a deployment story, and the deploy/ directory is where a serious setup would start.

A fourth point is the scope of the SDK. This repository is the backend, not Label Studio itself, and it is not a general model serving system. If you want batching, autoscaling, model registry, or a prediction cache, none of that is here, and pulling it in from elsewhere means running two systems rather than one.

## Conclusion

label-studio-ml-backend is the right pick for a team that already runs Label Studio and wants a model in the loop for pre-annotation, interactive suggestions or retraining on submitted annotations, because the example gallery shows the whole surface and the quickstart is three commands; it is the wrong pick if you only need to load pre-annotated data into Label Studio, since the documentation itself points you at the predictions import instead, and before you deploy you should set LABEL_STUDIO_URL and LABEL_STUDIO_API_KEY, because the backend cannot reach the media data without them.

## FAQ

### How do I start a Label Studio ML backend?

Clone the repository, change into the example directory for the model you want under label_studio_ml/examples, and run docker-compose up. The service is then served at http://localhost:9090, which is the address you point Label Studio at.

### What environment variables does the Label Studio ML backend need?

In most cases you need LABEL_STUDIO_URL and LABEL_STUDIO_API_KEY so the backend can access the media data in Label Studio. Some examples need more: docling requires DOCLING_SERVICE_URL, DOCLING_SERVE_API_KEY and LABEL_STUDIO_URL for uploads, and langchain_search_agent requires OPENAI_API_KEY, GOOGLE_CSE_ID and GOOGLE_API_KEY.

### Do I need the ML backend to import pre-annotated data?

Probably not. The documentation says that if you just need to load static pre-annotated data into Label Studio, running an ML backend might be overkill, and you can import pre-annotated data instead.

### What is the difference between pre-annotation, interactive mode and training?

Pre-annotation means pre-annotated data appears when you open the labeling page or after running predictions for a batch. Interactive mode means you see interactive predictions while performing actions on the labeling page. Training means the model state is updated based on the submitted annotations.

### What runtime does the Label Studio ML backend use?

Flask above 2.3, with colorama, requests and semver as supporting dependencies and the label-studio-sdk installed directly from its GitHub repository rather than from PyPI. The package requires Python 3.8 or newer and installs a console script named label-studio-ml.

## Sources

- [HumanSignal/label-studio-ml-backend on GitHub](https://github.com/HumanSignal/label-studio-ml-backend)
- [Issues](https://github.com/HumanSignal/label-studio-ml-backend/issues)
- [License: Apache-2.0](https://github.com/HumanSignal/label-studio-ml-backend/blob/master/LICENSE)
- [README](https://github.com/HumanSignal/label-studio-ml-backend/blob/master/README.md)
- [Releases](https://github.com/HumanSignal/label-studio-ml-backend/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/humansignal-label-studio-ml-backend