autodistill/autodistill: Roboflow's route from unlabeled images to a small deployed model

Why Roboflow treats human labeling as distillation

Autodistill opens from an unusual premise: that manual annotation is itself a form of distillation from a very capable base model, executed slowly and by hand. The README puts it plainly, arguing that the real engineering problem is the gap between a foundation model that knows a lot about a lot and a production model that has to know a lot about a little. So the big multimodal model does the labeling and the small supervised network learns from the result, which is also the only arrangement that puts inference somewhere the big model cannot reach.

The repository carries the weight of a larger Roboflow product line. setup.py names Roboflow as author and [email protected] as the contact address, the README advertises a hosted labeling service run by the same company, and the banner links four sibling projects together: notebooks, inference, autodistill, and collect. At 2,784 stars and 224 forks with 52 issues open, this is the most visible project in that family.

The two official descriptions say different things, and neither is wrong. The GitHub repository is described as images to inference with no labeling, use foundation models to train supervised models. The package metadata in setup.py describes it as distill large foundational models into smaller, domain-specific models for deployment. The first sells the outcome, the second sells the mechanism, and the README spends most of its length on the second.

Six primitives and one rule about task compatibility

Autodistill names six primitives and wires them into a fixed pipeline. Unlabeled data goes into a Base Model. An Ontology decides how that model is prompted and what the resulting labels are called. The output is a Dataset. A Target Model consumes the Dataset and emits a Distilled Model, which is the set of weights you actually deploy. A Task sits underneath all of it.

The Task is the rule that makes the pluggable design safe. A Task defines what a Target Model will predict, and the Task attached to the Base Model, the Ontology and the Target Model must match before the three can be combined at all. At the moment that means one task, detection, which carries both object detection and instance segmentation through the same code path. The README is direct that classification support is still ahead rather than shipped, and it names an upcoming GPT-4 multimodal variant among example Base Models, which dates the text well before the releases stopped. The repository topic tags already carry image-classification anyway, so the tag list advertises a capability the documentation defers.

Worth noticing is how little working code the README offers. The one end-to-end demonstration is an animated GIF of a Target Model finding milk bottles and bottlecaps after training on an auto-labeled dataset, tied to a YouTube walkthrough rather than to anything in the repository. For a library whose entire argument is that you can skip human labeling, that is a thin empirical case: no accuracy numbers, no comparison against a hand-labeled baseline, and no statement about how many labels a human should still expect to correct before training.

Two dependency lists that describe different machines

The install story has two halves that do not agree. setup.py declares six install requirements: opencv-python at 4.6.0 or newer, supervision with no version bound at all, tqdm, Pillow at 7.1.2 or newer, PyYAML at 5.3.1 or newer, and click. requirements.txt lists eight, and it pins one of them hard:

opencv-python
supervision==0.25.0
tqdm
Pillow>=7.1.2
PyYAML>=5.3.1

Three names appear only in requirements.txt: roboflow, pyyaml and scikit-learn. So a plain install of the published wheel gives you a different dependency set from a checkout installed with pip install -r requirements.txt, and the released package never asks for roboflow or scikit-learn at all. Whether that is a bug depends on which code paths you exercise, since a plugin that reaches for the Roboflow client will fail on the wheel and work on the checkout.

The unpinned supervision line is the quieter problem. The v0.1.26 release note says the change was to use supervision 0.19.0rc3 or newer for better NMS support, and the intent survives in the exact pin requirements.txt settled on. Neither the note nor the packaging metadata states that floor as a constraint, so an install from PyPI can resolve a much newer supervision and meet an API difference at runtime instead of at install time. If you hit something that looks like a supervision regression, that resolution order is the first thing to check.

The rest of the packaging is old-school in ways that matter when something breaks. There is no pyproject.toml, so version discovery reads autodistill/__init__.py with a regular expression, a trick setup.py annotates by linking a decade-old Stack Overflow discussion. python_requires is >=3.7 while the classifier list says only Programming Language :: Python :: 3, with no minor version named. The console script is registered like this:

    entry_points={
        "console_scripts": [
            "autodistill=autodistill.cli:main",
        ],
    },

Three releases, two of them sixteen minutes apart

The release history is short and oddly spaced. v0.1.2, published on 9 June 2023, calls itself the initial release of Autodistill with base and target model architecture for vision models. The numbering then jumps to v0.1.25 and v0.1.26, both published on 13 February 2024, sixteen minutes apart: v0.1.25 added an npm_setting to the label method on DetectionBaseModel, and v0.1.26 updated the supervision dependency for better NMS support.

That was the last tag. The repository's pushedAt is 29 September 2026, so commits have been landing for roughly two and a half years with no new release attached to them. Both facts hold at once and they point in different directions. PyPI hands you v0.1.26, which is well over a year older than the code in the repository and predates every fix since. The main branch gives you current work, but it still reports itself as a 0.1.x version, so nothing in the version string tells you which of the two you actually have.

A workable decision rule: if the capabilities you need existed in early 2024 and you want a reproducible environment, take the tag and pin supervision yourself, because the metadata will not do it for you. If you are chasing a bug fix or a newly added plugin, install from the git URL, expect to track a moving branch, and read the diff between your commit and v0.1.26 before assuming the release notes describe what is running. A third option is to read the issue tracker first, since 52 open issues against three releases is where a stalled project usually shows itself.

A tree that promises more than the packaging confirms

The repository is larger than what it ships. Alongside the README, LICENSE, CONTRIBUTING.md and Makefile there are directories for the autodistill package itself, for automation, for docs and for test, plus mkdocs.yml, CITATION.cff, models.csv and a dog.jpeg sitting at the root. The presence of mkdocs.yml next to a docs directory, next to a homepage pointing at docs.autodistill.com, tells you the documentation is built from this repository, which means the published site tracks main rather than any particular tag.

models.csv is the file that does not line up with the packaging. setup.py carries a comment about allowing models.csv in the package, sets include_package_data to true, and declares package_data for models.csv. The tree places that file at the repository root rather than inside the autodistill package directory, and there is no MANIFEST.in in the tree. Whether the built wheel actually ships the model table is therefore not something the repository answers outright; it turns on how include_package_data resolves a root-level file during the sdist step. If you are consuming autodistill as an installed package instead of a checkout, check your installed files before assuming the catalog is there.

There is a smaller mismatch of the same flavor. setup.py excludes tests when it calls find_packages, but the directory in the tree is called test. Whatever find_packages decides about a directory with no __init__.py, the string in the exclusion tuple does not match the directory name, so that exclusion is not doing anything on purpose.

A lint gate wired to never fail

The Makefile is short and explains how releases are actually cut. A style target runs black and isort over autodistill, and check_code_quality runs black --check, then isort --check-only, then flake8 twice. The first flake8 pass selects only the error classes that indicate real breakage, syntax errors and undefined names, and it can stop a build. The second pass is different:

	# exit-zero treats all errors as warnings. E203 for black, E501 for docstring, W503 for line breaks before logical operators
	flake8 $(check_dirs) --count --max-line-length=88 --exit-zero  --ignore=D --extend-ignore=E203,E501,W503  --statistics

--exit-zero makes that invocation report its findings and always exit successfully, so the stylistic half of the quality gate can never block anything. The comment above it is candid about the reason, which at least means nobody is hiding it. There is also no target that runs the test directory even though test is in the tree, and nothing outside autodistill is linted, since check_dirs is set to that one package. PYTHONPATH is exported as the current directory for the whole file, which is convenient locally and meaningless once the package is installed.

The publish target is the last part worth reading:

publish:
	python setup.py sdist bdist_wheel
	twine check dist/*

Packaging still goes through setup.py with no build backend, twine check runs against whatever happens to be in dist, and the upload line that follows passes credentials as environment variables on the command line, so they are visible in the process list for the duration of the upload. The dev extra pins black at 22.3.0, a March 2022 release, so newly formatted code follows a formatter that is far behind the current one.

Library or hosted service, and how the docs show their age

The README offers two routes and leans toward both. A callout near the top points at the Roboflow hosted version of Autodistill for labeling images in the cloud, while the installation section makes clear that the library is modular: the autodistill package defines the interfaces, and Base Model and Target Model plugins implement specific models. In practice no install is just autodistill, because every pipeline needs at least one plugin on each side of the Task.

The plugin story also explains the documentation layout. A models.csv at the repository root looks like the machine-readable version of a catalog that the site builds from mkdocs.yml and the docs directory, and the README quicklinks point at a supported models section inside the document itself, so a reader can usually find the list without leaving the page. The catalog you get from the repository and the catalog you get from an installed wheel may not be the same set of models, for the packaging reason described earlier.

Where the writing shows its age is in its examples rather than its structure. GPT-4 appears as having an upcoming multimodal variant, and the only demonstration is that GIF of a milk bottle detector. The README does concede the Base Models are not perfect yet, calls the system an early one that needs help testing, and says through creative prompting and few-shotting the useful cases can be widened. Combined with releases that stopped in February 2024 while commits kept coming, the reasonable reading is that the pipeline design settled early and the surrounding project did not.

Editorial conclusion

The real contribution here is the six-primitive contract: Task, Base Model, Ontology, Dataset, Target Model and Distilled Model, tied together by a rule that forces every component in a pipeline to agree on the task. That contract is small enough to read in one sitting, which is why the handful of files that actually ship describe the whole system. The awkward parts sit around the edges: a requirements.txt carrying dependencies the package metadata never mentions, a supervision floor stated in a release note and nowhere in the install metadata, a models.csv whose presence in a wheel depends on packaging behavior nobody documented, and a lint pass that cannot fail the build. None of that touches the pipeline design, and all of it disappears if you install from a checkout and pin supervision yourself. Start by pointing one Base Model plugin at a small folder of images, read the labels before training rather than after, and treat the 2024 tag as a floor rather than as a current version.

Frequently asked questions

Official sources

1. autodistill/autodistill on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases