LightlyTrain: A Single Python Entry Point for Pretraining, Fine-tuning and Distillation

The gap LightlyTrain is aimed at: unlabeled images and a small labelled set

Most vision teams end up with more images than labels. LightlyTrain's pitch is that you can pretrain a DINOv2 or DINOv3 vision foundation model on that unlabeled pool, then fine-tune a detector or segmentation model on the labelled subset. The README frames it as covering "the entire model development lifecycle from pretraining DINOv2/v3 vision foundation models on your unlabeled data to fine-tuning transformer and YOLO models on detection and segmentation tasks for edge deployment." The audience is therefore teams with domain imagery that generic COCO or ImageNet weights do not fit, and a deployment target where latency and model size matter. The topic list on the repository points at the same scope: object detection, semantic segmentation, instance segmentation, depth estimation, distillation and embeddings. If your problem is a plain image classification job on a large labelled dataset, this is more machinery than you need.

What the package actually contains: methods, backbones and export paths

The repository is a Python library with a docs site at docs.lightly.ai/train, a Docker image referenced from the badges, and a changelog that carries the release history. The methods named in the README and news entries are: DINOv2 pretraining (added in 0.9.0), DINOv3 support for pretraining via distillation (0.11.0), Distillationv3 (0.15.0), DINOv2 semantic segmentation fine-tuning (0.10.0), EoMT semantic segmentation with a DINOv3 backbone, LTDETR and LTDETRv2 object detection, LTDETRv2 instance segmentation (0.17.0), and PicoDet for low-power embedded devices (0.14.0). Backbones include DINOv2, DINOv3, EdgeCrafter ECViT, ConvNeXt variants and YOLO. Export is part of the story: the 0.16.0 and 0.17.0 notes mention ONNX and TensorRT export out of the box, and 0.14.0 added FP16 precision for ONNX/TensorRT export across all tasks. One claim in the 0.17.0 note is worth flagging rather than repeating: the release text says LTDETRv2 instance segmentation matches the accuracy of the original ECSeg implementation while being 10-20% faster. That number comes from the project's own release note, not from an independent run.

How the pieces fit together, as far as the material shows

The material does not include an architecture diagram or a module map, so the internal data flow cannot be described from source. What can be said is structural. There is a pretraining or distillation stage that consumes unlabeled images and produces a backbone checkpoint. There is a fine-tuning stage that consumes that checkpoint plus a labelled dataset and produces a task model. There is an export stage that turns the task model into ONNX or TensorRT, optionally in FP16. The pretrained checkpoints named in the tables (for example dinov3/vitt16-ltdetr-coco) are hosted artifacts you can start from without running pretraining at all, which means the pipeline is not all-or-nothing: you can enter at fine-tuning and skip the unlabeled-data stage entirely. The README's model table is the clearest evidence of how the stages connect, because each row pairs a backbone name with a task and a set of deployment numbers.

The COCO table is the most concrete thing in the repository

The README publishes single-scale COCO 2017 validation results for the detection models. LTDETRv2 small reports 50.7 mAP at 5.4 ms latency with 9.9M parameters at 640x640; the medium variant reports 53.1 mAP at 7.95 ms and 21.1M parameters; the large variant reports 56.0 mAP at 10.78 ms and 33.6M parameters. The older DINOv3 LTDETR rows sit alongside them: convnext-tiny at 54.4 mAP with 61.1M parameters and 13.3 ms, convnext-large at 60.0 mAP with 230M parameters and 42.3 ms. The README states latency is measured with TensorRT on an NVIDIA T4 at batch size 1, with models optimized using tensorrt==10.13.3.9. Two things follow from reading the table rather than skimming it. First, the accuracy-versus-size trade-off is steep: the convnext-large row buys 3.3 mAP over the small LTDETRv2 row at roughly 23 times the parameters and about eight times the latency. Second, a T4 at batch size 1 is a specific deployment profile. Your numbers on a different GPU, a different batch size, or an ONNX runtime instead of TensorRT will not match, and the README does not claim they will.

Getting it running: install, then check the docs for your version

The install path in the README is one command: pip install lightly-train, on Python 3.8 through 3.13, on Windows, Linux or MacOS. A Docker image is referenced from the badge block and documented at docs.lightly.ai/train/stable/docker.html. Beyond that, the README does not show a training invocation, so the actual entry points (the CLI subcommands, the Python API calls, and the config keys for batch size, backbone selection, dataset path and export format) have to come from the documentation site rather than from this repository page. That is a real friction point for evaluation: you cannot judge the ergonomics of the training loop from the README alone. The one configuration detail that is stated explicitly is the TensorRT version used for the published latency numbers, tensorrt==10.13.3.9, which is the version to pin if you want to compare your own measurements against the table.

Licence is the decision, not an afterthought

LightlyTrain is AGPL-3.0. The README states plainly: "Using LightlyTrain at work, in production, on the edge, or to build proprietary models? You likely need a Commercial License." For a library that ends up linked into a deployed inference artifact, AGPL-3.0 is a materially different proposition from a permissive licence, and the project itself is directing commercial users to a separate agreement. This is not a detail you resolve after the proof of concept. If your model weights or your service are proprietary, the licence question determines whether the project is usable at all, and the answer is a legal one that depends on how you link, distribute and expose the software. I am not in a position to give that advice, and neither is the README; it says "likely," which is an invitation to ask, not a clearance. Treat the licence as the first gate, not the last checkbox.

Where it is the wrong tool

The scope is vision, and specifically the tasks the docs enumerate. If your problem is tabular, text, audio, or a vision task outside the documented set, nothing here helps. The pretraining stage assumes you have unlabeled images in sufficient quantity to be worth the compute; if you have a few thousand labelled images and no unlabeled pool, the fine-tuning entry point is the only relevant part, and the pretraining machinery is dead weight. The published numbers are single-scale COCO validation on a T4, so a team deploying to a CPU-only edge device or a non-NVIDIA accelerator has no evidence in this repository about what to expect. The README also does not discuss distributed training, checkpoint resumption, or how much GPU memory a given backbone needs, and those are the questions that decide whether a training run finishes. Finally, the release cadence visible in the news list is fast: 0.9.0 in July 2025, 0.17.0 in July 2026. Frequent minor releases with new model families mean API surface can move, so pinning a version and reading that version's docs is the safer path than tracking main.

Alternatives and the difference in approach

The most direct alternative for the fine-tuning half is the Ultralytics YOLO package, which the README's own topic list places in the same space. The difference is where each puts the pretraining. Ultralytics starts from its own pretrained YOLO checkpoints and treats fine-tuning as the whole workflow; LightlyTrain adds a self-supervised pretraining and distillation stage in front of fine-tuning so you can adapt a foundation backbone to your unlabeled domain before the labelled task training begins. That extra stage is the product. If your domain is close to COCO, the pretraining stage buys you less and Ultralytics-style fine-tuning is the shorter path. For the segmentation half, the same logic applies against generic DINOv2 or DINOv3 fine-tuning scripts: those give you a backbone and a loss, while LightlyTrain gives you the EoMT and LTDETR heads, the export path, and the published COCO comparison rows in one package. The trade is that you accept AGPL-3.0 and the project's release cadence in exchange for that integration.

Editorial conclusion

Adopt LightlyTrain if you have unlabeled domain images and a labelled evaluation set, and you want one installed package to cover DINOv2/v3 pretraining, LTDETR or YOLO fine-tuning, and ONNX or TensorRT export. Do not adopt it if your product is closed source and you have no intention of taking the commercial licence the README points to, or if you need a task outside the documented set. Verify first that the task and backbone you need appear in the docs for your pinned version, and confirm the licence position with your own counsel before training anything you intend to ship.

Official sources

1. License: AGPL-3.0
2. lightly-ai/lightly-train on GitHub
3. Project website
4. README
5. Releases