RF-DETR: A DINOv2-Backed Detector You Fine-Tune Rather Than Train From Scratch
RF-DETR is a real-time object detection and segmentation model architecture developed by Roboflow, SOTA on COCO, designed for fine-tuning. [ICLR 2026]
At a glance
- What is it?
- Roboflow's rfdetr package wraps a DINOv2 vision transformer backbone behind a single API for detection, segmentation and preview keypoints. The Apache 2.0 core is the part worth evaluating; the XL and 2XL weights sit under a different licence.
- Who is it for?
- Adopt rfdetr if you have a labelled detection or segmentation dataset and want a COCO-strong starting point you can fine-tune in a Python>=3.10 environment, and if the Apache 2.0 sizes (N through L) cover your accuracy target. Do not adopt it if you need the XL or 2XL accuracy and cannot accept PML 1.0 terms, or if you need a model small enough for embedded inference, since even RF-DETR-N carries 30.5M fused parameters.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem RF-DETR targets: fine-tuning, not architecture research
Training a detection transformer from scratch is expensive and usually unnecessary. Most teams arrive with a few thousand labelled images and a deadline. RF-DETR is positioned for exactly that situation: a real-time transformer architecture, built on a DINOv2 vision transformer backbone, that the README describes as designed for fine-tuning and reported as state-of-the-art on Microsoft COCO and RF100-VL. The repository also exposes a single API across detection, instance segmentation, and keypoint detection, the last marked preview. That combination is the pitch. You are not expected to design a backbone or run a long pretraining schedule; you are expected to point the trainer at a dataset and let the pretrained representation do the heavy lifting. The audience is therefore narrower than the topic list suggests. It is engineers who already have annotations, a GPU, and a decision to make about which detector to fine-tune, not researchers looking for a new architecture to extend.
DINOv2 backbone, one API, and three task heads
The architecture is a DETR-style transformer detector sitting on a DINOv2 vision transformer backbone. The README states this directly and does not go further into the decoder design, so the internal attention structure is not something this material lets me describe. What is visible is the surface: the package supports object detection, instance segmentation, and keypoint detection through what the README calls a single, consistent API. The published model sizes were produced by neural architecture search, and the same NAS method is offered on the Roboflow platform for finding an architecture for your own dataset. That last point matters for how you read the size table. RF-DETR-N through RF-DETR-L are the NAS-discovered Apache 2.0 configurations, at 384x384, 512x512, 576x576 and 704x704 input resolutions respectively. RF-DETR-XL and RF-DETR-2XL are the Plus tier, shipped in a separate rfdetr_plus package under PML 1.0, at 700x700 and 880x880. The licence boundary is drawn along the size axis, not the task axis, so choosing a bigger model changes your legal position as well as your accuracy.
Reading the benchmark table before you trust it
The README publishes a detection table comparing RF-DETR sizes against YOLO11 variants. Two methodological notes are stated there and they change how the numbers should be used. First, all COCO accuracy figures were measured in-house for every model shown, computed with pycocotools in SAB over the full 5,000-image val2017 split, and may differ from vendor-reported figures; rows marked with a dagger are quoted from the original authors' paper instead. Second, latency was measured on an NVIDIA T4 using TensorRT, FP16, and batch size 1. Parameter counts are deployment (fused) nn.Module counts from model.parameters(), not the raw tensor count of the saved checkpoint. The practical consequence is that the latency column tells you about a TensorRT FP16 deployment path on dated hardware, not about a PyTorch eager forward pass on your machine. Treat the table as a ranking signal and measure your own stack. The accuracy gap it reports is large: RF-DETR-N is listed at 48.4 COCO AP50:95 against 37.4 for YOLO11-N, and RF-DETR-L at 56.5 against 49.9 for YOLO11-L. Parameter counts run the other way. RF-DETR-N carries 30.5M fused parameters against 2.6M for YOLO11-N, and the Apache 2.0 sizes cluster tightly between 30.5M and 33.9M.
Getting it running: install, source build, and the API surface
The install path is one command in a Python>=3.10 environment: pip install rfdetr. If you want unreleased work, the README gives a source install from the develop branch via pip install https://github.com/roboflow/rf-detr/archive/refs/heads/develop.zip, with an explicit warning that those updates are still in development and may not be as stable as the latest published release. The default branch is develop, which is consistent with that warning: what you get from the repository archive is not the same as what pip resolves. The README shows the training entry point as a fine-tuning flow against a dataset, with a Colab notebook linked for fine-tuning RF-DETR on a detection dataset and a Hugging Face Space for trying the model. It does not, in the material available here, spell out the full set of CLI flags or config keys, so I cannot quote them. What I can say is that the package is distributed on PyPI under the name rfdetr, that Plus components live in a separate rfdetr_plus distribution, and that the documentation site at rfdetr.roboflow.com is where the configuration details would live. Verify the exact training arguments there before writing a pipeline around them.
Where RF-DETR is the wrong tool
The parameter counts are the clearest limitation. RF-DETR-N at 30.5M parameters and 384x384 input is the smallest Apache 2.0 option, and it is roughly an order of magnitude heavier than a YOLO11-N at 2.6M. If your target is a Jetson-class device, a mobile NPU, or a browser runtime, that gap is not a tuning problem. There is no nano-scale RF-DETR in the published table. The second limitation is the licence split. The two highest-accuracy configurations, RF-DETR-XL at 58.6 COCO AP50:95 and RF-DETR-2XL at 60.1, are PML 1.0, not Apache 2.0. If your accuracy target sits above what RF-DETR-L delivers, you are no longer evaluating an Apache 2.0 project. The third is keypoint detection, which the README labels preview. Preview means the API may move and the quality bar is not the same as the detection path. Building a product on it now means accepting that. Finally, the benchmark latency was measured on an NVIDIA T4 with TensorRT FP16 at batch size 1, so any deployment that does not use TensorRT, or that batches differently, should not expect those numbers to transfer.
How this differs from a YOLO-style detector
The comparison the README itself draws is against YOLO11, and the difference in approach is structural rather than incremental. YOLO11 is a convolutional detector family, with the smallest variant at 2.6M parameters and AGPL-3.0 licensing. RF-DETR is a transformer detector with a DINOv2 backbone, and the README frames the whole project around fine-tuning that pretrained representation. The trade the table shows is consistent: RF-DETR buys accuracy per frame at a much higher parameter cost, and it buys that accuracy with a permissive licence on the smaller sizes while YOLO11 carries AGPL-3.0 across the family. For a team shipping a closed-source product, that licence difference is often the deciding factor before accuracy is even considered. For a team shipping to constrained hardware, the parameter difference decides it in the other direction. Neither is universally better; they sit at different points on the accuracy, weight and licence axes.
Maintenance, release cadence, and the licence boundary
The release history shows a steady cadence through 2026: 1.9.4 with export and augmentation correctness fixes, 1.10.0 titled Faster Training & Inference, and 1.10.1 with CUDA compile fixes and memory reduction, all within roughly two weeks. That is a project still actively changing its training and inference internals, which is good for correctness and bad for anyone pinning behaviour. The 1.10.1 CUDA compile fixes in particular suggest that building against specific CUDA and PyTorch combinations has been a moving target. Budget for version pinning and for re-validating after upgrades rather than assuming a drop-in bump. On licensing: the open-source rfdetr package and the Apache-designated models are Apache 2.0, while Plus components including RF-DETR-XL and RF-DETR-2XL are PML 1.0. Those are two different distributions with two different sets of terms. I am not in a position to interpret PML 1.0 for your use case; read the licence text and, if the distinction affects your product, get it reviewed. The practical point is that the licence you operate under is determined by which model size you load, so the decision has to be made before the training run, not after.
Editorial conclusion
Adopt rfdetr if you have a labelled detection or segmentation dataset and want a COCO-strong starting point you can fine-tune in a Python>=3.10 environment, and if the Apache 2.0 sizes (N through L) cover your accuracy target. Do not adopt it if you need the XL or 2XL accuracy and cannot accept PML 1.0 terms, or if you need a model small enough for embedded inference, since even RF-DETR-N carries 30.5M fused parameters. Before committing, verify three things on your own data: your measured latency for the chosen size at your resolution, whether the keypoint path is stable enough for production given it is marked preview, and how the segmentation head behaves on your smallest objects.
Community notes