Adjust a real-time object detection and segmentation model to your own data
This tool handles finding specific objects in camera footage, drawing their precise outlines, and identifying human joint positions. Developed by Roboflow, RF-DETR is a model architecture designed to perform these computer vision tasks in real time. Presented at ICLR 2026, the model shows top-tier accuracy on the COCO dataset and is optimized for fine-tuning on your own datasets.
ICLR 2026 presentation and real-time performance
RF-DETR is a recent model presented at ICLR 2026 that prioritizes the balance between real-time processing and accuracy. Based on the DINOv2 vision transformer, it recorded outstanding performance compared to other real-time models on the Microsoft COCO and RF100-VL benchmarks. All COCO accuracy figures were measured directly on the full set of 5,000 val2017 images using pycocotools, while latency was measured on an NVIDIA T4 GPU with TensorRT, FP16, and a batch size of 1. This provides a consistent baseline that can be directly compared with vendor-reported figures.
Detection, segmentation, and keypoints: one API
This repository supports object detection, instance segmentation, and keypoint detection (preview). Each task follows the same API structure, so you do not need to significantly change your code structure when switching from a detection model to a segmentation model. Detection models are available in various sizes from Atto to 2XLarge, segmentation models range from Nano to 2XLarge, and keypoint models are currently available as a preview version. All models can be easily loaded and used in a Python environment through the rfdetr package.
License distinctions and commercial use conditions
Licensing varies by model size and component. The rfdetr package itself, along with Nano, Small, Medium, and Large detection models, all sizes of segmentation models, and keypoint models, are under the Apache 2.0 license. This means they can be used freely without license costs in closed-source commercial products or self-hosted environments. In contrast, Atto, Femto, Pico, XL, and 2XL detection models are included in the rfdetr[plus] extension package and follow the Roboflow Platform Model License (PML 1.0). Commercial use is included when deploying these models on the Roboflow platform, but an Enterprise add-on is required for commercial deployment via self-hosting.
Finding the right architecture for your data
The public model sizes of RF-DETR were generated through Neural Architecture Search (NAS). The Roboflow platform offers the same NAS method, allowing you to directly discover the architecture best suited to your specific dataset. A single training run yields results for all model sizes, and this process has been improved to deliver better performance than open-source checkpoints and NAS results reported in previous papers. This is a useful feature for developers who want to build a model optimized for their data characteristics rather than using a standard model.
Pre-use considerations
A Python 3.10 or higher environment is required, and base models are installed with pip install rfdetr. If you plan commercial self-hosting and use Atto, Femto, Pico, XL, or 2XL models, you must review the terms of the PML 1.0 license. Keypoint detection is still in the preview stage, so you need to verify stability before applying it to production environments. Additionally, since benchmark figures were measured on specific hardware (NVIDIA T4) and conditions, latency may vary depending on the hardware specifications of your actual deployment environment.