# yolort: a YOLOv5 runtime that keeps preprocessing inside the model graph

> yolort is a Python reimplementation of YOLOv5 built for deployment, using dynamic shapes so that letterbox preprocessing and non maximum suppression can live inside the exported graph for LibTorch, ONNX Runtime, TVM and TensorRT. It suits teams that must ship a detector to a non PyTorch runtime, and it will frustrate anyone who needs current YOLOv5 features or a maintained release cadence.

**zhiqwang/yolort** — yolort is a runtime stack for yolov5 on specialized accelerators such as tensorrt, libtorch, onnxruntime, tvm and ncnn.

- Repository: https://github.com/zhiqwang/yolort
- Website: https://zhiqwang.com/yolort
- Stars: 730 · Forks: 149
- Language: Python
- License: GPL-3.0
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/zhiqwang-yolort

## Why the model graph, not the Python code, does the preprocessing

yolort is presented as yet another implementation of the Ultralytics YOLOv5 detector, with one significant difference from the original. The project adopts a dynamic shape mechanism, and inside that mechanism it embeds both the preprocessing step, the letterbox resize, and the postprocessing step, non maximum suppression, into the model graph.

That choice is the whole point. In a conventional PyTorch pipeline the letterbox resize lives in Python, the NMS lives in Python, and both have to be reimplemented by hand for every backend you deploy to. A C++ inference target therefore inherits a pile of image preprocessing code that has nothing to do with the detector weights.

By moving both operations inside the graph, the exported artifact becomes self contained. The stated consequence is that it simplifies the deployment strategy and makes it easier to deploy the object detection on LibTorch, ONNX Runtime, TVM, TensorRT and similar targets.

The project names its design principle explicitly. It follows the design principle of DETR, and the quoted line is that object detection should not be more difficult than classification, and should not require complex libraries for training and inference. The README also notes that yolort now adopts the same model structure as the official YOLOv5.

## Installing yolort on top of an existing PyTorch install

The installation path is short because the project deliberately has no extra compiled components and keeps its package dependencies minimal. PyTorch and torchvision are not bundled. The instructions say to follow the official PyTorch instructions first, and to install PyTorch 1.8.0 or later and torchvision 0.9.0 or later. The requirements file repeats that PyTorch and torchvision should not be added as package dependencies and should come from the official instructions.

From PyPI the install is a single command:

```shell
  pip install -U yolort
```

The from source route clones the repository and installs in editable mode:

```shell
  # clone yolort repository locally
  git clone https://github.com/zhiqwang/yolort.git
  cd yolort
  # install in editable mode
  pip install -e .
```

The README also notes the Python support range as 3.6 through 3.10 on the version badge, which is worth checking against your runtime, and it flags that PyTorch should be installed before yolort so the correct build for your platform and CUDA version is chosen.

Evaluation on COCO needs one more package, pycocotools, and the documented install pulls it from the cocoapi source tree rather than from PyPI:

```shell
  pip install -U 'git+https://github.com/ppwwyyxx/cocoapi.git#subdirectory=PythonAPI'
```

The README suggests cloning the source if you want the latest functionality rather than the published package, which matters here for the reasons in the maintenance section.

## Running inference in Python and loading an official YOLOv5 checkpoint

The Python API is deliberately small. The models are imported from the yolort.models namespace, and the same object serves for inference and for loading weights.

```python
  from yolort.models import yolov5s

  # Load model
  model = yolov5s(pretrained=True, score_thresh=0.45)
  model.eval()

  # Perform inference on an image file
  predictions = model.predict("bus.jpg")
  # Perform inference on a list of image files
  predictions = model.predict(["bus.jpg", "zidane.jpg"])
```

The score threshold is a constructor argument, and the model is switched to evaluation mode before predicting. The predict method takes a single path or a list of paths.

The second entry point is torch.hub, which lets you pull the pretrained model straight from the repository without installing anything:

```python
model = torch.hub.load("zhiqwang/yolort:main", "yolov5s", pretrained=True)
```

The third is the bridge to the official upstream project. The YOLOv5 class can load a checkpoint trained with ultralytics/yolov5:

```python
from yolort.models import YOLOv5

# Download checkpoint from https://github.com/ultralytics/yolov5/releases/download/v6.0/yolov5s.pt
ckpt_path_from_ultralytics = "yolov5s.pt"
model = YOLOv5.load_from_yolov5(ckpt_path_from_ultralytics, score_thresh=0.25)

model.eval()
img_path = "test/assets/bus.jpg"
predictions = model.predict(img_path)
```

That third path is the one that decides whether you need to retrain anything. If your weights already exist upstream, you point the loader at the checkpoint file and keep them.

## Deployment paths for LibTorch, ONNX Runtime, TVM and TensorRT

The deployment section of the documentation is organised by backend, and the sequence of additions recorded in the changelog shows how the back ends accumulated.

For LibTorch, a tutorial demonstrates how the model is converted into torchscript, and the repository provides a C++ example of inference with the serialized torchscript model, under the deployment/libtorch directory. For ONNX Runtime, a deployment pipeline is provided, and the runtime class PredictorORT is imported from yolort.runtime with an engine path and a device argument:

```python
from yolort.runtime import PredictorORT

# Load the serialized ONNX model
engine_path = "yolov5n6.onnx"
y_runtime = PredictorORT(engine_path, device="cpu")
```

The recorded feature history is a good guide to what the project considers proven. Support for exporting to TorchScript came on Oct. 8, 2020, a LibTorch C++ inference example on Nov. 4, 2020, graph visualization tools on Nov. 21, 2020, export to ONNX with ONNX Runtime Python inference on Nov. 17, 2020, TVM compile and inference notebooks on Feb. 5, 2021, an ONNX Runtime C++ interface example on Sep. 24, 2021, export to TensorRT with TensorRT Python inference on Dec. 25, 2021, and a TensorRT C++ interface example on Dec. 27, 2021.

So TorchScript and ONNX came first, and TVM, TensorRT and the C++ interfaces came later. The graph visualization tools are the interesting extra: they let you inspect what the exported graph actually contains, which is the practical way to confirm that the letterbox and NMS nodes ended up where you expect.

## The version story: tagged releases end in 2022 while the source says 0.7.0a0

The release history and the source version do not line up, and any adoption plan has to deal with that.

The listed releases are v0.6.3, described as a TensorRT Windows compatibility patch release, on 2022-05-04; v0.6.2, compatibility updates for PyTorch 1.11.0 and AutoAnchor support, on 2022-03-12; and v0.6.1, connecting with TorchVision's model banks, on 2022-03-08. Meanwhile setup.py carries the version string 0.7.0a0, which is an alpha marker rather than a published release.

The version in setup.py is built dynamically. It starts from that literal, then if the BUILD_VERSION environment variable is set it uses that value instead, and otherwise it appends the short git SHA of the checked out commit. A write_version_file function then writes the package version module with both the version string and the git version, and it records whether the installed torchvision extension reports a CUDA build.

The practical read for a consumer is that the published wheels and the current source are different things. If you pip install yolort you get a 0.6.x line build, and if you clone and install in editable mode you get 0.7.0a0 plus a commit hash, which is a different code base than the one the older PyTorch and TensorRT compatibility notes refer to.

## Maintenance state and the GPL-3.0 question

The repository is not archived. The last push was on 2026-04-10, which is more than six months before this article, and the tagged releases stop at 2022-05-04.

That gap is the single most important fact for an adopter. The project is not dead in the sense of being abandoned, since pushes continued into 2026, but it is not on a release cadence either. A team that wants a maintained YOLOv5 with new architectures and new upstream weights is looking at the upstream project instead, and this repository's value is the export and deployment layer, not the detector itself.

The licence is GPL-3.0, with the LICENSE file at the repository root. That is the permissive end of the spectrum compared with a proprietary detector, and it matters less for internal use than for distribution: a team shipping a product that links this code needs to check the copyleft obligations against their own distribution model, and that is a question for legal review rather than something this article can settle.

The build configuration is also worth noting for anyone planning a reproducible build. pyproject.toml configures usort with first party detection disabled and black with a line length of 110 and a target version of py37. The project carries a pre-commit configuration, continuous integration workflows, a codecov badge, and separate workflows for building and deploying the documentation.

## Where yolort is the wrong choice

Three cases stand out.

First, if you are staying in PyTorch. The whole value of this project is the export path to other runtimes, and if your deployment target is PyTorch itself then the official YOLOv5 repository gives you the same model structure without the indirection.

Second, if your inference runtime rejects the graph you produce. The design bets everything on dynamic shapes and on a graph that contains the NMS operator and the letterbox preprocessing. Backends differ widely in which operators they implement and how they handle a dynamic batch or dynamic spatial dimensions, and the graph visualization tools exist precisely so you can inspect the result. A runtime that only accepts a fixed shape, or that has no NMS implementation, will not accept this export without modification, and the documentation does not describe a fallback path that moves those operations back out of the graph.

Third, if you need an actively released detector with recent model families. The listed releases stop in 2022 and the source version is an alpha. A product team that needs ongoing upstream improvements and security fixes in the detection stack should not expect this repository to provide them.

There is also a licensing boundary worth naming. GPL-3.0 is fine for internal research and for most company tools, but if the deployment plan involves shipping the runtime inside a closed source product, that conversation needs to happen before the export pipeline is built, not after.

## Conclusion

Adopt yolort when your deployment target is a non PyTorch inference runtime and you need preprocessing and non maximum suppression to survive the export, which is the problem the project was built to solve. Do not adopt it for a pure PyTorch workflow or for a team that needs recent YOLOv5 releases, since the listed tags stop at v0.6.3 from 2022-05-04 while the source version string reads 0.7.0a0. Before you start, confirm that your runtime accepts a dynamic shape graph containing the NMS operator, check that PyTorch 1.8.0 or later and torchvision 0.9.0 or later are already in your environment, and decide early whether to consume the official YOLOv5 checkpoint through load_from_yolov5 or to train inside yolort itself.

## FAQ

### What is yolort and how is it different from YOLOv5?

yolort is a Python implementation of Ultralytics YOLOv5 that uses dynamic shapes so that letterbox preprocessing and non maximum suppression can be embedded in the model graph. That is what lets it be exported to backends such as LibTorch, ONNX Runtime, TVM and TensorRT.

### How do I install yolort?

Install PyTorch 1.8.0 or later and torchvision 0.9.0 or later first using the official PyTorch instructions, then run pip install -U yolort. The project has no extra compiled components and keeps its package dependencies minimal.

### Can yolort load checkpoints trained with the official ultralytics/yolov5?

Yes. The YOLOv5 class provides a load_from_yolov5 method that takes the path to an official checkpoint, so weights trained upstream can be used without retraining.

### Which PyTorch and Python versions does yolort support?

The documentation asks for PyTorch 1.8.0 or later and torchvision 0.9.0 or later, and the version badge lists Python 3.6 through 3.10. The requirements file repeats that PyTorch and torchvision should be installed from the official instructions rather than added as dependencies.

### What licence does yolort use?

The project is licensed GPL-3.0, with a LICENSE file at the repository root. The listed releases are v0.6.3 from 2022-05-04, v0.6.2 from 2022-03-12 and v0.6.1 from 2022-03-08, and the last push was on 2026-04-10.

## Sources

- [License: GPL-3.0](https://github.com/zhiqwang/yolort/blob/main/LICENSE)
- [Project website](https://zhiqwang.com/yolort)
- [README](https://github.com/zhiqwang/yolort/blob/main/README.md)
- [Releases](https://github.com/zhiqwang/yolort/releases)
- [zhiqwang/yolort on GitHub](https://github.com/zhiqwang/yolort)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/zhiqwang-yolort