# rtmlib runs RTMPose without the mmcv stack

> rtmlib repackages the RTMPose family of pose estimation models into a small library that needs only numpy, OpenCV and onnxruntime, so you can run wholebody, body, hand, animal and 3D pose pipelines without installing mmcv, mmpose or mmdet. The trade is that the model zoo is fixed at the URLs the project ships.

**Tau-J/rtmlib** — RTMPose series (RTMPose, DWPose, RTMO, RTMW) without mmcv, mmpose, mmdet etc.

- Repository: https://github.com/Tau-J/rtmlib
- Stars: 691 · Forks: 102
- Language: Python
- License: Apache-2.0
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/tau-j-rtmlib

## The dependency list is the whole pitch

The reason this library exists is stated in the first line of the README. rtmlib is a super lightweight library to conduct human and animal pose estimation based on RTMPose and ViTPose models without any dependencies like mmcv, mmpose, mmdet and so on. The dependency list that follows is short enough to read at a glance, and it is the entire argument. numpy, opencv-python, opencv-contrib-python, onnxruntime. The project metadata agrees, listing numpy, onnxruntime, opencv-contrib-python, opencv-python and tqdm, with an optional test extra that pulls in pytest, and requiring Python 3.10 or newer. The models themselves are ONNX files, which is the reason onnxruntime is enough to run them, and it is also the reason the acceleration options work. The README says you can optionally use other common backends like opencv, onnxruntime and openvino to accelerate the inference process, and TensorRT appears in the topic tags as well, with a note that openvino users need to add the path to the openvino libs directory inside their environment's site-packages into their environment path. The audience is a working engineer who needs pose estimation to work in a product, not a researcher who needs to modify a model.

## Solutions on top, detectors and estimators below

The API is deliberately layered, and the README documents it in three tiers. The high level APIs are the solutions: PoseTracker, Wholebody, Body, Body_with_feet, Hand, Animal, Custom and Wholebody3d, each living under rtmlib/tools/solution/. The low level APIs are the pieces. The detectors are YOLOX for human and multiclass use, RFDETR for human and multiclass, and RTMDet. The pose estimators are RTMPose for 17 keypoints, RTMO for 17 keypoints in a one-stage setup, RTMPose for 21 keypoints for hands, RTMPose for 26 keypoints, RTMW for 133 keypoints, DWPose for 133 keypoints and RTMW3D for 133 keypoints in 3D, plus the ViTPose family for 17, 25 and 133 keypoints and a 17 keypoint animal variant. Visualization is two functions, draw_bbox and draw_skeleton. The layering matters because it decides how much of the model zoo you have to care about. A solution bundles a detector and a pose estimator and hides the pairing, while the low level classes let you pick each half yourself. The Animal solution is the interesting case, since the README notes that the openpose-style skeleton is required for animals, and the to_openpose flag switches the output between the openpose and mmpose keypoint conventions.

## Choosing models by mode, or by det and pose

For the high level APIs you have two ways to specify the models, and the README states that you can pass mode or det plus pose arguments. Choosing by mode is the short version.

```Python
# By mode
from rtmlib import Wholebody
wholebody = Wholebody(mode='performance',  # 'performance', 'lightweight', 'balanced'. Default: 'balanced'
                      backend=backend,
                      device=device)
```

The three modes are performance, lightweight and balanced, with balanced as the default, and they map to a preset pair of detector and estimator models whose sizes and speeds differ. Choosing by det and pose means you name both models yourself, and the model files are given as URLs on download.openmmlab.com along with an input size tuple.

```Python
# By det and pose
from rtmlib import Body
body = Body(det='https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/onnx_sdk/yolox_x_8xb8-300e_humanart-a39d44ed.zip',
            det_input_size=(640, 640),
            pose='https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/onnx_sdk/rtmpose-x_simcc-body7_pt-body7_700e-384x288-71d7b7e9_20230629.zip',
            pose_input_size=(288, 384),
            backend=backend,
            device=device)
```

There is a third form, Custom, which takes the class names as strings alongside the model URLs, so you can mix, for example, a YOLOX detector with an RTMPose estimator. The practical point about the URLs is that they are archives that the library downloads and unpacks, and the input size is given explicitly because the preprocessing depends on it. If your deployment needs a specific model, the by det and pose form is the one to use, and if you want to keep the library's opinion, the mode form is the one that keeps you current.

## A wholebody image example, end to end

The quick start in the README is a single image through the Wholebody solution, and it is short enough to be the whole API in twenty lines. The first thing it shows is that a solution is callable on an image and returns two values, keypoints and scores.

```python
import cv2
from rtmlib import Wholebody, draw_skeleton

img = cv2.imread('./demo.jpg')

device = 'cpu'  # cpu, cuda, mps
backend = 'onnxruntime'  # opencv, onnxruntime, openvino
openpose_skeleton = False  # True for openpose-style (required for animals), False for mmpose-style

wholebody = Wholebody(to_openpose=openpose_skeleton,
                      mode='balanced',  # 'performance', 'lightweight', 'balanced'. Default: 'balanced'
                      backend=backend, device=device)
keypoints, scores = wholebody(img)
```

The scores are what let you filter weak detections, and the drawing step takes a threshold for that, kpt_thr=0.5 in the example, along with the to_openpose flag so the skeleton is drawn in the same convention the keypoints came in. The comment in the example points out that you can swap the original image for a black background of the same shape, which is the trick for a cleaner overlay when the image is busy. The video path uses PoseTracker instead, and the important parameter there is det_frequency, which detects every tenth frame and tracks in between, so the detector is not run on every frame of a stream.

## Tracking, the WebUI and a warning about keypoint conventions

The video example wires PoseTracker to a cv2.VideoCapture, with a comment noting that for a video file you use cv2.VideoCapture('./demo.mp4') instead of index 0. The tracker takes the solution class, the mode, the detection frequency, the backend and the device, and returns the same keypoints and scores pair per frame. A commented block shows how to build a custom class with functools.partial, carrying a det class, a detector URL, a detector input size, a pose class, a pose URL and a pose input size, which is the pattern to copy when you want a specific detector and estimator combination without writing a subclass. The WebUI is a separate script, webui.py, run with python webui.py, and the README notes that gradio has to be installed first, with the pip install gradio line shown as a comment. The keypoint convention issue is the one that catches people. The openpose_skeleton flag is not cosmetic: it selects whether the output uses the openpose style keypoint set or the mmpose style, and the README states that the openpose style is required for animals. If your downstream code indexes keypoints by position, a mismatch here silently produces wrong joints rather than an error, so the flag has to match the consumer of the data, not the model.

## Installing it, and what the packaging history reveals

The package is on PyPI and installs with a single command.

```shell
pip install rtmlib -i https://pypi.org/simple
```

Or from source.

```shell
git clone https://github.com/Tau-J/rtmlib.git
cd rtmlib

pip install -r requirements.txt

pip install -e .
```

The requirements file lists the same five packages, and the source install shows the optional extras commented out, onnxruntime-gpu and openvino. The repository layout shows what a working install looks like on disk, with a rtmlib/ package directory, a tests/ directory, a uv.lock, a .python-version file and a pre-commit configuration, plus one demo script per scenario at the root, including animal_demo.py, hand_demo.py, rtmo_demo.py, body_with_feet_demo.py, custom_rfdetr_demo.py, wholebody3d_demo.py and wholebody_demo.py. Those demo scripts are the fastest way to see the intended usage of each solution. The pyproject.toml also carries an unusually candid comment about a packaging bug that was fixed, recording that the version number used to live in two out of sync places and that requires-python was declared as 3.10 in pyproject.toml while setup.py said 3.7, so a build with python -m build picked up the version from pyproject.toml but silently fell back to the setup.py value for Requires-Python, and the published package never enforced Python 3.10 as intended. The version is now read dynamically from rtmlib/version.py, with setup.py removed.

## What the library does not do, and where it will stop you

Two boundaries matter for a decision. The first is that this is an inference library. The README documents how to download, configure and run models, and the repository has a tests/ directory, but nothing describes training, fine tuning or a dataset pipeline, so if your requirement is a model of your own on your own data, rtmlib is not the place to start. The second is that the model zoo is fixed to the URLs the project ships. The det and pose arguments take a URL, and every URL in the documentation points at download.openmmlab.com, so you can substitute your own model file, but you have to supply it and know its input size and its keypoint convention, and the library will not tell you which ONNX model is compatible with which solution class. The device and backend values are also a closed set in practice, with cpu, cuda and mps listed as devices and opencv, onnxruntime and openvino as backends, and a mismatched pairing will fail at model load rather than at import. On the maintenance side, the version numbering tells you where the project is. The releases on record are 0.0.16 on 2026-08-03, 0.0.15 on 2026-02-10 and 0.0.14 on 2025-11-05, and the last push was on 2026-08-03, so the code is being worked on. A 0.0.x line with a dynamic version and a project that has already had a packaging inconsistency is a normal but slightly unsettled state, and the licence is Apache License 2.0, which is permissive enough for commercial use and for redistribution of the library itself, though the model weights come from elsewhere and carry their own terms.

## Conclusion

rtmlib suits a team that wants high quality pose estimation in a service or a desktop app and does not want a research framework on its dependency list, since the whole install is numpy, OpenCV, onnxruntime and a model download. It is the wrong tool if you need to train a pose model, since nothing in the documentation describes a training path, and it is the wrong tool if you need a model outside the shipped zoo, because the model files come from fixed download.openmmlab.com URLs. Before you adopt it, read the mode and det plus pose arguments for the solution class you plan to use, since choosing by mode picks a preset pair of models for you, and check that the device and backend values you pass are ones the library accepts, since the README lists cpu, cuda and mps as devices and opencv, onnxruntime and openvino as backends.

## FAQ

### What dependencies does rtmlib need?

Only numpy, opencv-python, opencv-contrib-python, onnxruntime and tqdm, with Python 3.10 or newer. The package metadata lists exactly those, and the README says you can optionally use other backends like opencv, onnxruntime, openvino or tensorrt to accelerate inference.

### How do I install rtmlib?

From PyPI with pip install rtmlib -i https://pypi.org/simple, or from source with git clone https://github.com/Tau-J/rtmlib.git followed by pip install -r requirements.txt and pip install -e ., with onnxruntime-gpu and openvino shown as optional extras.

### What is the difference between the mode argument and the det and pose arguments in rtmlib?

For the high level solution APIs you can either pass a mode of performance, lightweight or balanced, with balanced as the default, which selects a preset pair of models, or pass det and pose arguments naming the detector and pose estimator model URLs and their input sizes yourself. There is also a Custom form that takes the class names as strings alongside the URLs.

## Sources

- [Issues](https://github.com/Tau-J/rtmlib/issues)
- [License: Apache-2.0](https://github.com/Tau-J/rtmlib/blob/main/LICENSE)
- [README](https://github.com/Tau-J/rtmlib/blob/main/README.md)
- [Releases](https://github.com/Tau-J/rtmlib/releases)
- [Tau-J/rtmlib on GitHub](https://github.com/Tau-J/rtmlib)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/tau-j-rtmlib