Multitarget-tracker: Hungarian Assignment and Kalman Filtering in a C++ MOT Pipeline

The gap Multitarget-tracker fills between detection and trajectory

A detector tells you where objects are in one frame. It does not tell you which box in frame 101 is the same car as the box in frame 100, and it does not survive a frame where the detector drops the object entirely. Multitarget-tracker is a C++ library and command line tool that handles that second problem: it takes per-frame detections, assigns them to existing tracks, and smooths the resulting trajectories. The README describes it as a "Multiple Object Tracker, Based on Hungarian algorithm + Kalman filter", and the repository topics list the intended applications directly: car-counting, car-tracking, face-tracking, people-tracking, abandoned-detector. The audience is an engineer building a video analytics pipeline in C++ who already has a detector and needs the association layer. It is not a detector training framework and it is not a Python-first research codebase, even though setup.py exists and declares a package named pymtracking.

Hungarian assignment, LAPJV, and the distance metrics that feed them

The core loop is a linear assignment problem solved once per frame. The README lists two solvers: the Hungarian algorithm exposed as tracking::MatchHungrian with O(N³) complexity, and LAPJV exposed as tracking::MatchBipart with O(M*N²) complexity. Which one you want depends on how many simultaneous targets you expect; the README presents LAPJV as the alternative, noting it is the algorithm used by scipy's linear_sum_assignment. Both consume a cost matrix, and the cost matrix is built from one of three distance metrics: center distance (tracking::DistCenters), bounding box distance (tracking::DistRects), or Jaccard/IoU similarity (tracking::DistJaccard). That choice matters more than the solver choice in most scenes. IoU degrades when objects move fast relative to frame rate, because consecutive boxes barely overlap, while center distance degrades when objects are large and close together. Smoothing sits on top: Kalman filters are available in linear (tracking::KalmanLinear) and unscented (tracking::KalmanUnscented) forms, with constant velocity or constant acceleration state models, and you choose between tracking position only (tracking::FilterCenter) or position plus size (tracking::FilterRect). The README also documents a visual search fallback for when targets disappear, with KCF, CSRT, DaSiamRPN, Vit and Nano trackers listed under tracking::TrackKCF and its siblings.

Building Multitarget-tracker from source with CMake

The README gives the build sequence directly. Clone the repository, create a build directory, run cmake with the feature flags you need, then make. The flags shown are USE_OCV_BGFG, USE_OCV_KCF, USE_OCV_UKF, BUILD_ONNX_TENSORRT, BUILD_ASYNC_DETECTOR and BUILD_CARS_COUNTING. Note that the cmake invocation in the README is written as "cmake . ..", which is unusual; the conventional form would be cmake .. from inside the build directory. Treat the README's line as the documented command and be aware it may need adjusting on your system.

git clone https://github.com/Smorodov/Multitarget-tracker.git
cd Multitarget-tracker
mkdir build && cd build
cmake . .. \
  -DUSE_OCV_BGFG=ON \
  -DUSE_OCV_KCF=ON \
  -DUSE_OCV_UKF=ON \
  -DBUILD_ONNX_TENSORRT=ON \
  -DBUILD_ASYNC_DETECTOR=ON \
  -DBUILD_CARS_COUNTING=ON
make -j

After the build completes you should have a MultitargetTracker executable. The README's usage guide gives the argument list, including --example, --start_frame, --end_frame, --out, --gpu, --async, --res, --settings and --batch_size. A first run against the bundled sample video looks like this:

./MultitargetTracker ../data/atrium.avi -e=1 -o=../data/atrium_motion.avi

The README states that this example uses the motion detector path and writes an annotated video to ../data/atrium_motion.avi. While the window is open, m toggles play and pause, any key steps forward when paused, and Esc exits. If you want to embed the tracker rather than run the binary, the README shows the library entry point: include mtracking/Ctracker.h, construct a TrackerSettings, call settings.SetDistance(tracking::DistJaccard), and pass the settings to BaseTracker::CreateTracker. The README does not document the full set of TrackerSettings fields, so expect to read the headers for anything beyond the distance metric.

Where the detection side gets thin

The README's detector list is the weakest part of the documentation. It names background subtraction methods (VIBE, SuBSENSE, LOBSTER, MOG2, MOG, GMG, CNT) and two deep learning paths (tracking::DNN_OCV through OpenCV's DNN module, and tracking::Yolo_TensorRT). The "Latest Features" section then lists YOLOE, YOLOv26, YOLOv26-obb, YOLOv26-seg, RT-DETRv4, D-FINE, D-FINE seg, YOLOv13, RF-DETR, YOLOv12 and ByteTrack support. Several of those entries say the model works with TensorRT after exporting a PyTorch checkpoint to ONNX and running the -e=3 example. What the README does not give is a worked export command for any of them, nor a mapping from model family to the correct example number. If your detector is not one of the listed families, the integration path is undocumented. The same section mentions a "Big code cleanup from old style algorithms and detectors" that removed background subtraction detectors, some VOT trackers, face and pedestrian detectors, and the Darknet backend for old YOLO. Anyone following a tutorial written before that cleanup will find APIs that no longer exist. There is no migration note in the README.

Threading model and the cost of low-FPS detectors

The README describes three processing pipelines: synchronous (SyncProcess, single-threaded), asynchronous with two threads (AsyncProcess, which decouples detection from tracking), and a fully asynchronous four-thread mode intended for low-FPS deep learning detectors. That third mode is the honest admission in the documentation. When a detector runs at 5 FPS and your video is 30 FPS, a synchronous pipeline wastes most of its time waiting. The threaded mode exists to keep tracking running on frames the detector has not seen, which means the tracker is extrapolating from the Kalman filter between detections. That is a real trade-off rather than a free win: during the gap, track positions come from the motion model, so a target that changes direction sharply will drift until the next detection corrects it. The README does not state how many frames the tracker will coast before dropping a track, and it does not document the association threshold that decides when a detection becomes a new track versus matching an existing one. Those are the two parameters most likely to determine whether your output is usable, and they are not in the README.

Multitarget-tracker compared with a Python tracking library

The obvious alternative for a Python user is a tracking library that ships as a pip package with pretrained weights and a documented Python API. The difference in approach is where the work sits. Multitarget-tracker is a C++ core with a CMake build and a long list of compile-time flags; the Python side is a setuptools wrapper around that core, declared in setup.py as pymtracking, and the README does not document how to install or use it. A Python-native library inverts that: you install it in one command, you pass detections or frames in Python, and the performance ceiling is lower but the iteration speed is higher. If your pipeline is already C++ and you need to control the assignment metric and the Kalman state model directly, Multitarget-tracker gives you those knobs at the type level. If you are prototyping and want to compare three trackers in an afternoon, the build step alone will cost you more than the experiment is worth.

Licence, maintenance and what upgrading costs

The repository is licensed Apache-2.0, which permits commercial use and modification provided you keep the licence and notice files and state significant changes. That is more permissive than a GPL-style copyleft tracker, and it matters if you are shipping a closed product. The README does not discuss the licences of the third-party dependencies it lists (OpenCV and contrib, Vibe, Non Maximum Suppression, Ini file parser, Circular Code), so check those separately before distribution. On maintenance, the last push to the repository was on 2026-09-03, and the only release listed is v1 (Classic version) from 2025-10-31. There is no changelog in the README, and the "Latest Features" list is the closest thing to release notes. Practically, that means upgrading means reading the README diff and the commit history, not a versioned migration guide. Budget for build breakage when you move to a newer OpenCV, since the README pins nothing and lists OpenCV as a dependency without a version.

Editorial conclusion

Adopt Multitarget-tracker if you need a C++ tracking core that accepts detections from OpenCV DNN or TensorRT YOLO models and you are willing to build it from source. Skip it if you need a pip-installable, documented Python tracking library, because the README gives no installation steps for the pymtracking package beyond the presence of setup.py. Before committing, verify which CMake options your detector path needs and whether the Python bindings expose the settings you intend to tune.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. Releases
5. Smorodov/Multitarget-tracker on GitHub