DeepTracker puts four tracking algorithms behind one C++ interface

One interface, four algorithms, one line to switch

The central design decision is stated early: all trackers implement the same thin ITracker interface, with update, reset and frame_id, so switching algorithms is a one line change.

That is a small API and it is the right one for this problem, because the interesting comparison between trackers is not about their interface, it is about their behaviour on the same footage. A YOLO detector feeds one of four classic tracking algorithms, and the detector is the same in every case, so a difference in the output is a difference in the tracker.

The four choices are laid out in a table with their association model and whether they need a ReID model.

ByteTrack associates on motion, using IoU and confidence, and has no ReID. It is a two stage association: high score detections are matched first, then the low score leftovers recover tracks that were briefly lost to occlusion. That second stage is the whole idea, since a detector that is briefly unsure about an object usually still has a poor box for it rather than no box, and a plain IoU matcher would treat that as a new detection.

OC-SORT also uses motion only, described as observation-centric, and is built from three pieces: OCM, a velocity-direction cost, OCR, re-association with the last observation, and ORU, virtual observation back-fill on re-acquisition. The claim is that it handles occlusion and motion blur without any appearance model at all.

So the four cover a spectrum: two that are motion only, one that adds an optional appearance model, and one that requires it.

The track lifecycle is fixed, and it is the same for all four

Every tracker follows the same track lifecycle, and the parameters are given as constants rather than as options.

A new track starts tentative. It is confirmed after n_init = 3 consecutive associations, so a track that never accumulates three hits is never confirmed and eventually disappears. It is kept alive while lost up to track_buffer = 30 frames, which is the grace period during which a track that has gone unmatched can still be recovered rather than being abandoned and started again.

And association never crosses object classes. A person track is never matched to a car detection, even if the boxes overlap, which removes a category of error that a naive implementation gets wrong in crowded scenes.

These two constants, n_init and track_buffer, are the main knobs a user of a tracker normally wants to change, and here they are fixed. The tracker parameters live as compiled-in presets in src/core/tracker_config.cpp, through TrackerConfig::defaults_for, and the set covers confidence gates, track buffer, association thresholds, the GMC method and the ReID settings. To change a value you tune a preset and rebuild.

That is a genuine limitation for interactive work. In a GUI where you are comparing trackers, being unable to change the track buffer without a recompile is a real cost, and the alternative would be a configuration file or a runtime setter. The trade buys a small, predictable code path with no configuration parsing, and it suits a tool whose presets are meant to be reasonable defaults.

DeepSORT is the one that keeps its own loop

The architecture section describes two different implementations sharing one interface, and the difference matters for anyone extending the code.

BaseTracker is a template-method pipeline, and the stages are listed in order: detection split, predict, first association, recovery, second association, unconfirmed association, new-track init, pool merge. ByteTrack, OC-SORT and Deep OC-SORT are described as policy combinations on this pipeline through virtual hooks, which means they differ by overriding a step rather than by rewriting the loop.

DeepSortTracker is different. It is a self-contained algorithm with its own cascade matching loop, and it implements ITracker directly rather than inheriting the shared pipeline. The reason is visible in the algorithm description: a faithful implementation of Wojke et al. has an age-ordered cascade matching structure with a feature gallery and a second IoU stage, and that does not fit the shared pipeline's shape without forcing it.

The result is a codebase where three trackers are variations on a theme and one is a separate implementation behind the same three-method interface. For a user that is invisible. For a maintainer it is a decision with a cost, since fixes to the shared pipeline do not automatically apply to DeepSORT.

The association utilities are shared, though. They cover IoU and embedding distance matrices, score fusion, and a Jonker-Volgenant linear assignment solver, and the LAPJV solver is what makes the Hungarian-style matching tractable for a frame with many tracks and many detections.

The appearance gallery is what separates the two ReID trackers

DeepSORT and Deep OC-SORT are the two that use appearance, and they treat it differently, which is the most important distinction in the table.

DeepSORT is described as a faithful implementation of Wojke et al., ICIP 2017, with age-ordered cascade matching, a feature gallery of the last 100 frames matched by minimum cosine distance, squared-Mahalanobis motion gating with a chi squared threshold of 4 degrees of freedom and 9.4877, and a second IoU association stage for tentative and just-lost tracks. ReID is required for it, and it falls back to ByteTrack when no ReID model is available.

The gallery detail is the interesting part. Keeping the last hundred frames of embeddings per track, and matching by minimum cosine distance against them, is what lets a track survive a long occlusion, because the appearance of the object is remembered even when the detector is not seeing it. The Mahalanobis gate sits in front of that, discarding candidates whose predicted position is too far from the last observation under the motion model.

Deep OC-SORT is the other way round. Its ReID is optional, and it engages automatically when the model file is present, otherwise it runs as plain OC-SORT. It adds ReID feature fusion and Global Motion Compensation for moving-camera footage, and GMC is a choice in the interface, with none, sparseOptFlow, orb, sift and ecc offered as methods.

So the two trackers with appearance support differ in how central it is. For DeepSORT it is the algorithm. For Deep OC-SORT it is an enhancement on top of a motion-centric tracker, and the motion-centric part is what makes it usable on footage where a reliable appearance model is not available.

Models are ONNX files and the runtime picks the provider

Detection and appearance features run through ONNX Runtime, so models are plain .onnx files that can be upgraded without recompiling. The repository ships two of them, models/yolo26s.onnx for detection at 640 by 640 input with the person class by default, and models/yolo26s-reid.onnx as the appearance encoder producing L2-normalized embeddings.

Inference defaults to the CPU execution provider, and CUDA, DirectML, OpenVINO and other providers can be selected without touching the tracking code. That separation is the useful design decision: the choice of accelerator is an environment concern, and the tracking algorithms do not have to know anything about it.

The models are found automatically next to the executable, in a models/ directory, with the search going upward through parent directories. The upward search is what makes the deployed layout work, since the executable is deployed to a build/ folder together with the DLLs and models, and that folder is portable and can be copied anywhere. The scripts/ directory holds export_yolo26_onnx.py, which converts a PyTorch model to ONNX, so the model pipeline is reproducible rather than a binary someone dropped in.

The external/ directory holds prebuilt OpenCV and ONNX Runtime and is explicitly not committed, which is the usual arrangement for large prebuilt dependencies.

Windows and Qt Creator are the build path

The build instructions are numbered and Windows specific.

First, install Visual Studio 2022 with MSVC v143 for x64, and Qt 6.9.x with the MSVC 2022 64-bit kit. Second, place OpenCV and ONNX Runtime under external/, at external/opencv/build and external/onnxruntime. Third, open src/app/gui/deeptracker_gui.pro in Qt Creator, then click Build and Run.

The executable is deployed to a build/ directory together with the required DLLs and the models, and that folder is portable and can be copied anywhere. All the paths in the .pro file are relative, so the project builds from any location, which is a small courtesy that saves a lot of trouble with hard coded SDK paths.

The generated makefiles in the repository root show the environment the project file was produced with, qmake 3.1 from Qt 6.9.3 with the win32-msvc spec, and they carry hard coded paths such as C:\Qt\6.9.3\msvc2022_64\bin\qmake.exe, which is the usual thing to expect from a checked in qmake makefile and the reason the .pro file is the file to build from.

Using the application is a four step sequence. Run build/track_gui_qt.exe, pick an input video such as data/people_walking.mp4 with the output path filled in automatically, choose a tracker along with the confidence threshold and the object class, and press Start tracking. For Deep OC-SORT you also select a GMC method. The progress bar reports frame progress and the number of active tracks, the log panel records model loading and tracker selection, and a palette button in the title bar switches the colour theme, with the choice remembered.

The same functionality is available headlessly through the command line demo:

track_demo --tracker deepsort --source video.mp4 --out result.mp4

with the supported --tracker values being bytetrack, ocsort, deepocsort and deepsort.

Editorial conclusion

DeepTracker suits a team evaluating trackers on the same footage, in the same application, with the same detector, rather than comparing papers, and it suits a Windows developer who wants a desktop tool for inspecting tracking output frame by frame. It does not suit a deployment that needs GPU inference on the target machine without configuring an execution provider, and it is the wrong tool if you need to change tracker parameters at run time, because the presets are compiled in. Verify first that the model files sit where the discovery logic expects, that the ReID model is present if you plan to use DeepSORT or the appearance side of Deep OC-SORT, and that your build environment has the specific Qt and MSVC versions the project file was generated with.

Frequently asked questions

Official sources

1. Issues
2. README
3. Releases
4. shaoshengsong/DeepSORT on GitHub