MOTIP reframes multiple object tracking as ID prediction

The one-sentence claim behind the paper

MOTIP stands for Multiple Object Tracking as ID Prediction, and the README's TL;DR states the whole idea in a single construction.

We propose a novel perspective to regard the multiple object tracking task as an in-context ID prediction problem. Given a set of trajectories carried with ID information, MOTIP directly decodes the ID labels for current detections, which is straightforward and effective.

The reframing is the contribution. Most trackers decide an identity as part of a matching step, comparing detections against existing tracks and deciding which, if any, they belong to. MOTIP inverts that: the identities are already known for the trajectories in context, and the model's job is to predict which of those identities belongs to each current detection. It becomes a labelling problem rather than an association problem, and the in-context wording matters because the set of candidate identities is supplied to the model at inference time rather than being fixed at training time.

That structure is why the project is built on Deformable DETR. A DETR-style transformer takes a fixed set of queries and produces a set of predictions, and here the queries correspond to carried trajectories, with the identity of each carried trajectory playing the role that a class label plays in ordinary detection.

The paper was accepted by CVPR 2025, with a revised version on arXiv at 2403.16848, and the authors are Ruopeng Gao, Ji Qi and Limin Wang at Nanjing University.

The project is Python, Apache-2.0 licensed, and the last push to the repository was on 2026-07-30.

A codebase that was rewritten rather than patched

The release history in the README is unusually clear about the fact that the current code is not the code the paper was first written against.

The first version of the paper was released at arXiv:2403.16848v1 on 2024.03.26, and the corresponding codebase was stored in the prev-engine branch, which is no longer maintained starting April 2025. The revised paper came out on 2025.03.25, and the new codebase was released on 2025.04.03, with the README's assessment that compared to the previous version, it is more concise and efficient.

That rewrite means the codebase in the main branch and the code behind the original arXiv version are different things. Anyone trying to follow an early tutorial or an older commit expecting it to match the current tree will be reading code that no longer exists in that form.

The repository layout confirms the structure of the rewrite. train.py sits at the root for training, alongside submit_and_evaluate.py for the evaluation path, runtime_option.py for runtime settings, and utils/, tools/, structures/, configs/, models/ and data/ directories. There is also a TrackEval/ directory at the top level, and a demo/ directory holding colormap.py and video_process.ipynb.

The presence of TrackEval/ as a vendored directory rather than a declared dependency is a detail worth noting for anyone reproducing a score: it means the evaluation code travels with the project, so the numbers you compute locally come from the same evaluation code the authors used.

Getting set up: five documents and no single README

The README's quick start is a list of links, and that is an accurate description of the project's documentation structure rather than a shortcut.

The five pointers are docs/INSTALL.md for instructions of installing required components, docs/DATASET.md for dataset download and preparation, docs/GET_STARTED.md for how to get started with MOTIP including pre-training, training and inference, docs/MODEL_ZOO.md for well-trained models, and docs/MISCELLANEOUS.md for other miscellaneous settings unrelated to the model structure, such as logging.

A sixth document, docs/TUTORIAL.md, was added on 2025.07.10, and the README is honest that it is updating gradually. Its stated purpose is to help understand and work with the model, and it is the document to read if you want to understand the architecture rather than just run it.

Because there is no install command in the README itself, the installation step lives entirely in docs/INSTALL.md, and the dependency versions for the tracked environment are not visible in what the README shows. The one thing the README does state about environment configuration is the runtime_option.py file at the root, which is where runtime settings belong.

There is one concrete inference capability the README does mention directly, and it is the easiest entry point for a new user. From 2025.04.06, you can use the video demo to perform nearly real-time tracking on your videos, and the file is demo/video_process.ipynb. The word nearly is the README's own.

Since the setup is split across documents, the practical order matters. Installation, then dataset preparation, then the getting started guide, then the model zoo for the checkpoint you actually want, with the tutorial alongside for understanding the design.

Inference speed, half precision, and old checkpoints

Two of the README's news entries are about what you can do with the released weights rather than about the architecture.

From 2025.04.05, the project supports FP16 for faster inference. For a tracking model that has to process a video frame sequence, halving the precision of the network is one of the few levers that actually moves throughput, and it is the kind of detail that is often missing from research releases where the demo runs at reduced resolution to compensate.

From 2025.04.11, there is support for loading previous MOTIP checkpoints from the prev-engine branch to inference, with details in docs/MODEL_ZOO.md. This closes the loop left open by the rewrite: a checkpoint trained under the old engine can be loaded by the new code, which means the models the paper reported on are still usable even though the branch itself is no longer maintained.

The two released tags line up with that period. v0.1 is labelled Default experiment resources and is dated 2025-04-02, the day before the new codebase was announced, and v0.2 is labelled MOTIP without Trajectory Augmentation and is dated 2025-04-11, the same day as the checkpoint compatibility news.

The v0.2 label is the one to read carefully. It says without Trajectory Augmentation, which means the published v0.2 checkpoint does not use that component, and a user comparing against a paper result needs to know whether the augmentation was in the loop.

The project also ships a citation entry that names the venue and pages, and it is worth reproducing exactly if you build on this work:

@InProceedings{{MOTIP},
    author    = {Gao, Ruopeng and Qi, Ji and Wang, Limin},
    title     = {Multiple Object Tracking as ID Prediction},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {27883-27893}
}

What the project is built on, and what that means

The acknowledgements section names three codebases, and they tell you what the architecture actually is.

This project is built upon Deformable DETR, MOTR, and TrackEval.

Deformable DETR contributes the transformer backbone with deformable attention, which is what allows a detection model to attend over a high resolution feature map with a manageable number of queries. MOTR contributes the tracking-specific design that MOTIP reframes, since MOTR is the end-to-end multiple object tracking work that carries identity through the transformer queries as well. TrackEval contributes the evaluation.

So MOTIP is not a detector plus a separate association step bolted together. It is a DETR-style detector whose queries carry identity, and the contribution is a change to how the identity is produced rather than to the detection machinery. That is why the previous-engine branch could be rewritten and why the model is described as more concise and efficient in the new version: a different formulation of the same task allows a smaller, cleaner implementation.

The venue for that lineage is worth understanding. MOTR was the end-to-end tracking work that introduced carrying track queries across frames, and MOTIP's identity prediction framing is a reaction to that line of research rather than an unrelated system.

The evaluation choice is also traceable. TrackEval is the standard evaluation toolkit for MOT benchmarks, and it is vendored here rather than pulled from upstream, so local evaluation results and the numbers reported in the paper are computed by the same code.

Where MOTIP fits, and where it stops

The comparison worth making is with the tracking frameworks a practitioner would otherwise reach for, and with the earlier MOTIP code.

Against a two-stage pipeline, where a detector produces boxes and a separate tracker associates them, the difference is that MOTIP makes identity part of the model's output. There is no matching stage to tune, no association threshold, and no separate tracker whose behaviour you have to reason about separately. The cost is that the model has to be run end to end, and the identity prediction is only as good as the set of trajectories you hand it, since the in-context set is what the model chooses from.

Against MOTR, the closest relative, the difference is exactly the framing. MOTR carries identity through the transformer queries and refines them over frames, and MOTIP's argument is that predicting which carried ID belongs to a current detection is a cleaner formulation of the same structure.

The limits are clear from the repository rather than from the paper. There is no compatibility table, so you cannot tell from the README which datasets the released checkpoints were trained on or what to expect; that is in docs/MODEL_ZOO.md and docs/DATASET.md. There is no performance table in the README, so the effectiveness claim rests on the paper and on your own reproduction. And the deployment story is thin: a notebook demo for nearly real-time tracking on videos, FP16 support, and no inference server, no exported model format and no documented throughput.

The maintenance picture deserves a plain statement rather than a reassurance. The last push was on 2026-07-30, the repository is not archived, and the two tagged releases are from April 2025. The previous-engine branch is explicitly no longer maintained starting April 2025. So this is a research codebase that was reorganised once, in a way that broke compatibility with its own earlier form and then repaired it for checkpoint loading, and it has been quiet for a few months since.

For a group planning to extend this work, that is fine: the code is small enough to read, the formulation is clean, and the model zoo gives you a starting checkpoint. For a group planning to ship a tracker, the missing pieces are the deployment artefacts rather than the research.

Editorial conclusion

MOTIP suits a researcher who wants to work on multiple object tracking with a detection-first architecture rather than a tracking-first one, since the README's claim is that decoding ID labels from carried trajectories is straightforward and effective, and the released code, configs and model zoo are enough to reproduce a training run. It is the wrong tool if you need a production tracker with a documented deployment path, because the repository is a research release whose documentation is split across seven files in docs/ and whose last push was on 2026-07-30. Before starting, read docs/INSTALL.md and docs/GET_STARTED.md, then check docs/MODEL_ZOO.md for the checkpoint that matches your dataset, since the previous-engine branch is no longer maintained starting April 2025.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. MCG-NJU/MOTIP on GitHub
3. Project website
4. README
5. Releases