SOTS: a benchmark collection of visual object trackers and their code

A collection of paper codebases, not a framework

The repository description is one line: single object tracking and segmentation, and the README's own subtitle is codes and comparison of recent single/multiple object tracking and segmentation.

That second half is the important qualifier. What is actually in the repository is a set of independent research codebases for published papers, plus a results table that puts them side by side. There is no shared API, no common entry point and no model registry. Each tracker has its own training script, its own configuration format and its own assumptions about data layout, because each one was written to reproduce one paper.

The supported list is seven papers across two task families. For single-object tracking there is VLT_SCAR and VLT_TT from NeurIPS 2022, CNNInMo and TransInMo from IJCAI 2022, AutoMatch from ICCV 2021, Ocean and Ocean+ from ECCV 2020, and SiamDW from CVPR 2019, which was selected as an oral presentation. For multi-object tracking there is OMC from AAAI 2022 and CSTrack from IEEE TIP.

The News section is a publication list rather than a changelog, and it carries one practical detail: for OMC and for AutoMatch the README states that the training and testing code has been released in this codebase, so not every listed paper has code here.

The repository is Python, the licence is not recorded, the default branch is SOT, and the last push was on 2026-04-14. There are no GitHub releases.

The branches, and why the default branch is not master

The Branches section is short and it changes what you get from a clone.

The SOT branch, which the README also calls master, is for the SOT trackers. The MOT branch is for the MOT trackers. And a v0 branch holds the old codebase supporting OceanPlus and TensorRT testing.

The instruction is to clone the branch to your needs, which is the practical form of the advice. The default branch named in the repository metadata is SOT, so a plain clone gives you the single-object trackers and none of the multi-object ones, and a user who wants CSTrack or OMC has to switch branches. Given that the repository name and the fork ancestry make it likely that a clone of master was the older default, the renaming to SOT reflects the split the maintainers actually wanted.

The v0 branch is worth knowing about for a specific reason rather than as a curiosity. It is the only place where the TensorRT testing path survives, so a user who needs to run a tracker with TensorRT rather than PyTorch has to go there, and they are then working with a codebase that is explicitly old.

The top level of the repository confirms the branch split in practice. The current branch has README.md, demo/, experiments/, lib/ and tracking/, and there is no data/, snapshot/, pretrain/ or dataset/ directory, because those hold datasets and model weights that are not committed. The images in demo/ are the figures used in the README, including per-tracker overviews such as CSTrack_CCN.jpg, Ocean_overview.jpg, Siamdw_overview.jpg, TransInMo.jpg and VLT.jpg.

The directory layout, and what is not in the repository

The README documents the structure with a tree, and the tree is more informative than it looks because it names the directories that do not exist in a fresh clone.

- `experiments:` training and testing settings - `demo:` figures for readme - `dataset:` testing dataset - `data:` training dataset - `lib:` core scripts for all trackers - `snapshot:` pre-trained models - `pretrain:` models trained on ImageNet (for training) - `tracking:` training and testing interface

The tree in the README shows the intended shape.

$SOTS
|—— experimnets
|—— lib
|—— snapshot
  |—— xxx.model
|—— dataset
  |—— VOT2019.json 
  |—— VOT2019
     |—— ants1...
  |—— VOT2020
     |—— ants1...
|—— ...

The split between data and dataset, and between snapshot and pretrain, encodes two different kinds of dependency. The training data and the ImageNet-pretrained backbones are inputs you download, and the pre-trained model snapshots are the checkpoints for evaluation. The `lib/` directory holds the core scripts for all trackers, and `tracking/` is the training and testing interface, which is where the per-tracker implementations and the shared entry points live.

None of the four data-bearing directories is in the repository listing, so a fresh clone gives you code and configuration but no weights and no data. The `dataset/` directory is described as the testing dataset and, in the tree, is shown with VOT2019.json and per-sequence directories, and the tree also shows a VOT2020 directory. So the testing data is expected in a specific layout, laid out as a JSON file plus one directory per sequence, and the sequence directories are named after the videos themselves.

The practical implication is that a user has to assemble the evaluation data in the expected shape before the test scripts will find it, and that the pre-trained snapshots have to be obtained separately for the numbers in the results table to be reproducible.

The benchmark table, and what the numbers mean

The performance section is a single wide table with one row per model and one column per dataset, and reading it requires knowing what the columns are.

The datasets are OTB2015, GOT10K, LaSOT, TNL2K, TrackingNet, NFS30, TOTB, VOT2019, TC128, UAV123, LaSOT_Ext and OTB-99-LANG. Some of these are single-object tracking benchmarks and some are for multiple objects, and the single-object trackers show dashes in columns they were not evaluated on, which is how you can tell which family a row belongs to. The MOT trackers, CSTrack and OMC, only have entries in the LaSOT and TrackingNet columns in this view, which is the reverse pattern.

The metric is success rate for the SOT rows, where higher is better, and the values in the table are in that range, from the low 0.2s to the high 0.7s. The table is a snapshot of published results rather than a re-run, and the README's phrasing, comparison, points to a separate repository for the side-by-side numbers.

Two observations are worth making. First, the most recent model, VLT_TT, has no entries in OTB2015, GOT10K, TrackingNet, NFS30, TOTB, VOT2019, TC128 or UAV123, and its presence is concentrated in LaSOT, LaSOT_Ext and OTB-99-LANG, which are the multilingual and language-inclusive benchmarks. Second, the older SiamDW has entries across the SOT benchmarks but dashes in the last two columns, which is a benchmark added later rather than a result that was never computed.

The News section also records a competition result for CSTrack, that it ranks 5 out of 4000 at the Tianchi Global AI Competition, and a commented-out passage in the README describes a variant, CSTrack_panda, with a stated score of A-0.6712 and B-0.6251 against a baseline of A-0.32 and B-0.34, and notes that it takes fifth place in the competition. That passage is commented out in the current README, so the numbers are not part of the live documentation.

What each tracker's own documentation says

The Tracker Details section gives one paragraph per method, and the VLT entry is the most explicit.

VLT explores a different path to achieve state-of-the-art tracking without a complex Transformer, which the README describes as multimodal vision-language tracking. The essence of it is a unified-adaptive vision-language representation, learned by the proposed ModaMixer and asymmetrical networks. The experiments show the approach boosts a pure CNN-based Siamese tracker to competitive or better performance compared with recent state of the art, and the README says this also benefits Transformer-based trackers. The stated hope is that the work inspires more possibilities for future tracking beyond Transformer.

Each tracker entry links to a paper, a raw results folder on Google Drive, and a training and testing tutorial inside the repository, at paths under lib/tutorial/. For VLT the tutorial is lib/tutorial/sot/sot_vlt.md, and the repository listing shows the lib/ directory present, so the per-tracker documentation ships with the code rather than living on a wiki.

That is a meaningful point about how this repository is maintained. A tracker is only usable if you can retrain it, and the retraining path depends on the training data, the ImageNet-pretrained backbone and the exact configuration under experiments/. The tutorials exist because reproducing a published number from the paper alone is usually impossible, and the per-experiment settings are what make the published numbers reachable.

Where SOTS is the wrong tool

Three cases point elsewhere. The first is a developer who wants a tracking library. Everything here is a set of paper codebases with their own training scripts, configuration formats and data expectations, and there is no pip-installable package, no unified interface and no version compatibility between them. A user who wants to track objects in a video today should use a maintained tracking library rather than reconstructing a 2021 checkpoint's training setup.

The second is a user who needs segmentation. The repository description and the README subtitle both mention segmentation, and the topics on the repository include no segmentation tag, but the visible tracker list is entirely tracking methods with the OTB, GOT10K, LaSOT and similar benchmarks, which are tracking benchmarks. The README text available here does not list a segmentation method among the supported trackers, so anyone expecting the segmentation half of the title to be present in this branch should check the lib/ directory before committing to it.

The third is a user who needs results they can trust as a comparison rather than as published numbers. The table records what each paper reported, and the columns are filled on different schedules, so a dash means not evaluated rather than zero, and comparing two models on a column where only one has a value tells you nothing. A recent paper that benchmarks against these methods will report its own numbers under a common protocol, which is a different and more useful kind of comparison.

The alternative worth weighing is a maintained tracking library such as a PyTorch tracking toolkit, where the models share one interface, the data loaders are unified and the evaluation is run for every method under the same conditions. The difference in approach is that SOTS gives you the authors' own code, which is the only way to get exactly the published behaviour, including the details a paper leaves out, while a maintained library gives you comparability and a stable API. A researcher reproducing a specific paper needs the first. A team building on tracking needs the second, and would use SOTS as a source of checkpoints and reference implementations rather than as a dependency.

Editorial conclusion

SOTS fits a researcher who wants to reproduce a specific published tracker, or who wants the authors' own numbers for several trackers on the same datasets. It does not fit a developer who wants a maintained tracking library or a pip-installable API, since each tracker is a research codebase with its own conventions, and it does not fit a user who needs segmentation output, since the description mentions segmentation but the visible tracker list contains none. Before you clone it, pick the branch your tracker lives on, SOT or MOT, because the default branch is SOT and the MOT trackers are not there, and expect to supply your own datasets under the data/ and dataset/ directories.

Frequently asked questions

Official sources

1. Issues
2. JudasDie/SOTS on GitHub
3. README