DAM4SAM: a distractor-aware memory for SAM2 tracking

The failure mode DAM4SAM targets: SAM2 memory picks the wrong object back up

Memory-based trackers keep a running store of past frames and re-read it to decide where the target is now. SAM2 is the reference implementation of that idea, and the DAM4SAM abstract states plainly that such trackers "still struggle with distractors". A distractor here is a second object that looks like the target: a teammate in the same jersey, a second car of the same model, a hand that enters the frame next to the tracked hand. Once a distractor is written into memory, later frames can retrieve it instead of the target, and the tracker locks onto the wrong instance. The damage is not gradual drift. It is a discrete switch.

The intended user is a researcher or engineer who already runs SAM2.1 and wants to test whether a different memory policy reduces those switches. The project is a paper artifact first: the abstract claims state-of-the-art results on six benchmarks "without additional training", and the README points to the CVPR 2025 preprint and the IJCV 2026 extension. There is no pip-installable package and no hosted endpoint. That shapes everything below.

How the distractor-aware memory plugs into SAM2.1

The README describes DAM as "a new plug-in distractor-aware memory (DAM) and management strategy", demonstrated on SAM2.1. The repository layout matches that description. The sam2/ directory holds the model code and the model configs, dam4sam_tracker.py wraps the tracking loop, and dam4sam_config.yaml carries the tracker settings. Inference goes through the SAM2 image predictor to turn an initial bounding box into a mask, which is then used to initialize the tracker.

That mask initialization step is worth understanding before you run anything, because it is where most first-run confusion comes from. On bounding box datasets such as LaSot, LaSoText and GoT-10k, the README states that the tracker "is initialized with masks, which are obtained using SAM2 image predictor, from ground truth initialization bounding boxes". So the pipeline is: ground truth box, SAM2 image predictor, mask, tracker. If the predictor produces a poor mask on the first frame, you are evaluating the memory strategy on top of a bad initialization, and the paper's numbers will not reproduce.

The memory itself is not a separate model you train. The abstract frames it as a management strategy layered on an existing tracker, which is why the README can claim benchmark results "without additional training". You are not fine-tuning SAM2. You are changing which frames it remembers and how it uses them.

Installing DAM4SAM and running the bounding box demo

The README gives a conda-based install. Clone the repository, create an environment pinned to Python 3.10.15, then install torch and the pinned requirements.

git clone https://github.com/jovanavidenovic/DAM4SAM.git
cd DAM4SAM
conda create -n dam4sam_env python=3.10.15
conda activate dam4sam_env
pip install torch==2.1.0 torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt

The torch index URL points at a CUDA 12.1 wheel build, so this assumes an NVIDIA GPU. There is no documented CPU path. If the SAM2 CUDA extension fails to build, the README names the symptom explicitly: `ImportError: cannot import name '_C' from 'sam2'`. The documented fix is to build the extension in place from the repository root.

python setup.py build_ext --inplace

The README adds a caveat worth reading twice: the repository still works with the warning present, but "some postprocessing SAM2 steps may be skipped". That is a silent behavioural difference, not a cosmetic warning. Fix the build before you trust a number.

Checkpoints come from a shell script rather than a download link. The README gives the command without listing which files it fetches.

cd checkpoints && \
./download_ckpts.sh

For a first real run, the quickest path is the bounding box demo. It takes a directory of frames, asks you to draw an initial box, estimates a mask on that frame, and writes predicted masks for every frame.

CUDA_VISIBLE_DEVICES=0 python run_bbox_example.py --dir <frames-dir> --ext <frame-ext> --output_dir <output-dir>

`--ext` defaults to jpg. `--output_dir` is optional; omit it and the script only visualizes instead of saving. Expect a window asking for a box, then a per-frame mask sequence in the output directory.

Evaluating on DiDi and the VOT workspaces

The repository supports evaluation on DiDi, VOT2020, VOT2022, LaSot, LaSoText and GoT-10k. VOTS2024 support is described in the README as coming soon, so treat it as absent. The DiDi dataset is the project's own distractor-distilled benchmark, and the workflow has two stages: produce raw results, then convert and score them with the VOT toolkit.

CUDA_VISIBLE_DEVICES=0 python run_on_didi.py --dataset_path <path-to-didi> --sequence <sequence-name>

Running without `--sequence` and with `--output_dir` processes the whole dataset and writes results to disk. DiDi is a bounding box dataset, so the raw mask output has to be converted before scoring. That is what move_didi_results.py does, and the README notes it converts as well as moves.

python move_didi_results.py --dataset_path <path-to-didi> --src <source-results-directory> --dst ./didi-workspace/results/DAM4SAM

Scoring runs through the VOT toolkit against the provided didi-workspace. The README warns that if you run the analysis more than once you must clear the cache directory, otherwise stale results are reused.

vot analysis --workspace <path-to-didi-workspace> --format=json DAM4SAM
vot report --workspace <path-to-didi-workspace> --format=html DAM4SAM

Reports land in didi-workspace/reports. For VOT2020 and VOT2022 the README points to vot_integration/vot2022_st, which contains two stack files: one for the baseline experiment and one for the real-time experiment. That distinction matters if you are comparing against published VOT numbers, because the two stacks are not the same experiment.

Where DAM4SAM is the wrong tool

The repository has no LICENSE file at its top level. The README does not state a licence for the project's own code. This is not a formality. The sam2/ directory is derived from Meta's SAM 2, and setup.py carries Meta's header with LICENSE = "Apache 2.0" and the URL https://github.com/facebookresearch/sam2. So you are inheriting an Apache 2.0 component inside a repository whose own terms are unstated. If you need a clear licence before shipping, that gap is the first thing to resolve, and the README will not resolve it for you.

The second limitation is operational. There is no released package, no versioned release, and no documented API surface. The tracker is driven through scripts. The last push to the default branch was on 2026-04-07, which is roughly five months before this writing; the repository is not archived, but the README does not document a release cadence, a changelog, or a rollback path. Upgrading means pulling master and re-reading the diff.

Third, the CUDA extension is load-bearing in a way the README understates. The documented fallback is that the tracker still runs with the import error present, minus some postprocessing steps. In a benchmark context, that is a different configuration producing different numbers. There is no documented way to detect at runtime which mode you are in.

Finally, this is not a general-purpose tracking library. If you want a maintained tracker with a stable interface and CPU support, the paper artifact is the wrong shape of software.

Alternatives: stock SAM2, and the multi-target fork

The most direct alternative is SAM2.1 without DAM. The difference is exactly the memory policy: stock SAM2.1 uses its own memory and management, and DAM4SAM replaces that with a distractor-aware one while leaving the underlying model untouched. If your sequences have no distractors, the plug-in has nothing to fix and you carry the extra dependency for nothing. If they do, running both on the same frames is the only way to see whether the policy change is what moved your numbers.

The README also points to a multi-target version of DAM4SAM at https://github.com/alanlukezic/d4sm. That is the right fork when your problem is tracking several objects at once rather than one object among lookalikes. Single-target DAM4SAM will not do it.

For the memory mechanism itself, the README names EdgeTAM and SAM3 only as related search terms, not as documented comparisons, and the repository does not benchmark against them. Treat any claim about how DAM4SAM compares to those as unverified by this material.

Maintenance, dependencies and what to check first

The dependency set is fully pinned. requirements.txt fixes versions down to patch level, including numpy==1.26.4, opencv-python==4.10.0.84, hydra-core==1.3.2 and iopath==0.1.10, and pulls the nvidia-cu12 wheels. Pinning is good for reproducing a paper and bad for living with over time: the pins will age, and lifting them means re-validating the tracker. pyproject.toml declares only setuptools>=61.0 and torch>=2.1.0 for the build, so the runtime pins in requirements.txt are the real constraint.

On licence, the honest reading is that the repository's own terms are undocumented while the vendored SAM2 code carries Meta's Apache 2.0 header in setup.py. Apache 2.0 includes patent and attribution terms, and the file header references a LICENSE file in the source tree. Whether that file is present in this repository is something you can check in seconds, and you should, before you build anything on top of it. Nothing here is legal advice.

What to verify first, in order: that the CUDA extension builds and the _C import succeeds; that download_ckpts.sh fetches the checkpoints the configs in sam2/ expect; and that run_bbox_example.py produces a sane mask on a frame where you already know the answer. Those three checks separate a working environment from one that runs and quietly skips postprocessing.

Editorial conclusion

Adopt DAM4SAM if you already have a working SAM2 environment, a CUDA GPU, and a distractor-heavy sequence you want to measure rather than watch. Do not adopt it if you need a packaged library, a stable API, or CPU-only inference. Before committing, verify the CUDA extension builds (python setup.py build_ext --inplace), check which checkpoints download_ckpts.sh actually fetches, and confirm the licence of the sam2/ directory you inherit, since the repository itself carries no LICENSE file.

Frequently asked questions

Official sources

1. Issues
2. jovanavidenovic/DAM4SAM on GitHub
3. Project website
4. README