BundleTrack: 6D pose tracking without CAD models, run through Docker

What BundleTrack solves that model-based trackers cannot

Most 6D object pose trackers assume a CAD model exists. Sometimes the exact instance model, sometimes only a category-level model shared across similar objects. BundleTrack's paper abstract states the opposite premise: the framework does not depend on 3D models at either the instance or category level. The target is a novel object, seen for the first time, with no offline training and no template library.

The intended user is a robotics researcher working on manipulation, where the object in front of the gripper may not be in any database. The repository topics list manipulation, robotics and pose-graph-optimization alongside the vision terms, which matches that audience. The README also points readers with CAD models elsewhere, to the author's se(3)-TrackNet repository, so the two projects are deliberately split by whether a model exists.

The cost of dropping the model requirement is visible in the setup. Instead of loading a mesh, the pipeline needs a segmentation network, a feature network with downloaded weights, and precomputed masks for the benchmark scenes. That is the trade: less prior knowledge about the object, more machinery around it.

Segmentation, feature extraction and memory-augmented pose graph optimization

The abstract names three components: deep learning for segmentation, robust feature extraction, and memory-augmented pose graph optimization for spatiotemporal consistency. The repository layout reflects them. transductive-vos.pytorch is the video segmentation network, whose weights go under transductive-vos.pytorch/pretrained. lf-net-release is the feature detection network, whose weights go under lf-net-release/release/models/indoor. The masks directory holds precomputed segmentation masks, which the README says to extract into BundleTrack/masks.

The pose graph is the part that carries the paper's claim of long-term, low-drift tracking under significant occlusions and object motions. Rather than trusting each frame independently, the system keeps a memory of past observations and optimizes consistency across them. The README does not document the graph structure, the window size, or how frames are added and marginalized, so anyone wanting those details has to read the paper rather than the repository.

The feature network runs as a separate service. The README instructs you to start lf-net-release/docker/run_container.sh in one terminal and run python run_server.py from lf-net-release in another, then to pass a port number to the tracking script. That is the data flow in practice: the tracker sends frames to a feature server over a socket, and the server returns detections. The CUDA implementation is what the abstract credits for the reported 10Hz for the entire framework.

Installing BundleTrack with the supplied Docker images

The README is explicit that from-scratch setup is complicated and not supported. The supported path is Docker, and the README says you do not need to know how Docker works, only a few basic commands. Install Docker first, then pull both images. The first is the tracker environment, the second is the feature network environment.

docker pull wenbowen123/bundletrack:latest
docker pull wenbowen123/lf-net-release-env:latest

Before starting the container, edit docker/run_container.sh and set the paths for BUNDLETRACK_DIR, NOCS_DIR and YCBINEOAT_DIR to where those directories live on your machine. The README does not describe what happens if one of these points at a missing directory, so check the file after editing. Then launch the container and build inside it.

bash docker/run_container.sh
cd [PATH_TO_BUNDLETRACK]
rm -rf build && mkdir build && cd build && cmake .. && make

The build uses CMake and produces the binaries under build. After that you still need data: the indoor feature weights extracted to lf-net-release/release/models/indoor, the segmentation weights to transductive-vos.pytorch/pretrained, and the precomputed masks to BundleTrack/masks. For a first real run on NOCS, start the feature server in a separate terminal, then run the tracker script with the scene id, port and model name.

bash lf-net-release/docker/run_container.sh
cd [PATH_TO_BUNDLETRACK]
cd lf-net-release && python run_server.py

python scripts/run_nocs.py --nocs_dir [PATH_TO_NOCS] --scene_id 1 --port 5555 --model_name can_arizona_tea_norm

Output goes to the debug_dir set in the config file, which the README says defaults to /tmp/BundleTrack/. If you need more detail, raise LOG to 2 or higher in config_nocs.yml. The README notes that eval_nocs.py deliberately perturbs the initial ground-truth pose to add noise for evaluation, so use run_nocs.py, not the evaluation script, if you want to see actual tracking behaviour.

Where BundleTrack is the wrong tool

The clearest boundary is in the README itself: if you already have a CAD model for your object, this is not the repository the author points you to. se(3)-TrackNet is the recommendation for CAD model-based tracking. Using BundleTrack in that situation means paying for segmentation weights, feature weights and a two-container deployment to solve a problem you do not have.

The second boundary is deployment. There is no pip package, no released binary, and no Python API documented in the README. The entry points are scripts under scripts/ driven by YAML config files, and the build is CMake inside a container. A team that wants to embed 6D tracking in a product pipeline will find no documented integration path here.

The third is hardware and environment. The abstract credits a CUDA implementation for the 10Hz figure, and the feature network runs as its own container with its own weights. The README does not state minimum GPU memory, supported CUDA versions, or whether CPU-only operation is possible. Anyone without a suitable NVIDIA GPU has no documented route.

Finally, the benchmark workflow assumes you can download and lay out large datasets. NOCS requires the dataset itself, converted ground-truth text pose files, and an addon archive, arranged in a specific tree with NOCS-REAL275-additional, real_test, gts/real_test_text and obj_models. YCBInEOAT requires a similarly specific folder layout per scene. This is a research reproduction setup, not a quick evaluation.

Alternatives and how their approach differs

The README's own alternative is se(3)-TrackNet, by the same author, for the case where an object instance CAD model is available. The difference is not incremental: se(3)-TrackNet can use the model, while BundleTrack is built so that no instance or category-level model is needed. If your objects are known and modelled, the model-based route removes the segmentation and feature-network dependencies that dominate BundleTrack's setup.

Among the search terms people use around this project, BundleSDF and FoundationPose appear as related systems, but the repository does not describe either one's mechanism, inputs or licence, so no comparison can be made from it alone. The honest statement is that the README positions BundleTrack against category-level 6D tracking and dynamic SLAM methods in its experiments, and against instance-CAD methods such as se(3)-TrackNet in its guidance, and that is the only comparison the repository supports.

Within the paper's own framing, the trade is information for infrastructure. Category-level trackers need a category model and offline training. BundleTrack needs neither, but the abstract notes that against instance-CAD methods it achieves comparable performance despite reduced information requirements, which is a claim about accuracy parity, not about being easier to run.

Maintenance, licence and what upgrading costs

The repository is not archived, and the last push was on 2026-04-13. There are no retrieved releases, so there is no versioned artifact to pin and no changelog to read before upgrading. Updating means pulling the Docker images again and rebuilding, or tracking the master branch, which the repository uses as its default branch.

Upgrade cost centres on the Docker images and the downloaded weights. The README tells you to pull wenbowen123/bundletrack:latest and wenbowen123/lf-net-release-env:latest, and latest is a moving tag. If either image changes, the build commands (cmake .. && make) and the run scripts under scripts/ are the surface that can break, and the README does not document rollback to a previous image. The weights themselves are separate archives hosted at archive.cs.rutgers.edu, not part of the images, so a re-pull does not refresh them.

The licence is Apache-2.0, as stated in the repository. That is a permissive licence, but the repository bundles or depends on third-party components with their own terms: the feature network under lf-net-release, the segmentation network under transductive-vos.pytorch, and the NOCS and YCB datasets, each of which carries its own licence and usage conditions. The README does not summarise those terms. Check each component's own licence before any redistribution, and treat the Apache-2.0 file at the repository root as covering the BundleTrack code only.

Editorial conclusion

BundleTrack is for robotics and vision researchers who need 6D pose tracking of objects they have no CAD model for, and who are willing to run two Docker containers and download several gigabytes of weights, masks and benchmark data. It is not for anyone who wants a pip install, a Python API, or a supported from-scratch build: the README states that setting up from scratch is complicated and not supported in this repo. Before committing, verify that your GPU and driver can run the supplied images, that the paths for BUNDLETRACK_DIR, NOCS_DIR and YCBINEOAT_DIR in docker/run_container.sh match your disk layout, and that you can keep the lf-net-release server running on the port you pass to scripts/run_nocs.py.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. README
4. wenbowen123/BundleTrack on GitHub