TTT3R: turning 3D reconstruction into a test-time training loop

The update happens while the sequence is being reconstructed

TTT3R is described as 3D Reconstruction as Test-Time Training, and the whole contribution is described in one line as a simple state update rule to enhance length generalization for CUT3R.

The problem it addresses is specific to feed forward reconstruction models. CUT3R ships checkpoints trained on between four and sixty four views, and the natural question is what happens when you hand the model a hundred frames or a long video. A model that only sees a bounded window during training tends to accumulate drift once the input sequence exceeds what it was fitted for.

The test-time training framing means the model updates its own state as it processes the sequence, rather than running as a frozen function over frames. The repository exposes this as a choice at inference time through the model_update_type argument, where the value ttt3r selects the new rule and the alternative is the plain model behaviour it modifies.

The paper is arXiv 2509.26645, accepted to ICLR 2026, and the badge set also marks it as a top five paper at China3DV 2026. The authors are Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger and Anpei Chen, and the project page is hosted at rover-xingyu.github.io/TTT3R.

The code is not written from scratch. The acknowledgements list CUT3R, Easi3R, DUSt3R, MonST3R, Spann3R and Viser as the repositories it builds on, and the directory listing shows src/ holding the vendored model code alongside the project's own entry points.

Building the environment needs Python 3.11 and compiled RoPE kernels

The setup has three parts and the middle one is the part people underestimate.

First the repository and the conda environment:

git clone https://github.com/Inception3D/TTT3R.git
cd TTT3R
conda create -n ttt3r python=3.11 cmake=3.14.0
conda activate ttt3r
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia  # use the correct version of cuda for your system
pip install -r requirements.txt
# issues with pytorch dataloader, see https://github.com/pytorch/pytorch/issues/99625
conda install 'llvm-openmp<16'
# for evaluation
pip install evo
pip install open3d

Two details in that block are fixes rather than boilerplate. The llvm-openmp pin below sixteen is there for a PyTorch dataloader issue, and the CUDA version comment tells you to match your own system rather than assume 12.1. cmake is pinned to 3.14.0 in the environment creation, which matters because the next step compiles an extension.

Second, the CUDA kernels for RoPE have to be compiled, the same way CroCo v2 does it:

cd src/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../../

That is a build step inside the checkout rather than a wheel from an index, so a plain pip install of the requirements will not be enough on its own. The requirements file itself is a plain list of packages, numpy pinned to 1.26.4 and pillow to 10.3.0, with viser and gradio for the interactive parts, lpips and hydra-core for the evaluation side, and transformers and accelerate for the model loading.

Third, the weights. CUT3R provides checkpoints trained on 4 to 64 views, named cut3r_512_dpt_4_64.pth, and the documented download is:

cd src
gdown --fuzzy https://drive.google.com/file/d/1Asz-ZB3FfpzZYwunhQvNPZEUA8XUNAYD/view?usp=drive_link
cd ..

The checkpoint name encodes the resolution, 512, and the training view range, 4 to 64, which is the clearest statement in the repository of where the length generalisation problem starts.

One demo command handles both a video and an image folder

The inference demo is a single script, demo.py, and its arguments are where the method is actually configured.

The command shape is:

CUDA_VISIBLE_DEVICES=6 python demo.py --model_path MODEL_PATH --size 512 \
    --seq_path SEQ_PATH --output_dir OUT_DIR --port 8080 \
    --model_update_type ttt3r --frame_interval 1 --reset_interval 100 \
    --downsample_factor 1000 --vis_threshold 5.0

The comments above it in the documentation say the input can be a folder or a video, and that the script runs inference with TTT3R and visualises the output with Viser on port 8080. The examples directory ships two sample inputs, examples/taylor.mp4 and examples/westlake.mp4, and the documented example run uses the Taylor video with a model path of src/cut3r_512_dpt_4_64.pth and an output directory of tmp/taylor.

Each argument carries meaning. model_update_type selects the state update rule, and the value used throughout the examples is ttt3r. frame_interval controls how often a frame is consumed from the input, reset_interval controls how often the model state is reset, and downsample_factor and vis_threshold control the point cloud that gets displayed, with a threshold of 5.0 in one example and 6.0 and 10.0 in others, which is a knob for how much of the noisy reconstruction to draw.

The reset_interval difference between the two examples is worth noticing. The Westlake example uses 100 and the Taylor example uses 50, which tells you the reset frequency is something you tune per scene rather than a fixed constant.

Results are written to the output directory, and the visualiser itself lives in viser_utils.py at the repository root.

Evaluation lives in its own guide and its own directory

The readme does not attempt to describe how the method is measured. It points to eval/eval.md for the details, and the top-level listing confirms there is a whole eval/ directory rather than a single script.

That separation is the honest structure for a paper repository. The demo shows the method running, the evaluation guide describes the protocol, and the extra packages installed for it are the evo package for trajectory evaluation and open3d for point cloud and geometry work. The requirements file also carries lpips, which is an image similarity measure, and hydra-core for configuration, so the evaluation is configured rather than hard coded.

There is also a datasets_preprocess/ directory at the top level, which is where the evaluation data preparation lives, and an add_ckpt_path.py script whose name suggests it adjusts the checkpoint path inside the configuration. The other entry points are demo.py and viser_utils.py, with src/ holding the model code.

So if you want to reproduce the paper rather than just see the effect, the path is: preprocess the datasets, follow eval/eval.md, and use the evaluation dependencies the readme installs. The readme deliberately stops short of summarizing what the numbers should be, and this article does the same.

For the citation, the repository ships a BibTeX entry for the arXiv preprint, which is the form most papers in this area use for the preprint rather than the camera ready version.

A custom licence and a May 2026 last push

The licence is the first thing to check and it is not a standard one. The repository carries a LICENSE file at the root that GitHub's classifier could not categorise, which means it is a custom licence rather than MIT, Apache or BSD. Nothing in the readme summarises the terms, so the file itself is the only place to read them.

The timeline is worth noting alongside that. The repository is not archived and the last push was on 2026-05-11, on the default branch named main. There are no GitHub releases at all. The paper was accepted to ICLR 2026 and the arXiv preprint is dated 2025, which means this is code published around a conference cycle rather than a released library.

The upstream dependencies are the other maintenance consideration. The code sits on CUT3R, which sits on DUSt3R, and the acknowledgements also name Easi3R, MonST3R and Spann3R, all of which move on their own schedules. A research repository like this tracks its upstream bases rather than vendoring stable copies, so an upstream refactor can break the build, and the pinned CUDA and Python versions in the install steps are the mitigations the authors provide.

The topics attached to the repository are 3d-reconstruction, rnn-model and slam, which is a good description of what the state update rule is doing: it turns a feed forward reconstruction model into something with recurrent state across the sequence.

Editorial conclusion

TTT3R suits a researcher reproducing the ICLR 2026 result, or an engineer with a long video whose scene drifts out of range for a model trained on up to sixty four views, and who can build CUDA extensions for the RoPE kernels. It does not suit a production pipeline that needs a stable package, because the repository has no releases, the licence is a custom file GitHub cannot classify, and the last push was on 2026-05-11. Before running it, build the conda environment with Python 3.11, compile the RoPE kernels in src/croco/models/curope, download the cut3r_512_dpt_4_64.pth checkpoint, and read eval/eval.md for the evaluation procedure.

Frequently asked questions

Official sources

1. Inception3D/TTT3R on GitHub
2. Issues
3. Project website
4. README