Scal3R moves the reconstruction loop to test time, for very long image sequences

The idea: train on the test sequence itself

Scal3R expands to Scalable Test-Time Training for Large-Scale 3D Reconstruction, and it was selected as a highlight paper for CVPR 2026, with a project page at zju3dv.github.io/scal3r and an arXiv entry numbered 2604.08542. The name tells you the departure from the usual pipeline. A conventional multi-view reconstruction model consumes a bounded set of images, runs a feed-forward pass and emits geometry and camera parameters. A test-time training approach instead adapts the model to the sequence being reconstructed, so the optimisation happens during inference rather than as a separate offline stage on a training corpus. The stated target is scalability to large-scale 3D reconstruction, and the topics listed for the repository are 3d-reconstruction, depth-estimation and large-scale. The authors come from Zhejiang University, with a long list including Tao Xie, Peishan Yang, Yudong Jin, Sida Peng, Xiaoyang Guo and Xiaowei Zhou. The README also records that the project builds on VGGT, VGGT-Long and LaCT, and thanks those authors for making their code and ideas publicly available.

Installing with the script and the conda environment it expects

Installation is a single script.

bash scripts/install.sh

The README explains what it does. It creates or reuses a conda environment named `scal3r`, installs the core dependencies from requirements.txt, and installs Scal3R in editable mode. By default it uses `uv pip` inside that conda environment, with a plain `pip` fallback available. Detailed installation instructions and PyTorch and CUDA guidance live in docs/install.md. The dependency set in requirements.txt is a conventional vision stack: einops, huggingface_hub, matplotlib, numba, numpy, opencv-python, open3d, Pillow, pykitti, pypose, PyYAML, rich, scipy and tqdm. The pyproject.toml adds torch and torchvision on top of that list, requires Python 3.10 or later, uses setuptools as the build backend, and declares a console script named scal3r-run pointing at scal3r.run:main. The presence of open3d and pypose is a hint about the pipeline, since pypose handles pose representation and optimisation and open3d handles the point cloud output.

Two checkpoints, and the download commands for both

The weights are not bundled and there are two of them, which is the first thing that trips people up. The model checkpoint lives on Hugging Face under the xbillowy/Scal3R repository, and a separate DINO SALAD checkpoint is fetched from a GitHub release.

mkdir -p data/checkpoints
hf download xbillowy/Scal3R scal3r.pt --repo-type model --local-dir data/checkpoints
curl -L https://github.com/serizba/salad/releases/download/v1.0.0/dino_salad.ckpt -o data/checkpoints/dino_salad.ckpt

Both land in data/checkpoints. The DINO SALAD checkpoint is a DINO-style backbone, and SALAD is a dense feature matching method, so the second file is the feature extractor the matching stage depends on rather than a second copy of the reconstruction weights. The README's news entries also point at the pace of the release: the inference code was released on 2026-04-10, inference acceleration was enabled on 2026-04-17, and point cloud and camera pose visualisation tools followed on 2026-04-23. The visualisation tools were later turned into a separate repository, since the README's top-level entries here include an assets directory but no viewer, and the TODO list still asks for a simple viser viewer.

Running inference and the arguments that matter

Inference runs as a module on a folder of images.

python -m scal3r.run --input_dir /path/to/images

You can pass an explicit tag and output directory.

python -m scal3r.run \
  --input_dir /path/to/images \
  --tag demo \
  --output_dir data/result/custom/demo

The documented arguments describe the whole cost model of the method. The model config path comes from `--config`, defaulting to configs/models/scal3r.yaml. The `--tag` controls the default output directory name when `--output_dir` is not set. The pair `--block_size` and `--overlap_size` control chunking for long-sequence inference, which is how a long sequence is split so that memory stays bounded while neighbouring chunks overlap. The pair `--save_dpt` and `--save_xyz` control whether depth maps and point clouds are exported at all. And `--offload_batches` with `--offload_outputs` control whether batches and outputs are offloaded to disk. That last pair is the escape hatch for machines that do not fit the working set in RAM, and it is worth knowing before you start a long run, because re-running without it is cheaper than discovering it mid-job.

What lands on disk after a run

By default, inference results are written to data/result/custom/<tag>/, with runtime artifacts in data/result/custom/<tag>/runtime/. The result directory typically contains mat.txt for the predicted camera poses, where each row is a raveled 4x4 camera-to-world transform matrix, and intri.yml plus extri.yml for camera parameters in EasyVolcap format. The depths/ directory appears when depth export is enabled, and points/ appears when point-cloud export is enabled. The EasyVolcap compatibility is a practical choice rather than a cosmetic one, because it means the output plugs into an existing visualisation and evaluation format instead of requiring a converter. The poses file being raveled 4x4 matrices is worth noting if you plan to load it into your own tooling, since it is a flat array per row rather than a nested structure. Runtime artifacts in their own directory is a small thing that makes it easy to keep or discard the expensive intermediate state separately from the results you actually want.

What is missing, and the licence

Two gaps are stated rather than hidden. First, this release currently includes inference only, and evaluation and benchmark code are not part of the public package yet, with the TODO list marking release of the evaluation code along with dataset preparation scripts as still open. That matters for anyone choosing between research codebases, because it means the numbers in the paper cannot be reproduced from this repository as it stands. Second, there is no training code and no mention of how the model was trained, which is consistent with a test-time training method whose training happens at inference time but still leaves the question of the base model open. The licence is the one GitHub cannot classify, so the terms are in the LICENSE file at the repository root and need reading before use in anything commercial, particularly since the outputs are reconstructed geometry and camera parameters that might carry obligations depending on the captured scenes. On maintenance, the repository has no GitHub releases, so version tracking is by commit, and the last push was on 2026-09-15.

Where Scal3R is the wrong choice

The clearest wrong case is a short sequence. If you have twenty images of a small object, a feed-forward model such as VGGT does the job, and the machinery that makes Scal3R handle a thousand images becomes overhead you do not need. The second wrong case is when you need to evaluate. Without the evaluation code or the dataset preparation scripts, you cannot reproduce the paper's comparisons or run the standard reconstruction benchmarks, so for a team whose deliverable is a benchmark table, this repository is incomplete as it stands. The third is hardware. The README points to PyTorch and CUDA guidance in the documentation, and the checkpoint sizes implied by a test-time training setup are not small, so a machine without a GPU is not a realistic target. The fourth is the dependency on upstream projects. Scal3R builds on VGGT, VGGT-Long and LaCT, and an implementation that inherits from three codebases inherits their version constraints too, which is a cost that a single self-contained implementation would not carry.

Editorial conclusion

Adopt Scal3R if you are reconstructing very long image sequences and the memory cost of a single monolithic forward pass is what stops you. Skip it if you need evaluation or benchmark numbers today, because the README states the release includes inference only and that evaluation and benchmark code are not part of the public package. Before you run it, download both checkpoints into data/checkpoints, since the model weights and the DINO SALAD checkpoint are separate, and plan on a CUDA-capable machine with PyTorch.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. zju3dv/Scal3R on GitHub