SLAM-Former puts camera pose and depth into a single transformer

One model for trajectory and reconstruction

SLAM-Former comes from the MARS Lab at Tsinghua University's IIIS, and the framing in the title is the whole idea: putting SLAM into one transformer. The paper is an arXiv preprint, arXiv 2509.16909, and the project has its own page, and the README records the acceptance to ECCV 2026 on 2026-06-18.

The problem it addresses is the usual shape of a monocular SLAM system. You have a sequence of images, and you want two things from it: where the camera was, and what the scene looks like in three dimensions. A conventional pipeline estimates each with its own model and then fuses the results, and the fusion step is where the errors compound. A single transformer that produces both quantities from the same sequence keeps them in the same representation, which is the argument the project makes implicitly by shipping one model rather than two.

The release history is a research project timeline rather than a software release history, and the repository has no GitHub releases at all. The preprint appeared on 2025-09-23, the training data was provided on 2026-02-26, SLAM code with KV pruning available arrived on 2026-03-04, and the training code followed on 2026-03-11 in a separate training branch. The last push to the repository was on 2026-08-10, and the project is not archived.

Three commands to get to a working environment

The setup is a clone, a conda environment on Python 3.11, and two pip commands. Nothing else is documented, and there are no prebuilt Docker images or wheels to start from.

git clone https://github.com/Tsinghua-MARS-Lab/SLAM-Former.git
cd SLAM-Former

The environment is created with conda, named after the project, and pinned to Python 3.11:

conda create -n SLAM-Former python=3.11
conda activate SLAM-Former 

Then the dependencies and the package itself are installed, the second command in editable mode so that the local source is what gets imported:

pip install -r requirements.txt
pip install -e .

The pinned requirements are worth a look before you start, because they describe the shape of the project. `torch==2.9.1` and `torchvision==0.24.1` are the deep learning base, and the list also includes `gsplat`, `open3d`, `trimesh`, `kornia`, `evo`, `lpips`, `accelerate` and `transformers`. Several of those are not generic utilities: `gsplat` and `open3d` point to a rendering and geometry pipeline, `evo` is a trajectory evaluation tool, `lpips` is a perceptual image metric, and `viser` and `rerun-sdk` are the two visualisation stacks the project uses. `moviepy==1.0.3` and `pyglet<2` are pinned to specific versions, which suggests those two are sensitive to the versions of their own dependencies.

The demo runs on an image folder, and the retention flag changes the result

The inference entry point is `slam/demo.py`, and the checkpoint comes from Hugging Face. You prepare a folder containing your image sequence and run:

python slam/demo.py \
    --ckpt_path .ckpt/checkpoint-10.pth.model \
    --image_folder /path/to/your/images/ \
    --output_dir ./output/result \
    --target_size 518 \
    --retention_ratio 0.5

The flag that deserves a second look is `--retention_ratio`. The README states that for evaluation you should use 100% kv, that is `--retention_ratio 1`, and points to issue #15 for details. The demo command above uses 0.5, which is the pruned setting. So the same script has two regimes, and the difference is not a cosmetic one: pruning the key value cache is what makes long sequences affordable, and turning it off is what the project considers the honest evaluation setting.

Anyone reproducing the paper's numbers, or comparing against another system, needs to run with `--retention_ratio 1`. Anyone running on a long sequence on limited memory is what the 0.5 default is for. The README does not quantify the accuracy cost of pruning, and it does not say what the memory saving is, so the trade has to be measured on your own sequences rather than read off a table.

`--target_size` is the other parameter to get right, and it is coupled to the checkpoint you download rather than being a free choice. The checkpoint list in the README is explicit about the pairing.

ConvHead checkpoints are matched to a target size, not interchangeable

There are three checkpoints, and each one is tied to a specific `--target_size` value. The plain v1 checkpoint is the one recommended for inference with `--target_size 518`. The `V1.1.pth` checkpoint is a ConvHead variant and is also recommended with `--target_size 518`. The `V1.1-long.pth` checkpoint is a ConvHead variant recommended with `--target_size 224`, and the README says it is trained with scaled sequence lengths and supports longer-sequence inference.

The ConvHead variants were added on 2026-05-11, and the README says the main point of ConvHead is to fix the grid artifact issue, with thanks given to Pi3X for the insight. Grid artifacts are the kind of defect that a visual inspection of a reconstruction would show immediately, and having a checkpoint that addresses them is a meaningful difference between the versions rather than a new feature list.

The coupling between checkpoint and target size is the trap here. Using `V1.1-long.pth` with a target size of 518, or v1 with 224, is not documented as supported, and the README does not say what happens if you get it wrong. Since the download links themselves are organised by size, with the 518 directory holding v1 and `V1.1.pth` and the 224 directory holding `V1.1-long.pth`, the intended pairing is visible from the file layout, but it has to be respected.

The longer-sequence variant is the one to reach for when a sequence does not fit, and it is the reason the 224 target size exists at all.

Two visualisation modes, and only one of them works on a server

The reconstruction can be watched or inspected afterwards, and the README is clear that they suit different environments.

The live mode adds `--vis` to the demo command. The 3D reconstruction process is then viewed interactively in Rerun, and the README says this mode is intended for local machines with a desktop session and is not recommended on remote servers, because it depends on launching an interactive viewer during inference.

The static mode is the one for a headless box. You run `slam/demo.py` without `--vis` so that it saves `final.ply`, `final_traj.txt` and `final_pc/`, and then start the browser based viewer:

python slam/visualize_results.py \
    --result_dir /path/to/output_dir \
    --port 8080

The viewer serves an HTTP page at `http://localhost:8080`, and the README recommends this mode for remote servers, with the port forwarded to your local machine and the forwarded URL opened in a browser. The forwarding step is a plain SSH tunnel:

ssh -L 8080:localhost:8080 user@remote-server

So the output of a run is three artefacts, a point cloud in PLY format, a trajectory file and a directory of point cloud data, and the visualisation step is a separate process that reads them. That separation is what makes the tool usable on a cluster, where nothing can open a window, and it also means the reconstruction can be inspected long after the run that produced it.

Training data and training code arrived in separate steps

The project released inference first and training later, which is the reverse of what a research codebase usually does. The training code landed on 2026-03-11 in a dedicated training branch, and the README points to that branch for details rather than folding it into the main branch.

The training data is hosted on Hugging Face under two dataset repositories. The main one, KevinConnorLee/SLF, collects ARKitScenes, MVS-Synth, ScanNet, ScanNet++, Blended-MVS and MegaDepth. The second, KevinConnorLee/preprocessed_Hypersim, holds Hypersim in a preprocessed form. The split is worth noting because it tells you the data pipeline is not uniform: most of the sources are published as raw datasets that you would otherwise have to harmonise yourself, while Hypersim is handed over already processed.

The `setup.py` at the repository root is minimal. The package is named `slamformer` at version 0.1.0, and the packages listed are `evals` and `src/slamformer` with their subpackages. An `evals` package in the distribution is a good sign for anyone planning to compare results, since the evaluation code travels with the installable package rather than living in a notebook. The root directory itself is small: `slam/` for the pipeline and scripts, `src/` for the model, `requirements.txt`, `setup.py`, `LICENSE` and `README.md`.

The weights are non-commercial even though the code is permissive

The licensing table at the end of the README is the most consequential part of the page, and it is short. The project adopts a dual licensing strategy. The code is BSD 3-Clause, and commercial use is permitted. The model weights, the checkpoints, are under CC BY-NC 4.0, and commercial use is strictly non-commercial.

So the model files cannot be shipped in a commercial product, a paid service or a device you sell, even though the surrounding code can. That is a real constraint on adoption, and it is easy to miss because the repository's licence badge and the SPDX identifier for the repository point at the permissive side.

A related question is what a different SLAM system would look like. The pipeline this project replaces is the one where depth and camera pose come from separate models and are then fused, and the point of the single transformer is to remove that fusion step. The cost of that consolidation is that the whole thing is one set of weights with one licence, so you cannot take the pose estimation and license it commercially while leaving the rest out.

The project is still moving. On 2026-08-08 the team released another SLAM transformer, SLAMFormer-Infinity, with its own project page, which suggests the current model is the first step in a line rather than the final one. The last push to the repository was on 2026-08-10.

Editorial conclusion

SLAM-Former suits a researcher or robotics team that wants a single model producing both camera trajectory and dense reconstruction from an image sequence, and it does not suit a product team that needs commercial rights to the model weights, because the checkpoints are released under CC BY-NC 4.0 while the code is BSD 3-Clause. Before you evaluate it, run the demo twice, once with the `--retention_ratio 0.5` setting from the README and once with `--retention_ratio 1`, which the README specifies for evaluation, so you can see what the KV pruning costs on your own sequences.

Frequently asked questions

Official sources

1. Issues
2. License: BSD-3-Clause
3. Project website
4. README
5. Tsinghua-MARS-Lab/SLAM-Former on GitHub