mvdust3r: MV-DUSt3R+ pose-free 3D reconstruction from a few RGB views

One forward pass, no pose estimation first

The short version in the README is that this is multi-view, pose-free, RGB-only 3D reconstruction in one step, and that it also supports new view synthesis and relative pose estimation. That is the whole pitch of the underlying paper, MV-DUSt3R+ Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds, presented at CVPR 2025 as an oral, and the two seconds in the title is the part that shapes the engineering. Traditional pipelines estimate camera poses for each image and then optimise a reconstruction, which means two error sources and two stages of runtime. The approach here folds the reconstruction and the pose prediction into a single model so that the output is a point cloud with camera poses attached. The README also notes the demo shows the point cloud along with the predicted camera poses, with 3DGS visualisation listed as future work, so what you get is geometry and pose rather than a rendered scene.

The two stage recipe fine-tunes from DUSt3R

The training scripts in the scripts/ folder reveal how the model is built, and the answer is that MV-DUSt3R+ is not trained from scratch. The repository includes the DUSt3R codebase, with croco/ and dust3r/ directories carrying the model code, and the checkpoint table states that the training is finetuned upon the pretrained DUSt3R model, whose weights are listed as DUSt3R_ViTLarge_BaseDecoder_224_linear.pth. Two fine-tuning stages are defined. Stage 1 trains on 8 views, stage 2 trains on a mixed set of 4 to 12 views, and the checkpoint names MVDp_s1.pth and MVDp_s2.pth correspond to those two stages. Plain MV-DUSt3R, the checkpoint called MVD.pth, is the version without the two stage recipe. The staged approach is the interesting design decision, since training on a fixed small number of views and then broadening to a range of view counts is what lets the model handle a variable number of inputs at inference time.

Weights are not in the repository

The repository carries a checkpoints/ directory, but the README is explicit that you have to download the weights yourself before trying the demo or the evaluation, and the files live on Hugging Face in the Zhenggang/MV-DUSt3R repository on Hugging Face. There are three model files to choose between: MVD.pth for MV-DUSt3R, MVDp_s1.pth for MV-DUSt3R+ trained on stage 1 with 8 views, and MVDp_s2.pth for MV-DUSt3R+ trained on stage 1 and then stage 2 with mixed 4 to 12 views. Training trajectories are also published on Hugging Face for ScanNet, which the README says to place in the trajectories folder. The tree at the repository root shows the supporting cast of the project: demo.py, train.py, inference_global_optimization.py and a batch variant of it, plus scripts/, datasets_preprocess/, requirements.txt and install.sh. The code for generating train and evaluation trajectories is under scripts/tuple_gen, which is the part you would need if you want to train on your own data rather than reproduce the paper.

Installation assumes a Linux server with CUDA 12.4

The README opens the installation section with a constraint rather than a promise: the authors only test this on a Linux server with CUDA 12.4. The steps are a clone, a virtual environment created by a shell script, and an optional step for compiling the CUDA kernels for RoPE, which the README notes are the same as in DUSt3R and Croco. The clone is:

git clone https://github.com/facebookresearch/mvdust3r.git
cd mvdust3r

The environment is then created under anaconda:

./install.sh

The README adds a parenthetical that the versions of pytorch and pytorch3d should be changed if you need another CUDA version, which tells you the script pins a specific stack. The optional kernel build is a compile step in the croco model directory:

cd croco/models/curope/
python setup.py build_ext --inplace
cd ../../../

The pinned requirements give a good picture of the stack: torch 2.5.1, torchvision 0.20.1, numpy 1.26.4, gsplat 1.4.0, along with tensorflow, h5py, scikit-learn, trimesh, opencv-python and gradio.

The Gradio demo is slower than the published claim

The demo is a Gradio UI, started with:

python demo.py --weights ./checkpoints/{CHECKPOINT}

The README warns that the demo's inference is slower than what is claimed in the paper, because of the overheads of Gradio and model loading, and points you at the evaluation code if you need the faster runtime. That caveat is worth taking seriously before anyone builds a product demo around the two second figure. The demo also has two inputs you need to understand. The input can be multiple images or a video, and the README says explicitly that a single image is not supported, which is a consequence of the method being multi-view by definition. A confidence threshold controls how many low confidence points get filtered out of the result, and a frame count setting applies only to video input, choosing how many frames are uniformly sampled for the reconstruction. The README links to a discussion thread with tips for improving quality, particularly when a scene spans multiple rooms.

Evaluation data is contract gated and training data is incomplete

The datasets are the sharpest practical constraint. The project trains and tests on five datasets, ScanNet, ScanNet++, HM3D, Gibson and MP3D, and the README instructs you to go to each dataset's own website, sign a contract, download and extract the data into the data folder. That is not a download link, it is an access negotiation with five different institutions, and it is the step where most evaluation attempts stall. Only the ScanNet evaluation trajectories have been released so far. The training side is explicitly incomplete: the README says the trajectories of the training data and the trajectory generation code are still being prepared, while the training scripts train_mvd.sh, train_mvdp_stage1.sh and train_mvdp_stage2.sh are provided and are said to give more information about the training. For evaluation there are three scripts, test_mvd.sh, test_mvdp_stage1.sh and test_mvdp_stage2.sh, which the README says should reproduce the results of the paper on ScanNet across tables 2, 3, 4, S2, S3 and S5.

CC BY-NC 4.0 closes the door on commercial use

The licence section is one line and it is the most consequential fact in the README: the project uses CC BY-NC 4.0. A Creative Commons attribution and non-commercial licence lets researchers redistribute and build on the work, but not use it commercially, which rules out dropping this model into a product. That is a hard boundary for anyone who was planning to build on the checkpoints, and it is worth reading the LICENSE file in the repository rather than relying on a summary. The repository metadata also classifies the licence as one GitHub cannot categorise, which is why the README link matters. There are no GitHub releases, so the weights and the code have no versioned artefacts to point at, though the last push was on 2026-10-21 and the update log shows the project has been actively adding pieces, with data generation code for Habitat-Sim covering HM3D, Gibson and MP3D arriving on 2025-05-10 after the ScanNet equivalent on 2025-01-30.

Editorial conclusion

Adopt mvdust3r if you are doing research on multi-view geometry, want to reproduce the numbers in the MV-DUSt3R+ paper, or need a reference implementation of two stage sparse view fine-tuning from the DUSt3R codebase. Do not adopt it in a commercial product, because the README states the project uses CC BY-NC 4.0, a non-commercial licence. Verify first that you have a Linux server with CUDA 12.4, since that is the only configuration the authors test, download the checkpoints from Hugging Face into the checkpoints folder before running the demo or evaluation, and read the data instructions because ScanNet, ScanNet++, HM3D, Gibson and MP3D each require signing a contract with the dataset owners before you can obtain them.

Frequently asked questions

Official sources

1. facebookresearch/mvdust3r on GitHub
2. Issues
3. Project website
4. README