DetectorFreeSfM runs structure from motion without a keypoint detector

Matching replaces detection, and that is the whole contribution

Structure from motion pipelines normally begin with a detector. A keypoint detector finds sparse, repeatable image points, those points are matched across frames, and the camera geometry is recovered from the correspondences. DetectorFreeSfM removes the first step and starts from dense matching instead, which is what the paper title and the CVPR 2024 publication refer to. The repository is the code release for that paper, from a group at Zhejiang University with collaborators listed in the README, and the project page is linked alongside the paper PDF. The team also reports first place in the Image Matching Challenge 2023, with a discussion thread linked in the README. The practical reason to care about a detector-free pipeline is texture. A detector needs corners to latch onto, and images with soft gradients, repeated patterns or large flat regions give it very little, whereas a dense matcher can find correspondences across a much wider area of the image.

Built on COLMAP and HLoc rather than from scratch

The README is explicit that the code is partially based on COLMAP and HLoc, and thanks the authors of both for their work. That inheritance shows in the output format and in the repository layout. The reconstruction result is written in COLMAP format and can be inspected with the colmap gui, which means the artefacts are readable by tooling that already exists rather than locked in a private format. The top level entries show the division of labour: src/ holds the implementation, third_party/ holds vendored code, tools/ holds helper scripts, SfM_dataset/ holds the data directory, and the two configuration trees hydra_configs/ and hydra_training_configs/ separate the evaluation and training setups. There is also an environment.yaml and a requirements.txt, a .gitmodules file for the submodules the code expects, and eval_dataset.py and train_multiview_matcher.py as the two entry points. A separate scripts/ directory holds dataset-specific shell wrappers.

Hydra configs pick the dataset and the matcher from the command line

Configuration is done with Hydra, and the command line is where you see it most clearly. To run the demo you first edit line 22 of hydra_configs/demo/dfsfm.yaml to point at the absolute path of the repository, then run a single command:

python eval_dataset.py +demo=dfsfm.yaml

The result lands in a directory under the example scene named for the matcher and the run configuration, in a colmap_refined subfolder. The evaluation commands work the same way, with a config group prefixed by a plus sign and the matcher chosen by overriding a single key. On ETH3D, the coarse matcher can be set to loftr_official, aspanformer or matchformer, and the same three options appear for the triangulation evaluation, which is a separate config group:

python eval_dataset.py +eth3d_sfm=dfsfm.yaml neuralsfm.NEUSFM_coarse_matcher='loftr_official'
python eval_dataset.py +eth3d_sfm=dfsfm.yaml neuralsfm.NEUSFM_coarse_matcher='aspanformer'
python eval_dataset.py +eth3d_sfm=dfsfm.yaml neuralsfm.NEUSFM_coarse_matcher='matchformer'
python eval_dataset.py +eth3d_tri=dfsfm.yaml neuralsfm.NEUSFM_coarse_matcher='loftr_official'
python eval_dataset.py +eth3d_tri=dfsfm.yaml neuralsfm.NEUSFM_coarse_matcher='aspanformer'
python eval_dataset.py +eth3d_tri=dfsfm.yaml neuralsfm.NEUSFM_coarse_matcher='matchformer'

The IMC and TexturePoorSfM datasets are not run through eval_dataset.py directly but through shell wrappers in scripts/, one of which is eval_imc_dataset.sh and the other eval_texturepoorsfm_dataset.sh. A configuration system this uniform is a genuine advantage when you want to sweep matchers, since changing the matcher is an override rather than an edit.

The dataset folder names are compulsory, not conventional

Data preparation is the step most likely to stall a first run, and the README treats it as a hard requirement. The directory structure the system expects is a SfM_dataset folder holding one directory per dataset, each with scene directories, each scene holding an images directory, and optionally an intrins directory and a poses directory for evaluation. The three names images, intrins and poses are marked as compulsory because the system identifies content by those names, so a dataset arranged slightly differently will not be found even if the images are correct. Cameras are given as text files named camera_name_1.txt and poses as pose_name_1.txt. The README also states that intrins and poses are used for evaluation, so they are optional for inference and required for scoring. Preparation instructions live in a separate DATASET_PREPARE.md document, and training data follows the same MegaDepth format described there.

Speed comes from Ray workers, chunk sizes and batch tuning

Three tuning levers are documented, and each one is a configuration key rather than a code change. The first is parallelism across scenes: setting ray.enable=True and ray.n_workers to your GPU count lets many scenes in a dataset be evaluated at the same time. The second is parallelism inside a single scene, for a scene with many images such as Bridge in the ETH3D dataset, where sub_use_ray=True and sub_ray_n_worker set the worker count for the coarse matching and multi-view refinement matching phases. The third is the memory budget for the refinement phase. The README explains that tracks are chunked in refinement matching, with neuralsfm.NEUSFM_refinement_chunk_size set to 2000 so the process works on a GPU with less than 12GB of VRAM, and that a larger chunk size can be used for a faster run on larger cards. That is a real constraint rather than a recommendation, and it tells you the reference configuration targets consumer GPUs.

Training the multiview refinement matcher takes eight GPUs by default

The trainable component in this repository is the multiview matching refiner, and the entry point is a separate script with its own configuration tree. The dataset has to be downloaded and formatted following DATASET_PREPARE.md first, then a single command starts the run:

python train_multiview_matcher.py +experiment=multiview_refinement_matching.yaml paths=dataset_path_config trainer=trainer_config

The GPU identifiers live in hydra_training_configs/trainer/trainer_config.yaml, and the README states that eight GPUs are used for training by default. The training stack is pinned in requirements.txt, and the pins are old enough to matter: pytorch-lightning at 1.3.5, ray at 2.7.1, opencv_python at 4.4.0.46, pycolmap at 0.3.0, einops at 0.3.0, kornia at 0.4.1, torchmetrics at 0.6.0 and pickle5 at 0.0.11, with yacs, joblib, timm, hydra-core, omegaconf, wandb, rich and natsort left unpinned. An old pin on pytorch-lightning and a pycolmap at 0.3.0 are the two that will shape your install the most.

A custom licence and an unversioned repository

The licence is the item to check first, because the repository is listed as carrying a custom licence that GitHub cannot classify, which is a signal to open the LICENSE file rather than assume a standard identifier. The rest of the project metadata is thin on release discipline. There are no GitHub releases, so there is no tagged version to depend on, and the last push to the repository was on 2026-09-15, a little over a month before this review was written. For research code that is normal, but it means a dependency on this repository is a dependency on a moving commit unless you pin one. The practical pairing to plan for is a research reproduction, where the citation entry in the README is the artefact that matters, and a production use, where the absence of tags, the eight-GPU training default and the 12GB VRAM target in the refinement chunking are the constraints to price in before you start.

Editorial conclusion

DetectorFreeSfM suits a research group that needs to reproduce or extend the CVPR 2024 results, or a pipeline that has images with weak corners where a keypoint detector struggles, since the matching stage is the point of the release. It does not suit a production reconstruction service on modest hardware, because the documented training configuration uses eight GPUs by default and the evaluation path expects a CUDA device for the multi-view refinement. Before adopting, read INSTALL.md and follow DATASET_PREPARE.md to build the required folder names, and check the LICENSE file directly, because the repository carries a custom licence that GitHub cannot classify and the last push was on 2026-09-15 with no tagged releases.

Frequently asked questions

Official sources

1. Issues
2. README
3. zju3dv/DetectorFreeSfM on GitHub