GeoCalib: single-image camera calibration as an optimization problem, not a regression

The problem GeoCalib targets: intrinsics and gravity from a single frame

Classical camera calibration needs a target, multiple views, or both. You photograph a checkerboard from several angles, detect corners, and solve. That workflow assumes you control the capture. A large amount of real imagery does not offer that: a photo pulled from a dataset, a frame from an archive video, a picture taken before anyone thought about calibration. GeoCalib addresses that case. The README states that it estimates the camera intrinsics and gravity direction from a single image only, and that it does so by combining geometric optimization with deep learning. The gravity estimate is the part that distinguishes it from a plain focal-length regressor. Knowing which way is down in the camera frame is what lets the same code produce a horizon line, up-vectors and an undistorted view in the interactive demo. The audience is narrower than the topic list suggests. This is for people doing structure from motion or visual localization who have imagery with unknown or stale intrinsics, and for researchers who want a differentiable calibration block they can plug into a larger pipeline. It is not a tool for a photographer who wants a lens profile.

A network proposes, a geometric solver disposes

The README describes the method as geometric optimization combined with deep learning, and the repository layout carries that through. The inference package is geocalib, and geocalib/camera.py holds a Camera object that the README says can be extended to add new models. The camera model is therefore a first-class object in the code, not a fixed output head. The network supplies the quantities the optimizer needs, and the solver enforces the geometry that relates them, which is why the same weights can be run under pinhole, simple_radial, radial or simple_divisional without retraining. That is a different bet from a network that regresses intrinsics directly. A regression head has no internal notion of consistency between the horizon and the focal length; the optimization does. The practical consequence is visible in the API: calibrate takes a camera_model argument and a priors argument, so the solver can be run with one unknown held fixed while the rest is estimated. The README also exposes confidence outputs. The interactive demo has a key to show the confidence heatmap for the up-vectors and latitudes, which implies per-pixel uncertainty is produced rather than a single scalar. Whether that confidence is calibrated is not something the README claims, and I have not verified it.

Installing geocalib and running the first calibration

The README gives two install paths, both editable installs from the repository. The first is a clone followed by python -m pip install -e . from the repository root. The second skips the clone and installs straight from git with python -m pip install -e "git+https://github.com/cvg/GeoCalib#egg=geocalib". Python 3.9 or newer is required, and the README describes the inference package as needing only minimal dependencies. The minimal example constructs GeoCalib(), moves it to a CUDA device when one is available and otherwise to CPU, loads an image with model.load_image into a tensor in the range [0, 1] with shape [C, H, W], and calls model.calibrate(image). The returned dictionary has camera and gravity entries. There is also a torch hub path, model = torch.hub.load("cvg/GeoCalib", "GeoCalib", trust_repo=True), which is the shortest route to a working model if you already have torch installed. For a live check, python -m geocalib.interactive_demo --camera_id 0 opens a window on the camera feed. The demo keys are worth reading before you run it: h for the horizon line, u for up-vectors, l for the latitude heatmap, c for the confidence heatmap, d for the undistorted image, g for a virtual grid, b for a virtual box, 1/2/3 to switch between pinhole, simple radial and simple divisional, and q to quit. If --camera_id is omitted, the demo asks for a droidcam IP address instead of using a local camera.

Camera models, priors and the principal point you cannot move

Four camera models are supported through the camera_model parameter. pinhole is the default and models fx and fy with no distortion. simple_radial adds one polynomial distortion parameter, k1, for weak distortion. radial adds k1 and k2 for stronger distortion, and the README credits that model to a pull request from RuibinMa. simple_divisional handles strong fisheye distortion with a single parameter, following Fitzgibbon's 2001 CVPR paper on simultaneous linear estimation of multiple view geometry and lens distortion. The weights matter here. The README says the default weights are optimized for pinhole images and that you should construct the model with weights="distorted" when you intend to use a distortion model, then pass camera_model="simple_radial" or one of the others to calibrate. Getting that pairing wrong is the most likely first mistake. The constraint that will decide adoption for many readers is stated plainly: the principal point is assumed to be at the center of the image and is not optimized. If your principal point is offset, no camera_model setting fixes it. Priors are the escape hatch for partial knowledge. Passing priors={"focal": focal_length_tensor} holds the focal length fixed, and priors={"gravity": gravity_direction_tensor} holds gravity fixed. The README does not describe what happens when a prior is inconsistent with the image, nor how the two priors interact if both are supplied.

Batching, shared intrinsics and rigid multi-camera rigs

Single-image calibration ignores the fact that most real footage comes from one camera. GeoCalib handles that with two arguments. Passing a list of tensors, each shaped [C, H, W], to calibrate with shared_intrinsics=True estimates one intrinsic set across the whole batch. For a rig of rigidly mounted cameras with known relative rotations, you pass camera_R_rig, described as the rotation from the rig frame to camera i, and the model estimates a single shared gravity direction for the entire rig. Adding shared_intrinsics=True on top of that is recommended when the cameras are identical. This is the most interesting part of the API, because it is where the geometric formulation pays off. A per-image regressor would produce a different gravity vector for each camera in the rig and leave you to reconcile them. Here the constraint is built into the solve. The cost is that you must know the rig rotations accurately. The README gives no guidance on how sensitive the shared-gravity estimate is to error in camera_R_rig, and it does not say what happens if the rotations are wrong. Treat that as an open question to test on your own rig rather than an assumption to trust.

Evaluation lives in siclib, and so does the training code

The geocalib package is inference only. Evaluation and training sit in a second library, siclib, installed separately with python -m pip install -e siclib from inside the repository. The README describes siclib as a single-image calibration library and says evaluation commands write results to outputs/results/. The worked example is LaMAR, with the note that running the evaluation commands downloads the dataset to data/lamar2k and takes around 400 MB of disk space. The command for GeoCalib trained on OpenPano is python -m siclib.eval.lamar2k --conf geocalib-pinhole --tag geocalib --overwrite. A second block evaluates DeepCalib trained on the same OpenPano dataset, and the README excerpt cuts off mid-command, so the exact invocation for that baseline is not fully recoverable from the material available here. The training set is OpenPano, distributed separately with download instructions in the repository. The practical split is clean: if you only want to calibrate images, install geocalib and ignore siclib. If you want to reproduce the paper's numbers or fine-tune on your own data, you need siclib, the LaMAR download and the OpenPano training set, and the disk and setup cost rises accordingly.

Where GeoCalib is the wrong tool

The fixed principal point is the first hard boundary. Any application that needs cx and cy as free parameters cannot be served by this code as documented. The second boundary is the input itself. The method infers geometry from image content, so it depends on the scene containing the cues that reveal it. The README does not state what those cues are or how the method behaves on images that lack them, and it publishes no failure analysis. That silence is worth noting: a single-image calibrator has no way to tell you that it is wrong, only a confidence output whose reliability is not documented. The third boundary is the runtime. This is a PyTorch package with a geometric solver on top, so the deployment surface is Python plus torch. There is no mention of an ONNX export, a C++ runtime or a mobile build. If your target is an embedded device or a browser, you are looking at a port, not an install. The fourth is the release cadence. There is a single tagged release, v1.0 from September 2024, while the last push to the default branch is dated August 2026. The code is moving; the version tags are not. Pinning to v1.0 gives you a stable point that may be well behind main, and installing from git gives you the current code with no release boundary. The README does not describe a deprecation policy or a compatibility guarantee between the two.

What GeoCalib is not: the DeepCalib comparison in the repository

The evaluation section names DeepCalib as a baseline trained on the same OpenPano dataset, which makes the contrast concrete rather than rhetorical. DeepCalib is a network that predicts calibration parameters directly from the image. GeoCalib keeps a network in the loop but puts a geometric optimizer between the prediction and the final answer, which is what allows priors, shared intrinsics across a batch and a single gravity direction across a rig. The difference shows up in what you can express. With a direct regressor, fixing a known focal length means retraining or post-hoc blending. With GeoCalib it is a priors dictionary entry. The trade is that the optimizer adds a component that can fail to converge or converge to a poor solution, and the README does not document convergence behaviour or iteration counts. A second comparison is implicit in the camera model list. Tools that assume a pinhole camera and a single focal length are simpler to deploy but produce visibly wrong results on fisheye footage. GeoCalib's simple_divisional model exists for exactly that case, and the weights="distorted" flag is what makes it usable. If your imagery is already rectified and pinhole, the extra models buy you nothing and the default weights are the right choice.

Licence, maintenance and what to verify before you depend on it

GeoCalib is Apache-2.0, which permits commercial use and modification provided the licence and notices are preserved, and it includes a patent grant. That is a permissive licence, but it is not legal advice and the obligations around attribution and modified-file notices should be checked against your own distribution model. The dependency situation is the maintenance question. The README says the inference package needs only minimal dependencies and Python 3.9 or newer, but torch is one of them and torch is a large dependency with its own release cycle and CUDA compatibility matrix. The evaluation path adds siclib, a LaMAR download of roughly 400 MB and the OpenPano training set, so the research workflow has a materially higher setup cost than inference. Upgrades are the other cost. With one tagged release against an actively pushed main branch, you are choosing between a stable point and current code, and the README offers no upgrade notes. Verify three things on your own data before you build on this: that your principal point really is near the image center, since that assumption is not relaxed anywhere in the API; that the weights you load match the camera_model you pass, because the default weights target pinhole; and that the gravity output is stable across images from the same camera, which is the quickest way to find out whether the scene cues your imagery provides are sufficient.

Editorial conclusion

Adopt GeoCalib if you have a single image or a batch from one camera and you want focal length, distortion and gravity without a calibration target, and if you can accept a principal point pinned to the image center. Do not adopt it if you need a full intrinsic matrix with a free principal point, or if you cannot install PyTorch at all, since the package is torch-based and the README lists no ONNX or C++ export path. Before committing, run the inference example on your own images, then run the LaMAR evaluation command with --conf geocalib-pinhole to see the numbers on the benchmark the authors chose, and check that your camera fits one of the four supported models.

Official sources

1. cvg/GeoCalib on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases