torch-uncertainty declares two dependencies and expects you to bring PyTorch

Two required dependencies, and PyTorch is not one of them

The dependency list is the first surprising thing about this package, and it is a consequence of a deliberate decision.

Installation asks you to put the PyTorch version you want in your environment first, then install the package from PyPI:

pip install torch-uncertainty

The framework's own required dependencies are an array manipulation library and a plotting library. PyTorch itself is not among them.

That is unusual for a framework whose entire purpose is deep learning, and it is not an oversight. GPU wheels are tied to CUDA versions and compute capabilities, and a package that pinned torch would be forcing one of those choices onto every user. Handing that decision back is the reason the install has two steps.

The build backend is also unconventional: the project builds with a modern Python build backend rather than the traditional setuptools, and the manifest declares that backend alongside a narrow version range for it.

Supported Python versions run from 3.10 to just under 3.14, with each intermediate version named explicitly in the classifiers. Everything else arrives through extras: experiment tracking and dataset hosting libraries, image processing libraries for vision tasks, a scientific computing library for distribution work, a time-series library, and a catch-all group holding the scikit-learn-adjacent and Laplace-style pieces.

So the base install stays small and every capability you do not use stays uninstalled.

Four task families, and the same harness covers all of them

The API is multi-level, and the organising principle is that uncertainty should be measurable the same way regardless of what you are predicting.

Four task families are supported: classification, regression in both probabilistic and pointwise forms, segmentation, and pixelwise regression such as monocular depth estimation. Each gets uncertainty-aware training and evaluation routines on top of Lightning.

Above that sits a fully automated evaluation layer covering proper scores, calibration, selective classification, out-of-distribution detection and distribution shift metrics. The claim is that you get the whole set rather than picking one metric and arguing about it later.

Below that are layers, models, metrics and losses designed to drop into an existing network, so you can adopt one component without committing to the training loop.

And on top sits post-processing in the scikit-learn mould, temperature scaling and conformal prediction among them, which is where a trained model gets its probabilities fixed without retraining.

The community method list is long by design, and the project says so: it aims to include as many methods as possible and invites contributions. Deep ensembles and their variants dominate the list, alongside dropout-based sampling, variational approximations, checkpoint and snapshot ensembles, weight averaging, deep evidential regression, quantile regression, and test-time adaptation.

The framework holds official implementations of papers from several major conferences, and work built on it has appeared in tutorials at three computer vision conferences in 2024.

Selective classification under shift got its own metric family

The release history is a useful record of where the project's attention has gone, because the version summaries name their own subjects.

The most recent listed release is about regression uncertainty and metric robustness. The one before it names selective classification under distribution shift, an experiment tracking integration, and reworked tabular and segmentation metrics. Before that came a reworked methods API, better documentation, and entropy fixes.

That sequence suggests the recent work has been about making existing measurements trustworthy rather than adding more methods, which is the opposite of where a young research package usually spends its effort.

The selective classification with distribution shift family is the clearest example. Four metrics are named as a set, two area-under-risk style measures, a coverage at a fixed risk threshold, and a risk at a fixed coverage threshold. Reporting them together is the point: a single one of those numbers is easy to cherry-pick.

The calibration side follows the same pattern of multiplicity, with an expected calibration error alongside improved reliability diagrams, smoothed, adaptive and classwise variants, and a quantile calibration error for regression.

Out-of-distribution detection gets three standard measures, and the proper scores are the Brier score plus two negative log-likelihood variants.

Corrupted datasets are the input side of the same idea, with a set of modernised corruptions used to build shift benchmarks, hosted on the model hub rather than distributed in the package.

The contributor image permits root SSH login on purpose

A prebuilt container is provided for people running experiments on cloud GPU instances, alongside a Dockerfile for building your own. It is worth reading how it is put together, because it makes assumptions that would be wrong in production.

The base is a PyTorch runtime image with CUDA and cuDNN pinned. A build argument selects the dependency extra, so the same file produces a GPU or a CPU environment, and the package manager is installed by piping its install script into a shell.

The interesting parts are the SSH configuration and the exposed ports. The image enables root login over SSH and public key authentication, and exposes the SSH port alongside an experiment tracking UI port and a notebook port. That combination is entirely reasonable for a single-user experiment box you reach over a network, and entirely wrong for anything shared.

There is a quiet bug-avoidance note at the end. The workspace directory is where the code and dependencies live, but it is also the volume you will mount to attach your experiment files, and a mounted volume hides whatever was copied there during the build. So the entrypoint script is deliberately installed outside the workspace instead.

That is a small thing that would otherwise cost an afternoon: a container that mysteriously ignores your entrypoint because a volume is mounted over it.

Post-processing treats calibration as a separate, swappable step

The post-processing layer is where a trained model gets its confidence numbers fixed, and the range of methods is the point.

Scaling comes first, with temperature, vector and matrix scaling plus Dirichlet calibration, and then the binning family: histogram binning, isotonic regression, and Bayesian binning into quantiles.

Conformal prediction is included with two adaptive variants for classification and a conformal quantile regression variant for regression, so the coverage guarantee machinery is present rather than something you would have to implement.

Two methods are notable for what they claim. One is a direct epistemic uncertainty predictor that works by training an error predictor and using its output as a post-hoc epistemic score, rather than by modifying the model. The other wraps the Laplace approximation library rather than reimplementing it.

Monte Carlo batch normalisation sits in the same layer, which is a reminder that some Bayesian-flavoured methods are training-time choices rather than post-hoc ones.

The split between this layer and the augmentation layer is worth internalising. Augmentation methods here are mixup variants plus the corruption set, and they change what the model sees during training. Post-processing changes only what you do with the finished model, which is why it is the cheaper of the two to adopt and the easier to audit.

Editorial conclusion

torch-uncertainty suits research groups that want several uncertainty methods side by side under one evaluation harness rather than one bespoke pipeline, since having calibration, selective classification and distribution shift metrics already wired to the same trainers is the thing it saves you. It suits you less if you want a single opinionated method, because it is explicitly a collection rather than a recommendation. Before relying on a number, check which of the four tasks your problem actually is, and read the container notes if you plan to run on a cloud GPU instance, since the shipped image is built for experimentation rather than for keeping secrets.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. torch-uncertainty/torch-uncertainty on GitHub