TorchMetrics: distributed PyTorch metrics without the accumulation boilerplate

The problem TorchMetrics solves: metric state that survives batching and device splits

Compute accuracy on a single batch and you get a number that means nothing about your model. The useful number requires summing correct predictions and total predictions across every batch of the epoch, and on multi-GPU or multi-node runs it also requires combining partial counts across processes before dividing. Written by hand, that is a small pile of boilerplate repeated for every metric, and it is easy to get wrong in ways that do not raise an exception. A metric that resets at the wrong moment or averages per-batch ratios instead of computing a global ratio produces a plausible but incorrect number.

TorchMetrics targets that specific gap. The README describes the library as "a collection of 100+ PyTorch metrics implementations and an easy-to-use API to create custom metrics," and lists automatic accumulation over batches and automatic synchronization between multiple devices as the two headline behaviours. The audience is anyone training PyTorch models who needs evaluation numbers that hold up across a full dataset, and especially anyone running distributed training where the reduction step is not optional. PyTorch Lightning users get an additional layer: module metrics are placed on the correct device automatically and can be logged natively.

How the module metric mechanism works: state, update, compute

The central abstraction is the module metric, which holds internal metric states that the documentation compares to the parameters of a PyTorch module. You call the metric object on a batch, and it updates those states rather than returning a final answer each time. When you want the number for everything seen so far, you call compute().

That design has a direct consequence: calling the metric repeatedly is not idempotent. Each call folds the new predictions and targets into the accumulated state, so the same metric object used across an epoch gives you a running result, and the README's example prints per-batch accuracy and then a final accuracy over all data from the same object. Synchronization across devices is handled inside the metric, which is why the README states that module metric usage remains the same when using multiple GPUs or multiple nodes. The README also mentions metric arithmetic, meaning metric objects can participate in arithmetic operations.

The trade-off is that metric state is real state. It lives on a device, it must be reset between epochs if you want per-epoch numbers, and it interacts with checkpointing and with the distributed process group. None of that is free, and the documentation's framing of states as module-like parameters is the honest description: you are managing objects with lifecycle, not calling pure functions.

Installing TorchMetrics and running a first accuracy metric

The README gives the simple installation from PyPI. It pulls in the base requirements only; the specialized metrics for audio, image and text live behind extras.

pip install torchmetrics

The README also documents conda and uv as alternatives, and lists extras for specialized metrics. Install the one matching the domain you evaluate in, or the all extra to pull every domain at once.

pip install torchmetrics[image]

A first real use is the multiclass accuracy example from the README. You construct the metric with the task type and the number of classes, move it to the device you compute on, then call it once per batch and call compute() at the end. The printed per-batch values are the accuracy on that batch alone; the final value is accuracy over all ten batches.

import torch
import torchmetrics

metric = torchmetrics.classification.Accuracy(task="multiclass", num_classes=5)
device = "cuda" if torch.cuda.is_available() else "cpu"
metric.to(device)

for i in range(10):
    preds = torch.randn(10, 5).softmax(dim=-1).to(device)
    target = torch.randint(5, (10,)).to(device)
    acc = metric(preds, target)
    print(f"Accuracy on batch {i}: {acc}")

acc = metric.compute()
print(f"Accuracy on all data: {acc}")

Two details matter for a first run. The task argument is required in this example, so a metric constructed without it will not behave as the snippet shows. And because the metric accumulates, reusing metric across a second loop without resetting continues the accumulation rather than starting fresh.

Where TorchMetrics is the wrong tool

The library assumes tensors. If your evaluation data is in NumPy arrays, pandas frames, or a list of Python floats, scikit-learn's metric functions take that input directly and return a number, with no metric object, no device placement and no accumulation semantics to reason about. TorchMetrics is not a general-purpose metrics library with a PyTorch backend; it is a PyTorch-native metrics layer. Converting your data to tensors to compute a single ROC AUC once is work with no payoff.

The second boundary is the extras split. The base install does not include the dependencies for audio, image and text metrics, and the README directs you to install those extras explicitly. A user who installs only torchmetrics and then imports an image metric will hit a missing dependency rather than a working metric. The README does not document what happens beyond that, so treat the extras list as part of your dependency planning rather than an optional detail.

The third boundary is state lifecycle. Because module metrics accumulate, a metric object that is accidentally shared between training and validation, or between two evaluation loops, produces numbers that mix both. The README's example is a single loop for a reason. The README does not document rollback or state-inspection behaviour, so if your workflow needs to snapshot and restore metric state mid-run, verify that yourself before relying on it.

TorchMetrics against TorchEval and scikit-learn metrics

The most direct alternative in the PyTorch ecosystem is TorchEval, which also provides metrics for PyTorch and is listed among the related searches for this project. Both are PyTorch-native, so the difference is not tensor support but API shape and scope. TorchMetrics is built around the module metric with internal state, an explicit compute() call, and a documented path to writing custom metrics by subclassing. If your mental model is "metric object that I feed batches and then query," TorchMetrics matches it. TorchEval is the other PyTorch-native option to evaluate when the metric you need is not in TorchMetrics' built-in list or when its API fits your code better.

The scikit-learn comparison is a different axis. scikit-learn metrics are functions over arrays: you pass the full set of predictions and targets and get a value back. There is no state, no device, and no distributed reduction. That is simpler and it is the right choice for offline analysis on a single machine. TorchMetrics exists precisely because that model breaks down when the data does not fit in one process or one device. Choosing between them is mostly a question of where your predictions live when you need the number: if they are already a tensor on a GPU inside a training loop, TorchMetrics removes the transfer and the manual reduction; if they are a NumPy array in a notebook, scikit-learn is less machinery.

Releases, maintenance and what the Apache-2.0 licence means here

The repository is not archived and the last push was on 2026-09-09, which is recent. The release cadence visible in the release history is a major-minor line with patch releases: v1.9.0 on 2026-03-09, v1.8.2 on 2025-09-03, and v1.8.1 on 2025-08-07. That pattern suggests the project ships minor versions roughly twice a year and patches in between, so pinning a minor version and taking patches is a workable upgrade posture. Budget for the minor-version upgrade as the real cost, not the patch.

The licence is Apache-2.0, stated in the README badge and present as a LICENSE file at the repository root. Apache-2.0 is a permissive licence with an explicit patent grant, and it permits use in closed-source products. It also carries notice and attribution obligations. This is a description of the licence identifier, not legal advice; read the LICENSE file and your own counsel's guidance before shipping.

One maintenance note that has nothing to do with the code: the README's top section is an advertisement for Lightning Cloud with pricing and product links. The library is Apache-2.0 and installs from PyPI, conda-forge and source, so the commercial material is separable from the software. It does mean the README is not purely documentation, and you should read the docs site rather than the README for reference material.

Who should adopt TorchMetrics, and what to check before you do

Adopt it if you run distributed training and currently maintain your own running totals, or if you want the same metric object to work on CPU, one GPU and many GPUs without rewriting the evaluation path. The README's claim that module metric usage remains the same across multi-GPU and multi-node runs is the core selling point, and it is the reason the accumulation and synchronization live inside the metric rather than in your loop.

Skip it if your evaluation is a one-shot computation over arrays, or if you are not already in PyTorch. The metric object, the device placement and the compute() call are overhead you would be paying for nothing.

Before adopting, check three concrete things. First, confirm the metric you need is among the built-ins, since the library's value is the list, not the base class alone. Second, install the matching extra if your metric is in audio, image or text, because the base package does not carry those dependencies. Third, verify the accumulation behaviour against your loop structure: call compute() when you want the number, and make sure the metric object is not shared between loops you intend to keep separate.

Editorial conclusion

Adopt TorchMetrics if you train or evaluate on more than one device and you currently hand-roll running totals, or if you want one metric interface across training and validation loops. Do not adopt it for a single-process script that computes one number once, and do not treat it as a replacement for scikit-learn metrics on NumPy arrays that never touch a GPU. Before committing, verify that the metric you need is in the built-in list, that its required extra is installed (pip install torchmetrics[image] and friends), and that your metric's update and compute semantics match how your batches are shaped, since TorchMetrics accumulates state across calls by design.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Lightning-AI/torchmetrics on GitHub
3. Project website
4. README
5. Releases