TorchMetrics: Stateful Metric Accumulation for Distributed PyTorch
Machine learning metrics for distributed, scalable PyTorch applications.
At a glance
- What is it?
- TorchMetrics wraps 100+ metric implementations in a torch.nn.Module interface so that accumulation across batches and synchronization across devices happen without hand-written reduction code. It is the right tool when metric state must survive sharding; it is the wrong tool when a single-process, single-batch score is all you need.
- Who is it for?
- Adopt TorchMetrics if your evaluation loop spans multiple batches or multiple devices and you are currently writing your own running-total logic. Do not adopt it for one-shot scoring of an in-memory NumPy array, or if you cannot install the extra dependency groups that metrics such as torchmetrics[audio] and torchmetrics[image] require.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The Problem TorchMetrics Targets: Metric State That Outlives a Batch
A training loop that reports accuracy usually starts with something small: compare predictions to targets, average, print. That works until the evaluation set no longer fits in one forward pass. At that point the average has to be carried across batches, and if the model is sharded across processes, each process only sees a slice of the data. The naive fix is a running sum divided by a running count, written by hand in every project, with a different bug in each one. TorchMetrics replaces that hand-written bookkeeping with a metric object that owns its own state. The README describes the module-based metrics as containing "internal metric states (similar to the parameters of the PyTorch module) that automate accumulation and synchronization across devices." That is the whole pitch, and it is a narrow one. The library is for engineers who already have a PyTorch training or evaluation loop and need the reported number to be correct when the loop is distributed. It is not a general statistics library, and it does not try to be one.
How Module Metrics Accumulate and Synchronize
A TorchMetrics metric is a subclass of torch.nn.Module. That inheritance is the mechanism, not a stylistic choice. Because the metric is a module, calling .to(device) moves its internal state to the same device as the model, and the state participates in the module lifecycle the same way parameters do. The README's single-device example shows the intended call pattern: construct the metric, move it, call it once per batch, then call compute() once at the end. The per-batch call returns the metric restricted to that batch. The final compute() returns the metric over everything the object has seen. Between those two calls, the metric has been updating internal state rather than recomputing from stored predictions, which is what keeps memory bounded on a large evaluation set.
Synchronization is the second half. Under DistributedDataParallel, each rank accumulates its own partial state. The library handles the cross-device reduction so that compute() returns a value over the full dataset rather than one rank's shard. The README states that module metric usage "remains the same when using multiple GPUs or multiple nodes," and the collapsed DDP example in the README is built on torch.distributed, torch.multiprocessing and DistributedDataParallel rather than on any Lightning-specific API. The README also notes that metric arithmetic is supported, which means metric objects can be combined in expressions instead of being reduced by hand. What the supplied material does not specify is the reduction strategy used for each individual metric, and that detail matters: a metric averaged across ranks and a metric computed from globally pooled state are not the same number. Verify the semantics of the metric you pick before trusting a multi-node result.
Installation and the Optional Dependency Groups
The base install is one command:
pip install torchmetrics
The README lists conda and uv as alternatives:
conda install -c conda-forge torchmetrics uv add torchmetrics
Specialized metrics are split into extras, and this split is the part most people miss. The README gives four:
pip install torchmetrics[audio] pip install torchmetrics[image] pip install torchmetrics[text] pip install torchmetrics[all]
A plain pip install torchmetrics does not pull in whatever the audio, image or text metrics depend on. If you import a metric from one of those families without the matching extra, the failure happens at import time, not at install time. The README also documents installing from source, either from the release/stable branch or from the master archive, and warns nothing about the difference in stability between the two. If you install from master you are tracking unreleased code; the release notes show v1.9.0, v1.8.2 and v1.8.1 as the tagged versions, so pinning to a released version is the safer default for a project that other people have to reproduce.
Where the Module Abstraction Gets in the Way
The stateful design has a cost that the README does not discuss. A metric object is sticky: once you have called it on a batch, its state reflects that batch forever, and the only way to start over is to construct a new instance or explicitly reset it. In a loop that evaluates several splits, or that reuses one metric object across epochs, forgetting the reset silently mixes data from different runs into one number. The failure is quiet. You get a plausible float, not an exception.
The second constraint is shape discipline. Metrics are typed by task and by input format. The README example constructs torchmetrics.classification.Accuracy(task="multiclass", num_classes=5) and feeds it softmax probabilities of shape (10, 5) alongside integer targets of shape (10,). A logits tensor passed where probabilities are expected, or a mismatched num_classes, produces an error or a meaningless score depending on the metric. The library cannot infer your task from the tensor.
The third case is the wrong-tool case. If your entire evaluation set fits in memory and you score it once, a metric object buys you nothing over a direct call to the underlying computation. The accumulation machinery is overhead you are paying for a loop that runs one iteration. Similarly, if your evaluation data lives in pandas or NumPy and never touches a GPU, the distributed-synchronization half of the library is dead weight.
TorchMetrics versus scikit-learn Metrics
The closest comparison for the non-distributed case is sklearn.metrics. The difference is architectural, not a matter of which has more functions. scikit-learn metrics are functions: you pass arrays in, you get a number out, and the arrays must already be complete. There is no state, no device placement and no reduction across processes, because scikit-learn assumes the whole dataset is in memory in one process. TorchMetrics inverts that. The metric is an object that lives alongside the model, consumes batches as they are produced, and only materializes the final number when compute() is called.
That inversion is what makes the library usable inside a training step. You cannot call a scikit-learn function on a tensor that is still on the GPU without detaching and moving it to host memory, which forces a synchronization point in the middle of your loop. TorchMetrics keeps the computation on-device and defers the result. The trade is that you give up the immediacy of a pure function: the answer is only correct after compute(), and only if nothing else has touched the object's state in between. If your workflow is offline analysis of a finished prediction file, scikit-learn is the simpler and more predictable choice. If your workflow is a live loop over sharded batches, TorchMetrics is solving a problem scikit-learn does not address at all.
Maintenance Cost and Licence Terms
The release cadence visible in the material is roughly two minor or patch releases per year in the recent window, with v1.9.0 in March 2026, v1.8.2 in September 2025 and v1.8.1 in August 2025. That is a moderate pace: frequent enough that the project is maintained, slow enough that a pinned version will not be invalidated every month. The upgrade cost is concentrated in the same place as the install cost. Because specialized metrics live behind extras, a version bump can change which dependencies those extras resolve to, so an upgrade should be checked against the metrics you actually import rather than against the base package alone.
The licence is Apache-2.0, which the README states explicitly and which the repository's LICENSE file carries. Apache-2.0 is a permissive licence with an explicit patent grant and a requirement to preserve notices when redistributing. If you vendor the library into a product, that notice obligation applies to you; if you merely depend on it, the practical burden is the same as any other permissive dependency. This is a description of the terms, not legal advice. Confirm the obligations against your own distribution model with whoever handles that for your organisation.
Editorial conclusion
Adopt TorchMetrics if your evaluation loop spans multiple batches or multiple devices and you are currently writing your own running-total logic. Do not adopt it for one-shot scoring of an in-memory NumPy array, or if you cannot install the extra dependency groups that metrics such as torchmetrics[audio] and torchmetrics[image] require. Before committing, verify that the specific metric you need exists in the 100+ built-in collection, confirm its expected input shape against your model's output, and check that your torchmetrics version matches the one your training framework pins.
Community notes