TorchXRayVision: One Preprocessing Chain and One Label Set Across Public Chest X-ray Datasets

The problem is label harmonisation, not model architecture

A researcher with a chest radiograph cohort faces two separate chores before any modelling happens. The first is that public datasets disagree about what a label means: one may annotate pneumonia, another Lung Opacity, another a set of findings that only partly overlaps. The second is that each dataset ships its own loading conventions, image encodings and metadata fields, so code written against one does not run against another. TorchXRayVision addresses both by exposing datasets through a common interface and a common preprocessing chain, with the stated goal that datasets can be swapped out with a single line of code. The audience is narrow and identifiable: researchers who want a pretrained baseline or a feature extractor rather than a model trained from scratch, and researchers who need to evaluate a method across several external datasets to study generalisation. The README frames the tradeoff directly, calling it a waste of time to train from scratch for clinical questions and describing metadata variation as the obstacle to multi-dataset evaluation. Nothing in the material suggests this is a deployment library. It is infrastructure for experiments.

The preprocessing chain is the part that actually unifies the datasets

The mechanism that makes a single model usable across sources is a fixed input pipeline. The README example reads an 8-bit image, calls xrv.datasets.normalize(img, 255) to map it into a [-1024, 1024] range, collapses the colour channel with img.mean(2)[None, ...], then applies torchvision.transforms.Compose([xrv.datasets.XRayCenterCrop(), xrv.datasets.XRayResizer(224)]). Only after that does the tensor reach the model. Every dataset in the library is expected to pass through the same normalisation and the same crop-and-resize pair, which is why a checkpoint trained on one cohort can be pointed at another. This is also the constraint: the model sees a 224x224 centre crop, so anatomy outside that crop is discarded before inference, and the resnet50-res512-all weights exist as the higher-resolution alternative for cases where that matters. The library does not describe per-dataset intensity harmonisation beyond the normalise call, so if your source images are not 8-bit the responsibility for putting them into the expected range sits with your own loader code.

Eighteen outputs, and only some of them are trained

This is the detail most likely to cause a silent error. Each pretrained model exposes 18 outputs, and the README states plainly that only the all weights have every output trained. For the other checkpoints, some targets are not trained and will predict randomly because they do not exist in the training dataset. The documentation points to the pathologies field on the dataset corresponding to the weights as the authoritative list of valid outputs. The available 224x224 checkpoints are densenet121-res224-all, -rsna (RSNA Pneumonia Challenge), -nih (NIH chest X-ray8), -pc (PadChest, University of Alicante), -chex (CheXpert, Stanford), and two MIMIC-CXR variants, -mimic_nb and -mimic_ch. There is also resnet50-res512-all at 512x512. The output dictionary in the README example includes labels such as Hernia and Fracture alongside Lung Opacity and Enlarged Cardiomediastinum, which is the union the all model was trained against. If you load the RSNA checkpoint and report a Hernia probability, you are reading a number the model never learned to produce. The library gives you the field to check. It does not stop you from skipping the check.

Beyond classifiers: autoencoders, segmentation and third-party baselines

The library is not only a classifier zoo. xrv.autoencoders.ResNetAE(weights="101-elastic") exposes encode and decode, trained on PadChest, NIH, CheXpert and MIMIC, which makes it usable as a representation extractor without a label head. For spatial output there is xrv.baseline_models.chestx_det.PSPNet(), whose output shape is documented as [1, 14, 512, 512] over 14 anatomical targets including Left Clavicle, Right Clavicle, Left Lung, Right Lung, Heart, Aorta, Mediastinum and Spine. The library also re-exposes models from other groups rather than only its own: xrv.baseline_models.jfhealthcare.DenseNet() for the CheXpert competition, xrv.baseline_models.chexpert.DenseNet(weights_zip="chexpert_weights.zip") for the official Stanford model, xrv.baseline_models.emory_hiti.RaceModel() with targets ["Asian", "Black", "White"], and xrv.baseline_models.riken.AgeModel(). Those last two are not pathology classifiers at all, which is a useful signal about what the project considers in scope: any model whose output can be evaluated on chest radiographs. It also means the baseline_models namespace mixes maintained checkpoints with wrappers around external work, and the documentation does not say which are actively updated.

Installing it and running the documented example

Installation is a single pip command: pip install torchxrayvision. The README's inline example then imports torchxrayvision as xrv, skimage, torch and torchvision, loads an image with skimage.io.imread, normalises it, applies the transform, converts with torch.from_numpy, loads xrv.models.DenseNet(weights="densenet121-res224-all"), and calls the model with img[None, ...]. The result is zipped against model.pathologies to produce a label-to-score dictionary. The alternative entry point is a script, process_image.py, invoked as python3 process_image.py ../tests/00000001_000.png -resize, which prints a preds dictionary with the same 18 keys. Note that the two documented runs produce different numbers for the same nominal label set, which is expected because they are different images, and the README presents them as illustrative output rather than as a benchmark. For the segmentation path the README points to scripts/segmentation.ipynb and for the model inventory to scripts/xray_models.ipynb. The repository's primary language is listed as Jupyter Notebook, so the notebooks are the intended reading path, not an afterthought.

Where it stops being the right tool

The first limitation is stated by the project itself and is the one that matters most: untrained heads on non-all checkpoints return random values. A pipeline that iterates over model.pathologies rather than the dataset-specific pathologies field will emit numbers that look like probabilities and are not. The second is resolution. The 224x224 centre crop is the default path for every DenseNet checkpoint listed, and fine findings such as Nodule or Fracture are exactly the kind of target where a centre crop at that size is a questionable input. The third is that this is a research library. Nothing in the supplied material describes calibration, decision thresholds, or clinical validation, and the output values are raw model scores, not diagnoses. The fourth is dataset access. The library provides loaders for public datasets, but the material does not describe download automation, credential requirements or per-dataset terms, and the repository's licence is recorded as NOASSERTION, which means the code licence is not declared in a form the tooling recognises. Anyone intending to redistribute derived models or data should read the actual licence files and the individual dataset agreements rather than infer terms from the GitHub metadata.

How it differs from training your own model on one dataset

The obvious alternative is to pick a single dataset, use its own loader, and train or fine-tune a torchvision model against its label definitions. That approach gives you full control over resolution, augmentation and the label semantics, and it avoids the union label space entirely. The difference in approach is concrete: a single-dataset pipeline optimises for performance on that dataset's distribution, while TorchXRayVision optimises for the ability to point the same code at NIH, PadChest, CheXpert, MIMIC-CXR or RSNA and get comparable outputs. The README describes the intended use case as merging and filtering datasets to construct specific distributional shifts for studying generalisation, which is a task a single-dataset pipeline cannot do without rewriting the loader layer. The cost of the library's approach is the 18-output union and the fixed preprocessing: you inherit label definitions you did not choose and an input size you did not choose. If your question is about one cohort and one finding, the library is mostly overhead. If your question is about whether a method transfers, it removes the loader work that would otherwise dominate the project.

Version cadence and what that implies for pinning

The release history shows 1.4.0 in September 2025, 1.5.2 in June 2026 and 1.5.4 in August 2026, with the last push on the main branch on the same day as the 1.5.4 tag. The cadence is irregular rather than continuous, and the minor-version jumps between 1.4.0 and 1.5.x suggest interface changes are possible across those boundaries. For a library whose main value is a stable preprocessing contract, that is worth noting: the transform classes and the normalize function are the load-bearing parts of any downstream pipeline, and a change to either invalidates cached features. Pinning an exact version in your requirements file is the cheap insurance, and it also makes the checkpoint names reproducible, since weights are selected by string such as densenet121-res224-all rather than by a content hash. The material does not describe a deprecation policy or a compatibility guarantee for the weights strings, so treat them as part of your pinned surface. Maintenance effort is otherwise low: there is no server component, no database and no service to operate, only a Python dependency and whatever disk the datasets and checkpoints occupy.

Editorial conclusion

Adopt it if you need a feature extractor or a baseline over a public chest radiograph cohort and can accept the label space the checkpoints were trained on; skip it if you need a single clinical decision with a calibrated probability, since the README itself warns that untrained heads predict randomly. Before anything else, print the pathologies field of the dataset matching your weights and confirm every target you plan to report on appears there, then check the licence terms of the underlying datasets you download, because the repository carries a NOASSERTION licence identifier and the data terms are separate from the code.

Official sources

1. Issues
2. mlmed/torchxrayvision on GitHub
3. Project website
4. README
5. Releases