timm (PyTorch Image Models): feature extraction, pruning, and the training scripts behind the model zoo

Every model now exposes forward_intermediates and prune_intermediate_layers

The October 24, 2026 entry in What's New states that `forward_intermediates()` and `prune_intermediate_layers()` are supported by every model. Support was newly added for CSPNet, NFNet, DLA, DenseNet, CoaT, ConViT, ConvMixer, CrossViT, HRNet, Inception-v3, Inception-ResNet-v2, NASNet, PNASNet, SelecSLS, SENet, Sequencer2d, VGG, Visformer, Xception and Aligned Xception. NaFlexViT and MobileNetV5 encoders gained `prune_intermediate_layers()`, and CoaT, ConViT, ConvMixer, CrossViT and Visformer gained `features_only` support. These are main-branch release notes; confirm the installed version contains the change and test the selected model's feature shapes, stages and downstream integration.

prune_head=True leaves unused parameters on CoaT, CrossViT and HRNet

With `prune_head=True`, pruning removes the whole head: head norm, pre-logits layers and attention pooling. The project says this avoids unused parameters in the stated pruning paths, which can allow DistributedDataParallel without `find_unused_parameters`; test the actual model and training path. There is a stated exception. CoaT, CrossViT and HRNet still leave unused parameters when you take only some of their parallel scales or branches. If you train one of those three under DDP with a partial set of branches, you still need `find_unused_parameters`, so the DDP simplification promised by the pruning change does not apply to you. Pass `prune_head=False` if you want to keep the head. Pruning a hierarchical model to an earlier stage also removes the final norm and head sized for the last stage, so `forward_features()` ends at the last kept feature and `num_features` and `head_hidden_size` match it.

Pool types and constructor arguments that used to pass silently now raise

Two consistency changes will surface in older code. First, pool types a model cannot apply now raise at creation or at `reset_classifier()` instead of being ignored or failing later in forward. Second, the October 13 entry says unknown constructor arguments raise `TypeError` everywhere. A script that passed a stray keyword to `create_model` and never noticed will stop at model construction after an upgrade. On the positive side, `head_hidden_size` now reports the actual pooled pre-logits width in all models, including concat pooling (`catavgmax`) and after `reset_classifier()`, and `reset_classifier()` keeps the current pooling. Code that sized a linear probe from `head_hidden_size` can trust the number instead of special casing families.

train.py resume, milestone snapshots and sweep.py

The repository root holds the scripts: `train.py`, `validate.py`, `inference.py`, `benchmark.py`, `onnx_export.py`, `onnx_validate.py`, `avg_checkpoints.py`, `clean_checkpoint.py`, `distributed_train.sh` and the newer `sweep.py`. In `train.py`, `--resume auto` resumes in place from the experiment's own `last.pth.tar` with checkpoint history, `model_best` and `summary.csv` handled as described in the notes. Exact reproducibility also depends on data, random state and the environment. `--stop-epoch N` stops without changing the schedule, and `--save-epochs N [N ...]` keeps `milestone-N.pth.tar` snapshots. YAML `--config` values are validated against the script's arguments. `sweep.py` runs grid or random search over any `train.py` argument on a pool of local GPUs and is resumable. Its WSD mode trains one stable trunk per hyperparameter combination and branches cooldowns from milestone checkpoints.

The plateau scheduler cannot resume exactly and is deprecated

`PlateauLRScheduler` (`--sched plateau`) is deprecated. timm schedulers are stateless functions of the epoch or update count. The metric driven plateau schedule never fit that model, and its state is not checkpointed, so resuming a plateau run was never exact. The practical consequence: if an existing recipe uses `--sched plateau` and gets interrupted, the resumed learning-rate path may differ from an uninterrupted run, so verify it before comparing results. The project's direction is `cosine` or the new WSD scheduler (`--sched wsd` with `--lr-decay-fn` and `--lr-stable-timescale`). Moving a recipe off plateau changes its learning rate curve, so expect to retune rather than reuse old results.

Foreach optimizers chunk by L2 cache, and EMA precision changes with dtype

The October 9 entry adds foreach (multi-tensor) implementations for RMSpropTF, LaProp, LAMB, LARS and AdaBelief. Foreach steps are chunked by device L2 cache size, and `TIMM_FOREACH_CHUNK_SIZE` overrides that. AdamWLegacy and NAdamW gain RMS update clipping via `clipping_threshold`, and NovoGrad gets decoupled and corrected weight decay variants named `novogradw` and `novogradc`. Fixes cover FP16 update stalls at small learning rates and low precision state handling for Kron, AdaBelief and NovoGrad, and `p.grad` is no longer modified in place. For model EMA, ModelEmaV3 uses stochastic rounding by default for bfloat16 EMAs, while float16 models keep a float32 EMA, so a float16 model and its EMA copy do not share a dtype.

Multi-label training reads targets three ways but only through four readers

Since September 25, 2026, `train.py` and `validate.py` accept `--task multilabel`, trained with BCE through `MultiLabelClassificationTask`. Targets can be lists of class indices, dense multi-hot vectors with `--target-format multihot`, or one binary field per class with `--target-key a,b,c`, using `/` paths into nested fields. Those formats are supported by the `hfds`, `hfids`, `tfds` and `wds` readers. If your multi-label data sits in a plain image folder layout outside those readers, the new flags do not reach it, and you will need to convert the dataset into one of the supported formats before the multi-label path applies.

One package with five runtime dependencies and a Python 3.8 floor

pyproject.toml names the package `timm`, builds it with `pdm-backend`, and reads the version from `timm/version.py`. It declares `requires-python = ">=3.8"`, with classifiers listing Python 3.8 through 3.12, and dependencies on `torch`, `torchvision`, `pyyaml`, `huggingface_hub` and `safetensors`. requirements.txt is stricter: `torch>=1.7`, `huggingface_hub>=0.17.0`, `safetensors>=0.2`, and it also lists `numpy`. The project's November 3, 2026 notes announce release 1.0.31; consult current releases and package metadata for later changes. What's New entries are dated October, so confirm the installed version actually contains a change before writing code against it. Documentation lives at huggingface.co/docs/timm, including a new Model API page.

Editorial conclusion

timm offers a broad collection of image models, pretrained weights and training utilities. Features and compatibility vary by version, architecture and environment: check the installed release and its documentation, and test feature extraction, pruning and checkpoint resume on the selected model before relying on them. When changing a training schedule or upgrading a recipe, validate the resulting learning-rate trajectory and results.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes