pytorch-optimizer: a loader-based collection of 100+ PyTorch optimizers, schedulers and losses
optimizer & lr scheduler & loss function collections in PyTorch
At a glance
- What is it?
- kozistr/pytorch_optimizer bundles research optimizers, LR schedulers and loss functions behind one import surface with name-based loading and optional Lookahead or Gradient Centralization wrapping. The convenience is real; the trade-off is that you inherit a large dependency surface and a versioning cadence tied to upstream research rather than to PyTorch itself.
- Who is it for?
- Adopt pytorch-optimizer if you are running optimizer comparisons and want AdaBelief, Ranger21, Muon or Scion reachable through one import and one create_optimizer call instead of a folder of vendored research scripts. Do not adopt it if you have settled on a single optimizer that torch.optim already ships, because you would be adding a large package for no behavioural gain.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 7 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem: research optimizers arrive as loose scripts, not installable modules
Most optimizer papers ship as a single file in a repository attached to the arXiv listing, written against whatever PyTorch version the authors had, with no test suite and no packaging. If you want to compare AdaBelief against AdaBound against Ranger21 on your own model, you end up copying three files into your tree, reconciling three different parameter-group conventions, and hoping none of them break when you upgrade PyTorch. pytorch-optimizer exists to remove that copying step. The README describes it as a production-focused optimization toolkit with more than 100 optimizers, more than 10 learning rate schedulers and more than 10 loss functions behind a consistent API, and it credits jettify/pytorch-optimizer as the inspiration. The intended user is someone doing training-method experiments, not someone shipping a model that trains fine on AdamW. The consistent API claim is the substance here: the value is not that a particular optimizer is better, it is that all of them accept model.parameters() the same way.
How the loader API and create_optimizer actually wire together
There are four entry points described in the README, and they differ in how much they do for you. The first is direct class import, from pytorch_optimizer import AdamP, then AdamP(model.parameters(), lr=1e-3), which behaves like any torch.optim class. The second is name-based lookup: load_optimizer('adamp') returns the class, which you then instantiate yourself. The third is create_optimizer(model, optimizer_name='adamp', lr=1e-3, weight_decay=1e-3, use_gc=True, use_lookahead=True), which takes the model rather than its parameters and applies the wrappers for you. That is the layer where the toolkit stops being a mirror of torch.optim. Gradient Centralization and Lookahead are not optimizers in their own right; they modify the update rule around whatever base optimizer you picked, and create_optimizer is the only documented path that composes them from flags. The fourth path is torch.hub.load('kozistr/pytorch_optimizer', 'adamp'), which fetches the class without a pip install, useful for a throwaway notebook but not something to put in a training container. Discovery is handled by three parallel functions: get_supported_optimizers, get_supported_lr_schedulers and get_supported_loss_functions. Each accepts a glob string or a list of globs, so get_supported_optimizers('adam*') returns a list of matching names. That is a plain string filter over the registry, which means it is only as accurate as the names the maintainers registered.
Getting it installed and running: requirements, extras and the version pin question
Installation is a single command, pip install pytorch-optimizer. The README states the requirements as Python >=3.8 and PyTorch >=1.10. Optional integrations are explicitly not installed by default, and the README lists three of them with links to their own installation instructions: bitsandbytes, q-galore-torch and torchao. That matters more than it looks. If you reach for a quantized-memory optimizer variant and it fails at import, the cause is a missing extra, not a bug in the optimizer. The documented usage pattern is short enough to paste: model = YourModel(), optimizer = AdamP(model.parameters(), lr=1e-3) for the direct path, or create_optimizer(model, optimizer_name='adamp', lr=1e-3, weight_decay=1e-3, use_gc=True, use_lookahead=True) when you want the wrappers. Note that create_optimizer takes the model and the others take parameters; mixing the two signatures up is the most likely early mistake. There is no configuration file and no environment variable in the material provided, so all behaviour is set through constructor arguments.
Version cadence and what upgrading costs you
The release history shows three releases in roughly a year: v3.9.0 in December 2025, v3.10.0 in March 2026, v3.10.1 in May 2026, with the repository last pushed in September 2026. That is a moderate cadence, not a slow one. The practical cost is that the package tracks the research literature, so new optimizers appear as new names rather than as changes to existing ones. Upgrading should therefore be low-risk for code that imports a stable class like AdamP, and higher-risk for anything built on create_optimizer, where the wrapper composition and the accepted keyword set are the parts most likely to shift between minor versions. Since the material does not include a changelog or deprecation policy, the safe move is to pin an exact version in your requirements file and read the release notes before moving. The licence is Apache-2.0, which permits commercial and closed-source use and includes a patent grant; that is a permissive choice consistent with PyTorch's own licensing. This is a description of the licence text, not legal advice, and if you are redistributing a modified copy you should read the notice and attribution requirements yourself.
Where it is the wrong tool: single-optimizer training and reproducibility-sensitive work
If your training script uses AdamW and you have no intention of changing that, this package adds a dependency for nothing. torch.optim.AdamW is already there. The second case is subtler. When you adopt an optimizer through a loader, you are trusting an implementation you did not read, and the README does not make a claim about numerical equivalence with the reference implementations it links to in its table. The table maps each optimizer to an official code repository and a paper, which tells you where the algorithm came from, not whether the port matches it step for step. For a paper reproduction where the exact update rule is the experiment, that gap is the whole problem, and you are better off with the authors' own code. A third case is the torch.hub path: it pulls the class from the repository's default branch, so it is not reproducible across time and should be treated as a convenience for exploration only.
The alternative: torch.optim plus a vendored file
The realistic alternative is not another collection package, it is torch.optim for the optimizers PyTorch ships plus a single vendored file for whatever the paper gave you. The difference in approach is about surface area. torch.optim has no loader, no name registry and no wrapper composition, so switching optimizers means editing an import and a constructor call, and adding a research optimizer means pasting one file you can read in full. pytorch-optimizer trades that transparency for breadth: one get_supported_optimizers call tells you what is available, and create_optimizer composes Gradient Centralization and Lookahead without you writing the wrapper. If you are sweeping ten optimizers, the collection wins on effort. If you are running one, the vendored file wins on auditability, because the code you are running is the code you read. A second alternative worth naming is the upstream jettify/pytorch-optimizer that this project credits as its inspiration; the README does not describe how the two differ, so if you are choosing between them you will need to compare their supported-optimizer lists yourself.
What to verify before you commit to it
Three things are checkable from the material and worth checking in your own environment. First, run get_supported_optimizers() and get_supported_lr_schedulers() in a scratch script and confirm the specific names you plan to use are present in the version you pinned, since the glob filters only match registered names. Second, if your target optimizer is one of the quantized or memory-efficient variants, confirm which optional package it needs and install that extra explicitly, because the README is clear that bitsandbytes, q-galore-torch and torchao are not pulled in by the base install. Third, if you plan to use use_gc=True or use_lookahead=True, test the composed optimizer on a small run before wiring it into a long job, because the wrapper path through create_optimizer is the part of the API with the most moving pieces and the least documentation in the supplied material. The documentation site at pytorch-optimizers.readthedocs.io is the place to look for per-optimizer argument details that the README does not reproduce.
Editorial conclusion
Adopt pytorch-optimizer if you are running optimizer comparisons and want AdaBelief, Ranger21, Muon or Scion reachable through one import and one create_optimizer call instead of a folder of vendored research scripts. Do not adopt it if you have settled on a single optimizer that torch.optim already ships, because you would be adding a large package for no behavioural gain. Before committing, pin the version, confirm your PyTorch build satisfies the stated >=1.10 floor, and check whether the specific optimizer you intend to use requires an optional extra such as bitsandbytes or torchao that pip will not install for you.
Community notes