AdapterHub adapters: Parameter-Efficient Fine-Tuning for HuggingFace Transformers

The problem adapters solves: many tasks, one frozen base model

Full fine-tuning stores a complete copy of the model per task. For a 7B or 13B checkpoint, that is tens of gigabytes per task, and every copy diverges from the others in ways that are hard to reconcile. The adapters library, from the AdapterHub team, attacks that directly: it is an add-on to HuggingFace Transformers that injects small trainable modules into a frozen base model, so the base weights stay shared and each task adds only a small artifact.

The README describes it as integrating 10+ adapter methods into 20+ Transformer models with minimal coding overhead for training and inference. That framing matters. This is not a training framework that replaces Transformers. It is a layer on top of it, which means your tokenizers, datasets and training loops keep working. The intended user is someone who already fine-tunes Transformers checkpoints and now has to manage more than one task, language or domain against the same backbone.

It also replaced the older adapter-transformers package. The README states that all previously trained adapters are compatible with the new library and points to a transitioning page for migration. If you have adapter-transformers code in production, that compatibility claim is the first thing to check against your own checkpoints.

How the adapter injection and composition mechanism works

Two entry points exist, and the difference is worth understanding before you write code. AutoAdapterModel.from_pretrained loads a base model that already has the adapter machinery wired in. If you have an existing model class, adapters.init(model) retrofits it in place. The second path is what lets you keep an existing classification head or training script.

Once a model is adapter-aware, model.add_adapter(name, config=...) registers a new module and model.train_adapter(name) freezes everything except that module. The config argument selects the method. A string such as "lora" picks a preset; a config object such as PrefixTuningConfig(prefix_length=20) or ParBnConfig(reduction_factor=4) sets the method's own hyperparameters. ConfigUnion stacks two methods on the same adapter name, which is how the README builds a combined prefix-tuning plus parallel bottleneck adapter.

Composition is the second mechanism, and it is where the library goes beyond simple LoRA swapping. adapters.composition exposes stacking and parallel operators. In the README example, two adapters are loaded from the Hub and wrapped in ac.Parallel(qc, sent) inside an AdapterSetup context manager, so both run side by side in a single forward pass. The documentation also describes adapter merging via task arithmetics. The practical consequence: routing, ensembling and weight interpolation happen at the adapter layer, not by rebuilding checkpoints.

Installing adapters and running a first inference

The README states that adapters supports Python 3.9+ and PyTorch 2.0+, and that PyTorch should be installed first. The PyPI path is a single command.

pip install -U adapters

If you prefer to track the repository, the README gives a source install: clone the repository, enter the directory, and run pip install . from there.

git clone https://github.com/adapter-hub/adapters.git
cd adapters
pip install .

The quickest real use is loading a pre-trained adapter rather than training one. The README's Quick Tour loads roberta-base through AutoAdapterModel, then pulls AdapterHub/roberta-base-pf-imdb with source="hf" and set_active=True. That last flag matters: without it the adapter is loaded but not applied, and your logits come from the unadapted base model.

from adapters import AutoAdapterModel
from transformers import AutoTokenizer

model = AutoAdapterModel.from_pretrained("roberta-base")
tokenizer = AutoTokenizer.from_pretrained("roberta-base")

model.load_adapter("AdapterHub/roberta-base-pf-imdb", source="hf", set_active=True)

print(model(**tokenizer("This works great!", return_tensors="pt")).logits)

What you should see is a logits tensor for the sentiment task the adapter was trained on. If you swap in your own base checkpoint, note that the adapter must have been trained against a compatible model; the repository does not make an incompatible pairing work.

Training a LoRA adapter on a model you already have

The retrofit path is the one most teams will use, because it does not require abandoning an existing model class. The README adapts a T5 sequence-classification model by calling adapters.init(model), adding a LoRA adapter by name, and then calling model.train_adapter("my_lora_adapter"). After that call, the README says to run your regular training loop.

import adapters
from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("t5-base")

adapters.init(model)

model.add_adapter("my_lora_adapter", config="lora")
model.train_adapter("my_lora_adapter")

The constraint to internalize is that train_adapter freezes the base weights. Any parameter you expected the optimizer to update outside the adapter will not move. If your training loop builds its optimizer from model.parameters(), check that it is picking up the adapter parameters rather than silently optimizing a frozen set. The repository's own test suite is organized by method, with pytest markers such as lora, bottleneck, prefix_tuning and ia3 declared in pyproject.toml, which tells you the methods are exercised independently rather than through a single shared path.

Where adapters is the wrong choice

Parameter-efficient tuning trades peak quality for storage and modularity. If you are fine-tuning one task on one model and never intend to serve a second, the adapter indirection buys you nothing, and a plain Transformers fine-tune is simpler to debug. The library also inherits Transformers' model coverage rather than exceeding it: the README claims 20+ models, not all of them, so a niche or very new architecture may simply not be supported.

The dependency surface is another real cost. The setup.py dependency list pins transformers~=4.57.6 and includes training and evaluation utilities such as datasets, evaluate, accelerate, sacrebleu and rouge-score. Some of that is for the examples and tests rather than the runtime, but a version conflict with your existing Transformers pin is a plausible failure mode. The Makefile shows the project's own test commands run against PYTHONPATH=src with pytest-xdist, so the suite is large; that is a maintenance signal for the maintainers, not a guarantee about your environment.

Finally, composition is powerful and under-documented in the README. Parallel and stacking operators change what the forward pass computes, and the README shows one example without discussing how conflicts between two adapters are resolved. Treat composed adapters as something to validate empirically on your own data.

How adapters differs from plain LoRA via PEFT

The obvious alternative is HuggingFace's own PEFT library, which also implements LoRA and prompt tuning against Transformers models. The difference is scope. PEFT centers on attaching a single method to a model. adapters centers on method plurality and composition: the README lists bottleneck adapters, AdapterFusion, MAD-X and invertible adapters, AdapterDrop, and more, and it exposes ConfigUnion and composition blocks as first-class API rather than as research add-ons.

That shows up in the adapter hub itself. Adapters are published under the AdapterHub namespace and loaded with source="hf", so a pre-trained adapter such as AdapterHub/roberta-base-pf-imdb is a named artifact you can pull in one call. If your workflow is one LoRA per task and nothing else, PEFT is the leaner dependency. If your workflow involves cross-lingual transfer, fusing several task adapters, or running adapters in parallel, adapters is built around exactly those operations. The two are not mutually exclusive in principle, but running both against the same model class is the kind of thing you should test rather than assume.

Licence, maintenance and the cost of upgrading

adapters is released under Apache-2.0, with the licence text in the repository root and the copyright header naming the AdapterHub Team. Apache-2.0 permits commercial use and modification and includes a patent grant; it also requires that you retain the licence and notice files. That is the shape of the terms, not legal advice, and if you redistribute a product built on it you should have your own counsel read the licence rather than this paragraph.

The last push to the default branch was on 2026-04-26, the same date as the v1.3.0 release. The previous release, v1.2.0, was on 2025-05-20. So the release cadence is roughly annual, not continuous, and the repository is not archived. Plan upgrades around that rhythm rather than expecting frequent patch releases.

The upgrade cost is concentrated in the Transformers pin. Because setup.py pins transformers~=4.57.6, moving to a newer Transformers minor version means waiting for an adapters release or testing against source. The README also notes that the library replaced adapter-transformers and that previously trained adapters remain compatible, so a migration from the legacy package is a rename and import change rather than a retraining exercise. Verify that claim against your own saved adapters before you delete the old environment.

Editorial conclusion

Adopt adapters if you serve several tasks over one frozen base model and want to swap or merge small modules instead of storing full checkpoints. Do not adopt it if you only ever train one task end to end, or if your model is not in the supported model list. Before committing, verify that your base checkpoint appears in the model overview, that your Transformers version satisfies the transformers~=4.57.6 pin in setup.py, and that the adapter you intend to load actually exists on the Hub under the AdapterHub namespace.

Frequently asked questions

Official sources

1. adapter-hub/adapters on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases