spacy-transformers: transformer weights inside a spaCy v3 pipeline

The gap spacy-transformers fills between spaCy and Hugging Face

spaCy pipelines are built out of components that read and write a Doc object, and those components expect tokenization that matches the pipeline's own tokenizer. Hugging Face transformers ship their own subword tokenizers, and their encoders emit one vector per subword rather than one per spaCy token. spacy-transformers is the package that sits in the middle. The README describes it as providing "spaCy components and architectures to use transformer models via Hugging Face's transformers in spaCy", and it lists automatic alignment of transformer output to spaCy's tokenization as one of its features. That alignment step is the reason the package exists rather than a thin wrapper being enough. The audience is narrow and specific: teams that already run spaCy v3 and want a pretrained encoder such as BERT, RoBERTa or XLNet to supply features for spaCy components they train themselves. The README's own warning is blunt about scope: the transformer component "does not support task-specific heads like token or text classification". If you want the predictions of a classification checkpoint rather than features for your own training run, this is not the package for that job.

One shared transformer, several components backpropagating into it

The architecture the README describes has a single Transformer pipeline component and one or more task components downstream of it. The feature list calls this "easy multi-task learning: backprop to one transformer model from several pipeline components". In practice that means a ner component and a textcat component can both read the same transformer output and both contribute gradient to the same encoder weights during training, instead of each carrying its own copy. The package also lets you control how much transformer data lands in the Doc, and how long documents are processed, both listed as customizable. Long-document handling matters because transformer encoders have a fixed window; the package exposes a way to decide how documents are chunked rather than forcing one strategy. Serialization and model packaging are described as working out of the box, which is what makes a transformer-backed pipeline shippable as a normal spaCy model directory. Everything here is registered through spaCy v3's config system, so the wiring lives in config rather than in Python glue code.

Install order and the CUDA bracket that trips people up

The README gives one install command: pip install 'spacy[transformers]'. It states that this pulls in all dependencies, including PyTorch and spaCy, and that you must install the package before you install the models. For GPU machines the README says to find your CUDA version with nvcc --version and add it in brackets, giving spacy[transformers,cuda92] for CUDA 9.2 and spacy[transformers,cuda100] for CUDA 10.0 as examples. If PyTorch refuses to install cleanly, the README points at the official PyTorch site for your operating system rather than offering a workaround of its own. The stated floors are Python 3.6+, PyTorch v1.5+ and spaCy v3.0+. That last one is not a soft preference. The README's top note says this release requires spaCy v3 and directs anyone on the older line to the v0.6.x branch, and a later note adds that previous versions built for spaCy v2.x "worked considerably differently". A v2 pipeline and a v3 pipeline are not interchangeable here, and the config-driven training flow the package assumes is a v3 feature.

What the transformer component will not do for you

The clearest limitation is stated in the README itself and it is worth repeating because it is the most common wrong expectation. A task-specific transformer model can serve as a source of features for training spaCy components like ner or textcat, but the transformer component gives no access to task-specific heads for training or inference. So a checkpoint fine-tuned for, say, token classification cannot simply be dropped in and queried for labels. You either train your own head on top of the encoder through spaCy, or you use the predictions of the existing checkpoint through a different route. The README names that route: the wrappers in spacy-huggingface-pipelines, which it describes as the way to incorporate task-specific transformer models into spaCy pipelines when you only want predictions. There is a second boundary implied by the release history. Release v1.4.0 is titled "Update transformers pin", and v1.3.8 is titled "Fix wheels on v1.3.7 release". A pin update as the headline of a release tells you the package tracks the transformers library closely and that an unpinned environment can drift out from under it. Pin both sides in your own lockfile.

Comparing the two Explosion routes for Hugging Face models

spacy-transformers and spacy-huggingface-pipelines solve adjacent problems with opposite data flows. spacy-transformers puts a transformer encoder inside the spaCy pipeline as a component, aligns its subword output to spaCy tokens, and lets you train spaCy components on those features, with the option of several components sharing one encoder. spacy-huggingface-pipelines, in the README's phrasing, is for when "you only want use to the predictions from an existing Hugging Face text or token classification model". The difference is where the task head lives and who trains it. With spacy-transformers you own the head and the training loop, and you get spaCy's Doc structure and serialization around it. With the pipelines wrapper you inherit someone else's head and its label set, and you get predictions without a fine-tuning step. Choosing the wrong one shows up as either a training run you did not need or a label set you cannot change. A third option is to skip spaCy entirely and call transformers directly, which is reasonable if your output is vectors or labels and you have no use for spaCy's tokenization, pipeline components or model packaging. The alignment work is the thing you would be reimplementing.

Maintenance surface and the MIT licence

The repository is not archived, and the most recent push recorded is 2026-03-27, so the project is still being touched. The release cadence visible in the supplied list is patch-oriented: v1.4.0 in March 2026, v1.3.9 in May 2025, v1.3.8 in February 2025. Two of those three release titles concern packaging or dependency pins rather than features, which is a fair signal of where the ongoing work sits. For an adopter, the practical cost is not in this package's own API, which is small and registered through spaCy config. It is in the versions underneath it: the transformers pin, the PyTorch build, the CUDA bracket, and the spaCy v3 line. Upgrading any of those can force a re-check of the others, and a model you trained and packaged will need to be re-validated against the new combination. The licence is MIT, which is permissive and imposes few conditions on redistribution, but it covers this package only. The transformer checkpoints you load carry their own licences, and those vary by model and by publisher. That is a question for whoever handles licensing on your side, not something this package resolves.

A note on where the documentation actually lives

The README is a signpost, not a manual. It links out to spaCy's usage page on embeddings, transformers and transfer learning, the training pipelines page, the layers and architectures page, the Transformer component API reference, and the architectures page listing registered functions. It also states that bug reports belong in spaCy's issue tracker rather than this repository's, with the discussion board for everything else. That arrangement is worth knowing before you start, because it means a search for spacy-transformers problems will often land in spaCy's tracker instead. It also means the README alone will not tell you the config block names or the registered function signatures; those are in the linked API pages. The README does confirm the pieces you need to plan around: the transformer component name, the automatic token alignment, the multi-task gradient sharing, the Doc customization, the long-document handling, and the serialization path. Everything past that is on spacy.io.

Editorial conclusion

Adopt spacy-transformers if you are already on spaCy v3 and want to fine-tune a spaCy component such as ner or textcat on top of a Hugging Face encoder, with several components sharing one transformer. Do not adopt it if you only need the predictions of an existing text or token classification checkpoint, or if you are still on spaCy v2; the README points the first case at spacy-huggingface-pipelines and the second at the v0.6.x branch. Before committing, verify that your installed spaCy is v3.0 or later, that your Python is 3.6 or later and your PyTorch is v1.5 or later, and check the transformers pin in release v1.4.0 against the checkpoint you intend to load, because that release exists specifically to move that pin.

Official sources

1. explosion/spacy-transformers on GitHub
2. License: MIT
3. Project website
4. README
5. Releases