spacy-llm: Adding LLM Components to spaCy Pipelines Without Training Data

What spacy-llm Solves and Who It Is For

spaCy is a production-grade NLP library built around supervised learning and rule-based components. Adding an LLM to a spaCy pipeline in a way that is serializable, configurable, and compatible with spaCy's pipeline API requires bridging two different programming models. spacy-llm does that bridging.

The package targets three user types. The first is a developer who wants to prototype an NLP task quickly without collecting labeled data; LLM prompting can serve as a viable approach for early exploration. The second is a developer who wants a gradual migration path: start with an LLM component, observe where it underperforms, collect data for those cases, and replace the LLM component with a trained model. The third is a developer who needs both a lightweight NLP task (text classification, for example) and a more demanding one (document summarization) in the same pipeline, where a cheap trained model handles the first and an LLM handles the second.

The README frames this last case clearly: a cheap text classification model can identify which documents need summarization, while the LLM handles that work. The two components coexist in one spaCy pipeline with a standard serialization format.

Tasks, Models, and the spacy-llm Component Architecture

spacy-llm adds an llm pipeline component to spaCy. The component takes a task and a model as configuration. The task defines how to format the prompt and how to parse the LLM's response into a structured spaCy output. The model defines which LLM to call and how.

The package provides built-in tasks for: named entity recognition, text classification, lemmatization, relationship extraction, sentiment analysis, span categorization, summarization, entity linking, translation, and raw prompt execution. Each task has a versioned factory (for example, spacy.TextCat.v3) that can be referenced in a config file or in Python code.

For the model, spacy-llm ships with interfaces to OpenAI, Cohere, Anthropic, Google PaLM, and Microsoft Azure AI, as well as to open-source models on Hugging Face, including Falcon, Dolly, Llama 2, OpenLLaMA, StableLM, and Mistral. LangChain models are also supported, which means any LangChain model or feature can be used in spacy-llm pipelines.

A map-reduce mechanism handles text that exceeds the LLM's context window by splitting it into chunks, running the prompt on each chunk, and fusing the results back together.

Installing and Running a First Pipeline

spacy-llm is installed with pip inside an existing spaCy environment:

python -m pip install spacy-llm

The simplest path to a working pipeline is through Python code. The README shows a text classification example using the llm_textcat factory:

import spacy

nlp = spacy.blank("en")
llm = nlp.add_pipe("llm_textcat")
llm.add_label("INSULT")
llm.add_label("COMPLIMENT")
doc = nlp("You look gorgeous!")
print(doc.cats)

This uses the default GPT-3-5 model from OpenAI. For more control, the README shows a config.cfg file approach that specifies task version and model explicitly. The task section names the factory (for example, spacy.TextCat.v3 for text categorization) and lists the labels. The model section names the LLM model factory (for example, spacy.GPT-4.v2). This config approach gives full control over the task version, the model version, and any model-specific parameters. The API key must be set as an environment variable before running; the README points to the documentation for API key setup. Without the key, the pipeline initialization fails at the point where it would make its first LLM call.

Supervised Learning vs. LLM Prompting: When to Use Which

The README is direct about the trade-off. LLM prompting is better for prototyping and for tasks with no labeled data. Supervised learning is better for production on a well-defined task: higher accuracy, better efficiency, more reliable and controllable output. The README states that a transformer model running on a single GPU will generally outperform LLM prompting on a task for which you have labeled data.

spacy-llm's value is not in replacing supervised models; it is in giving spaCy users a path to an LLM component that fits their existing pipeline structure. A team that has invested in spaCy tooling gets LLM access without leaving the spaCy ecosystem.

For tasks where LLM prompting genuinely beats supervised learning, such as tasks requiring common-sense reasoning or cross-document synthesis, the LLM component is the right permanent choice. For tasks where accuracy is critical and a few hundred labeled examples are available, the recommendation implicit in the README is to use the LLM component for rapid prototyping and then train a supervised model. spacy-llm supports that workflow by being serializable and configurable through the same system as the rest of spaCy.

Limitations and Constraints

The README explicitly marks spacy-llm as experimental and warns that breaking changes may occur in minor version updates. This is a serious constraint for production deployments: a minor version bump could require code changes.

LLM costs are a real concern. Every inference request calls an external API, and a pipeline that runs on thousands of documents will incur proportional API costs. The README does not discuss cost estimation or rate limiting, so teams need to budget these separately.

The map-reduce approach for long texts works but adds latency and cost. Splitting a document into overlapping chunks and calling the LLM once per chunk means the processing time scales with document length, not with a fixed inference cost. For very long documents, this can be significant.

spacy-llm does not support asynchronous execution; calls to the LLM are synchronous within the pipeline. For applications that need to process many documents concurrently, the synchronous model creates a bottleneck.

The package requires spaCy 3.5.0 or later and is compatible with Python 3.14 as of the v0.7.4 release. The Pydantic v2 migration was completed in v0.7.4 as well. Projects still on Pydantic v1 need to upgrade before using v0.7.4.

Maintenance Status and License

The last push to the repository was on 2026-03-27, which is approximately six months before the review date. The most recent release, v0.7.4, was published on 2026-03-24 and added Python 3.14 support, completed the Pydantic v2 migration, and updated dependencies. The prior release, v0.7.3, was from January 2025, indicating that the pace of releases is roughly one or two per year.

The project is maintained by Explosion, the company behind spaCy. The MIT license applies, permitting free use and distribution.

An alternative that overlaps in purpose is LangChain itself: LangChain provides LLM chains with output parsers that can be adapted to NLP tasks. The key difference is that LangChain is framework-agnostic, while spacy-llm is designed to integrate directly into spaCy's pipeline model. For teams already using spaCy, spacy-llm avoids maintaining a parallel LangChain dependency for the LLM portion.

Editorial conclusion

spacy-llm is the right choice for spaCy users who want to prototype NLP tasks using LLMs before committing to supervised training, or who need LLM-powered components alongside rule-based and ML-powered ones in the same pipeline. Teams that have already trained spaCy models and do not need LLM fallbacks have no reason to add it. The last push was on 2026-03-27, with the most recent release being v0.7.4 from 2026-03-24. Before adopting it for production, note that the README flags the package as experimental and warns that minor version updates may include breaking interface changes.

Frequently asked questions

Official sources

1. explosion/spacy-llm on GitHub
2. License: MIT
3. Project website
4. README
5. Releases