sense2vec: contextually-keyed word vectors for spaCy pipelines
🦆 Contextually-keyed word vectors
At a glance
- What is it?
- sense2vec extends word2vec by keying vectors to part-of-speech tags and entity labels, so multi-word phrases get their own entries. It is a small Python library for loading, querying and training those vectors, and it fits best inside a spaCy v3 pipeline.
- Who is it for?
- Adopt sense2vec if you already run spaCy v3 and want phrase-level similarity without building your own embedding stack. Do not adopt it if you need contextual embeddings from a transformer, if you cannot host a 573 MB or 4 GB vector directory, or if you are still on spaCy v2, since that path requires pinning sense2vec==1.0.3 and the v1.x branch.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 173 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem sense2vec solves that plain word2vec does not
Standard word2vec gives one vector per token. That collapses distinct meanings into a single point and leaves multi-word phrases without their own representation. sense2vec, based on the 2015 paper by Trask et al., keys each vector to a context label instead: a part-of-speech tag or an entity label. The README's own example makes this concrete. The query is the string "natural_language_processing|NOUN", not the bare phrase. The pipe and the tag are part of the key. That means "duck|NOUN" and "duck|VERB" can occupy different regions of the vector space, and "natural language processing" is addressable as a unit rather than as three separate tokens.
The audience is narrow but real. This is for engineers who already work in the spaCy ecosystem and want phrase-level similarity, nearest-neighbor lookup or annotation bootstrapping without assembling a training pipeline from scratch. It is not a general-purpose embedding service and it does not try to be one.
How the keying and query surface actually work
The library exposes a Sense2Vec class for standalone use and a Sense2VecComponent for spaCy pipelines. In standalone mode you construct the object and call from_disk with a path to an extracted vector directory, then index it with a key string. The README shows three operations: membership via "query in s2v", retrieval via "s2v[query]", and frequency via "s2v.get_freq(query)". Nearest neighbors come from "s2v.most_similar(query, n=3)", which returns tuples of key and score.
Inside a spaCy pipeline the same data is reached through extension attributes on a span. After adding the component with "nlp.add_pipe("sense2vec")" and loading vectors with "s2v.from_disk(...)", a span carries "_.s2v_freq", "_.s2v_vec" and "_.s2v_most_similar(3)". The README example slices "doc[3:6]" to get the phrase "natural language processing" and then queries it. Note the shape difference: the standalone call returns string keys like "machine_learning|NOUN", while the pipeline call returns tuples like (('machine learning', 'NOUN'), 0.8986967). The underscore in the key is a serialization detail; the tuple form is what the span attribute gives back.
The README also mentions optional caching of nearest neighbors for faster most-similar queries. It does not document the cache configuration in the material available here, so treat that as something to check in the source before relying on it.
Getting vectors loaded: install, download, unpack
Installation is a single pip command: "pip install sense2vec". Pretrained vectors are separate downloads attached to the GitHub release, not to the pip package. The README lists two: s2v_reddit_2015_md at 573 MB, trained on Reddit comments from 2015, and s2v_reddit_2019_lg at 4 GB, trained on Reddit comments from January to July 2019.
The large one ships as multi-part archives. The README gives the exact merge command: "cat s2v_reddit_2019_lg.tar.gz.* > s2v_reddit_2019_lg.tar.gz". After unpacking, point from_disk at the extracted directory. The README's own line is "s2v = Sense2Vec().from_disk("/path/to/s2v_reddit_2015_md")".
There is a Streamlit demo script in the repository for exploring vectors and nearest neighbors. It takes one or more paths to pretrained vectors as positional arguments, and the README shows it being run directly from a raw GitHub URL after "pip install streamlit". That is a reasonable way to sanity-check a download before wiring the component into anything larger.
Training your own vectors, and what that commitment looks like
The README states you can train your own vectors using a pretrained spaCy model, raw text, and either GloVe or Word2Vec via fastText. The training detail is behind a link in the README rather than spelled out inline, so the exact command surface is not something I can quote here. What is clear from the feature list is the shape of the dependency chain: spaCy supplies the tokenization, tagging and entity labels that become the keys, and an external trainer supplies the embedding step. If your spaCy model tags differently from the one used to build the pretrained vectors, your keys will not line up with the shipped vectors. That is a real coupling, and it is the main reason to prefer training your own set when your domain diverges from Reddit comments.
The repository also ships Prodigy annotation recipes. According to the README, these are for evaluating models, creating lists of similar multi-word phrases, and converting those lists into match patterns for rule-based NER or to bootstrap NER annotation. Prodigy is a separate commercial product from the same organization, so those recipes are only useful if you already have it.
Where sense2vec is the wrong tool
The vectors are static. Each key maps to one learned vector regardless of the sentence it appears in. A transformer-based encoder produces a representation that shifts with surrounding context; sense2vec does not. If your task depends on disambiguating a word by its sentence, this library will not do it, and no amount of keying by POS tag substitutes for that.
There is also a version cliff. The README is explicit that the spaCy v3 examples apply to version 2.0 of this library, and that spaCy v2 users must download "sense2vec==1.0.3" and check out the v1.x branch. That is a fork in the documentation, not a compatibility shim. Anyone on an older spaCy pipeline is effectively maintaining against an unmaintained branch.
The pretrained vectors are another constraint. They are trained on Reddit comments. For legal text, clinical notes or internal documentation, the vocabulary and the sense distinctions will be wrong in ways that are hard to measure without your own evaluation set. And the 4 GB archive is a genuine storage and memory commitment before you have written a line of application code.
How it compares to gensim's word2vec
gensim's word2vec, which this project lists among its topics, trains and queries vectors keyed by token. To get phrase vectors out of gensim you either pre-join phrases with a phrase detector before training or accept that multi-word expressions are averaged from their parts. sense2vec moves the keying into the model itself: the phrase plus its tag is the unit of training and the unit of lookup. The practical difference shows up in the query interface. With gensim you call most_similar on a token string. With sense2vec you call it on "natural_language_processing|NOUN", and the returned neighbors are also tagged keys. That extra tag is what lets the same surface form appear twice with different neighbors.
The cost of that design is that you cannot query a bare word. Every lookup needs a valid key, and the key has to have been seen during training. The README's standalone example guards this with an assertion that the query is in the model, which is a hint about how often that check matters.
Licence, maintenance and upgrade cost
The library is MIT licensed, which permits commercial use and modification provided the copyright notice and permission notice are retained. That is the standard permissive arrangement, but it covers the code. The pretrained vector archives are separate artifacts attached to a GitHub release, and the README does not state a licence for them in the material available here. If you plan to redistribute the vectors or ship them inside a product, that is the question to resolve first, and it is not a question I can answer from the README alone.
On maintenance: the release history shows v2.0.0 in February 2021, v2.0.1 in December 2022, and v2.0.2 in April 2023. The repository is not archived and the last push is recent, but the version cadence is slow and the releases are small. The real upgrade cost is not the library; it is the spaCy version underneath it. Moving from spaCy v2 to v3 is what forced the 2.0 rewrite, and the README keeps the v1.x branch around precisely because that migration is not free. Budget for the spaCy upgrade, not the sense2vec one.
Editorial conclusion
Adopt sense2vec if you already run spaCy v3 and want phrase-level similarity without building your own embedding stack. Do not adopt it if you need contextual embeddings from a transformer, if you cannot host a 573 MB or 4 GB vector directory, or if you are still on spaCy v2, since that path requires pinning sense2vec==1.0.3 and the v1.x branch. Before committing, verify that the pretrained Reddit vectors cover your domain vocabulary, and check that your spaCy model's tokenizer and tagger produce the same keys the vectors were trained with.
Community notes