What is Semantic search?

How the matching works

An embedding model is a neural network trained to put related texts near each other in a vector space. Training is contrastive: given a query and a passage that answers it, the loss pulls the two vectors together, and given passages that do not, it pushes them apart. Nothing about the training data contains rules for synonyms; the geometry of the space is the rule.

A search then consists of encoding the query, encoding every document once, and finding the nearest vectors. Similarity is usually a dot product or a cosine, and the ranking is a ranking of distances. The second step is the interesting engineering one. For a few thousand documents, comparing the query against all of them is fast enough. For millions, you need an approximate nearest neighbour index, most commonly a graph structure such as HNSW, which builds a navigable network of the vectors and searches it approximately. Approximate is the operative word: the index trades a small amount of recall for a large amount of speed, and the trade is a parameter rather than a fact.

Two decisions sit upstream of any of that and matter more than the index. What you embed: a whole document, or a passage. Embedding a whole document averages its content into one vector, so a long document about five unrelated things matches none of them well. Chunking a document into passages gives sharper vectors and loses the surrounding context, and where you cut determines what a chunk means. And which model: the embedding model is the ceiling on quality, its dimensionality is a memory and speed cost, and changing it means re-embedding everything, which on a large corpus is the single most expensive routine operation in this whole stack.

What a vector cannot represent

Semantic search is a neighbourhood search, and everything in a neighbourhood shares properties whether or not you wanted them.

Exact strings are the first casualty. A document containing error code ABC-1234 is not near a query containing ABC-1234 unless the model has decided those characters are meaningful features, and a fixed-size vector of a few hundred numbers has no room for the long tail of identifiers. Model numbers, file paths, chemical names, flags, and people's names all behave this way.

Negation and conditions are worse, because a vector encodes topical similarity, not truth. A document about when a refund is not available is very near a query about refunds, and the ordering between them is not meaningful.

Filters are the escape hatch, which is why metadata filtering exists in every serious vector store. A date range, a tenant, a document type and a permission scope cannot be expressed in the vector at all, and a system that tries to encode them in the embedding is doing it badly. Filtering is done during the search, not after it, which is also why filter support is the feature to check first in any store.

Ranking signals that are not about meaning are missing too. Recency, popularity, an editorial boost, a business rule about which document should win: these are properties of the system, not of the text, and a pure vector search has nowhere to put them.

And the result shape is worth naming. In keyword search, the tenth result is usually a different thing from the first. In semantic search, results five through ten are frequently near-duplicates of results one through four, because the model found one region of the space and stayed there. Which is a feature when the user wants breadth of coverage in a region and a bug when they want variety.

When it is the right tool, and when keywords are enough

Semantic search pays where the user knows what they want and not what to type.

It fits paraphrased and conversational queries. Asking how to undo a payment and a document about refunds share no keywords, and this is the case the technique was built for.

It fits when the vocabulary is unstable. Internal documentation, support tickets and developer forums all accumulate several words for the same thing, and a keyword index only finds the ones a user happened to pick.

It fits when the corpus is too large to browse, which is most documentation sets, most ticket archives and most message histories.

It fits code. yoanbernabeu/grepai is a Go CLI that indexes a repository with local embeddings so you can query code by meaning and trace callers before editing, per our analysis, and giancarloerra/SocratiCode is the enterprise version of the same idea, indexing a repository into Qdrant and exposing hybrid semantic search, dependency graphs and symbol-level impact analysis over MCP. Code is a good fit because developers describe behaviour, not identifiers.

It does not fit a small, structured corpus where exact field matching is what the user means. It does not fit when the query must be an exact string, which is a different feature and a simpler one. It does not fit when the corpus changes faster than you can re-embed it, because an index that is stale is worse than no index. And it does not fit when you have no way to judge whether it is working, because the failure mode of a bad embedding is a plausible set of wrong answers rather than an error.

The failure modes

Five of them, and they account for most complaints.

Chunking done by accident. Chunks that are too small lose the sentence that made them mean something, and chunks that are too large average several topics into one vector. Chunks that cut a table, a function or a numbered list produce embeddings of half an idea. This is the quiet one, because the search returns results rather than failing.

Mixing models in one index. Vectors from different models live in unrelated spaces, and an index containing both returns neighbours that are meaningless. Re-embedding is the correct fix and is not optional, which is why the embedding model should be recorded alongside the index.

Forgetting that the index is approximate. An approximate search returns a subset of the true nearest neighbours, and a recall problem in the index looks exactly like a relevance problem in the model. Tuning the model when the index is the problem wastes a week.

Filtering after searching. Retrieve the top hundred vectors, then drop the ones from the wrong tenant or outside the date range, and the result is a short list or an empty one. Filters belong inside the search.

Judging by the top result. Looking at the first answer and concluding semantic search works tells you very little. Without a set of queries with known relevant documents, the only honest measurement is a comparison, and the most common comparison is against nothing.

How the projects use it

Five groups, and the group tells you what you inherit.

Orchestration frameworks. deepset-ai/haystack is an Apache-2.0 Python framework for RAG, semantic search and agents, and our analysis of it is that it is strongest when you want explicit control over retrieval and routing and weakest when you want a managed platform. neuml/txtai is the all-in-one version: our analysis describes it as a Python library combining a vector embeddings database, an LLM pipeline system and an agent framework into one package built on Hugging Face Transformers and FastAPI, with the company behind it positioning it as an alternative to assembling separate components from several frameworks.

Code search. grepai and SocratiCode are the serious ones, and PawanOsman/OpenCursor is the small one: our analysis describes it as an MIT-licensed VS Code extension running an agentic chat against cloud subscriptions, API keys or fully offline models, shipping semantic codebase search and MCP support, with documentation thinner than its feature list.

Personal knowledge. brianpetro/obsidian-smart-connections is an Obsidian plugin that indexes a vault with a local embedding model and surfaces related notes in a list and a graph view, zero setup and no API key by default; our analysis notes it is zero-setup but that the core is source-available rather than conventionally licensed, which is worth knowing before you put your notes into it. eugeniughelbur/obsidian-second-brain uses a vault as shared memory for command-line coding agents with hybrid semantic search and scheduled agents that rewrite notes, and our analysis flags the design as opinionated about note format, which is the part to weigh.

Applications that embed a vector store. codedogQBY/ReadAny is an e-book reader for desktop and mobile that runs embeddings and a vector store locally, adds hybrid vector plus BM25 retrieval and syncs through WebDAV, with a signed binary install rather than a server. ssrajadh/sentrysearch indexes video chunks as embeddings in a local ChromaDB and returns a trimmed clip for a text or image query, and our analysis says it is a single-user CLI whose README is explicit that the cloud backends send your footage to an external API. AmintaCCCP/GithubStarsManager syncs your starred repositories, summarises and categorises them with a model you configure, and adds vector search and repository question answering on top; our analysis describes it as an Electron app, a Docker stack and a Cloudflare Worker under MIT licensing.

The pattern across all of them is the same: the vector index is the easy part, and chunking, filtering, evaluation and the licence of whatever you put in the index are where the work is.

In practice

Semantic search is a nearest-neighbour lookup over vectors, which makes it excellent at paraphrase and unreliable at exact strings, and that trade is the design. Get the chunking right before you get the model right, never mix embedding models inside one index, apply filters inside the search, and evaluate with a set of queries that have known answers rather than by looking at the first result. For a library to build on, neuml/txtai is the shortest route if you want the index and the pipeline in one package, and deepset-ai/haystack is the route that keeps retrieval explicit; for code, grepai and SocratiCode are the two to compare; and for anything that runs on somebody's notes or footage, read what the project says about where the data goes before you install it.