What is Hybrid search?

How the two systems are combined

The lexical half is a ranking function over words. Okapi BM25 scores a document by how often the query terms appear, with two corrections: term frequency saturates, so the tenth occurrence of a word adds much less than the second, and documents are normalised by length, so a long document does not win simply by containing more words. An inverse document frequency term rewards rare terms and penalises common ones. The result is a probabilistic ranking that is exact about the words you typed and completely indifferent to meaning.

The dense half embeds the query and each document separately and searches by vector similarity, usually with an approximate nearest neighbour index. It matches a document about refund policy against a query about returning a product, which is the case BM25 cannot do at all.

Merging them is the part people get wrong, because the two scores are not on the same scale. BM25 is unbounded above and grows with document length and term rarity; a cosine similarity sits between minus one and one. A weighted sum of the raw numbers is dominated by whichever system has the larger range, and changing the weights does not fix that.

Two standard answers. Score normalisation maps each list onto a common range before weighting, which is direct and leaves you with a parameter to tune. Reciprocal Rank Fusion ignores the scores entirely and uses only positions, combining lists as the sum of one over a constant plus the rank of each document. It has no scale problem, which is why it is a common default, and its cost is that it discards the margin: a document ranked first by a wide margin counts the same as one ranked first by a whisker.

Above the merge sits a third stage in many systems, a model that reads the query and each candidate together and reorders them. That is a separate technique with its own costs, and it is worth deciding deliberately whether the merge or the reranker is where your accuracy is missing.

Why each half fails on the other half's cases

The case for hybrid is not that one ranker is better. It is that their failures are disjoint, and each failure mode is easy to reproduce.

The lexical half fails on paraphrase. Ask for a way to return something you bought and a document titled how to send a purchase back may share almost no query terms. It also fails across languages, since a lexical index cannot match words it has never seen, and on a query that describes a concept rather than using the document's words.

The dense half fails on identifiers. Model and part numbers, error codes, file paths, chemical names, people's names, anything where the exact string matters and the nearest neighbour is a document that is topically similar and concretely wrong. A dense index is also only as good as the embedding model, and a new model means re-embedding the entire corpus, while adding a term to a lexical index is a re-index of one posting list.

That last asymmetry is the practical argument. Lexical retrieval is cheap to update, cheap to explain, and correct on the cases users complain about most loudly, because a user who pastes an error code wants that error code. Dense retrieval is what makes the natural-language questions work at all. Neither subsumes the other.

The corollary is that a hybrid setup has two indexes to maintain, and both have to cover the same corpus at the same version. A system that has drifted between them returns results whose two halves are describing different collections, and that failure is invisible in the output.

When it is worth it, and when it is a second system for nothing

Hybrid search pays when your queries and your corpus are both mixed, which in practice means most enterprise search, most developer documentation, and most product catalogues with model numbers in them.

It pays when users paste exact strings. A hybrid setup keeps the identifier query working and still handles the conversational one, and a purely dense system will quietly lose the first kind.

It pays when the first stage's recall is the actual problem. If you are evaluating an assistant's answer quality and the supporting passage is not in the retrieved set, no fusion weight and no reranker will help, and adding the other half of the hybrid is the fix.

It does not pay on a small corpus where either ranker alone is already finding everything. It does not pay when the problem is chunking or embedding quality, because a better second retriever cannot recover from a bad first representation. And it does not pay when you cannot evaluate it: fusion is a set of parameters, and without relevance judgements on your own queries you are tuning them against a feeling, which is worse than having one ranker and an obvious behaviour.

A third case is worth naming. If the corpus is small enough to score exhaustively, the argument for a sparse index weakens, and some systems simply rerank everything.

Finally, hybrid is not the same as adding a reranker. A cross-encoder reading query and document together often beats both halves for accuracy, at a cost that scales with the number of candidates. Whether you need the second retriever or the second pass is a question about your failure mode, and the two fix different things.

The traps

Five of them account for most disappointing hybrid results.

Naive score addition. Adding a BM25 score to a cosine similarity without normalising means the lexical half simply wins, or the dense half does, depending on the magnitudes involved. If a fusion feels like one of its halves is not contributing, this is why.

Fusion that throws away information. Reciprocal Rank Fusion needs no normalisation and is a reasonable default, and it also discards margins, so a decisive top hit and a coin-flip top hit are treated alike. When the two systems disagree strongly about a document, that disagreement is signal and RRF discards it.

Index drift. Two indexes over one corpus means two update paths. A document added to the vector store and not to the keyword index is invisible to half the system, and the symptom is a relevance result that changes when the merge weight changes.

Tokenisation. A lexical ranker is only as good as the analyzer that splits text into terms, and that is where languages without spaces need segmenters, where a poorly chosen stopword list changes the scores, and where a code corpus and a prose corpus want different treatment. Vector search has no such step, which is part of why it is easier to set up.

Chunk-level versus document-level. Lexical scoring degrades on very short fragments, because the length normalisation that stops long documents from winning starts penalising them, while dense retrieval on long documents blurs. The granularity at which you index the two halves is a real decision, and the two halves often want different ones.

How the projects divide

Four kinds of project here, and the kind tells you how much of the decision you still have to make.

Search engines where hybrid is a feature. meilisearch/meilisearch is an open-source search engine written in Rust and shipped as an HTTP server with a REST API, and our analysis of it describes typo tolerance and hybrid search, suited to teams that want fast results without operating a cluster, and explicitly the wrong tool if you need a general-purpose database or deep analytical aggregation. oramasearch/orama is the embeddable version: our analysis describes it as a JavaScript and TypeScript library with full-text, vector and hybrid modes, a declarative schema and a plugin system, which suits teams wanting search inside their own process and fits badly when the index has to outlive a single process.

Databases that put both under one query engine. infiniflow/infinity is an Apache-2.0 database that our analysis describes as putting dense vectors, sparse vectors, tensors and full text behind one query engine, shipping a single binary and a Python client, with the trade-off that you run a server process rather than importing a library. apache/doris is the analytical version, and our analysis describes it as an MPP SQL database for real-time analytics, lakehouse acceleration and hybrid search over structured, text and vector data.

The lexical side on its own, which is often the missing half. timescale/pg_textsearch adds a BM25 index access method and a scoring operator to PostgreSQL so relevance ranking happens in the database, and our analysis notes it supports PostgreSQL 17 and 18 and ships as source or prebuilt Linux binaries. xhluca/bm25s is the library form: our analysis describes it as implementing Okapi BM25 in Python with eager sparse scoring, so query time is mostly matrix lookups, along with where that design stops being the right choice.

Applications that put the two together. CortexReach/memory-lancedb-pro is an OpenClaw plugin storing preferences and decisions in LanceDB and recalling them in later sessions with hybrid vector plus BM25 retrieval, cross-encoder reranking and a management CLI, per our analysis, plus one hardware constraint to check before installing. weiwill88/Local_Pdf_Chat_RAG is the readable reference, running document parsing, chunking, embeddings, FAISS, BM25, reranking and generation as separate modules. TransformerOptimus/SuperCoder uses tree-sitter vectors, a call graph and BM25 together to move through a codebase structurally. And okf-memory/okf-agent-memory is the instructive counterexample: our analysis describes it storing an agent's project knowledge as Markdown under version control and retrieving it with an in-memory BM25 index, which is a good fit for memory you can diff in a pull request and a poor fit for anyone who needs semantic recall over unstructured notes.

In practice

Hybrid search is the standard answer to a corpus and a query set that are both mixed, and its two halves fail on disjoint cases, so the work is in the merge rather than in the choice. Normalise before you add scores, or use rank-based fusion and accept that margins are lost; keep the two indexes updating together; and make sure the analyzer suits the language and the content, because that is where a lexical half silently fails. meilisearch/meilisearch and oramasearch/orama make the decision for you, timescale/pg_textsearch and infiniflow/infinity give you the halves inside systems you already run, and xhluca/bm25s is the implementation to read when you need to know exactly what the lexical scoring is doing.