pg_bestmatch.rs

A Postgres extension that builds BM25 sparse vectors from your text, drops them into pgvecto.rs or pgvector, and is aimed at retrieval work where dense vectors fall short.

pg_bestmatch.rs is a PostgreSQL extension for BM25 text queries. It turns text into BM25 statistic sparse vectors, the kind used in retrieval pipelines where BM25 can beat dense vector based methods on many RAG benchmark tasks. The README positions it as a complement to search rather than a replacement: you still use pgvecto.rs or pgvector to actually search the vectors in Postgres.

The workflow starts with bm25_create, which builds BM25 statistics for a table and column and records them in a materialized view. From there, bm25_document_to_svector turns a passage into a sparse vector and bm25_query_to_svector does the same for a query, and the score comes from a dot product between the two. Tokenization currently runs through the HuggingFace tokenizer with the bert-base-uncased vocabulary, and the README notes that more tokenizer configuration might arrive later.

An important note in the README says HNSW indexing does not support these sparse vectors very well, because the high sparsity prevents effective navigation within the graph. In practice that means the vector search extensions handle the indexing, and HNSW is not the way to speed these up. It is a limitation worth knowing before you design around it.

A usage example walks through the Stanford LoCo benchmark, from loading the dataset to adding an embedding column and updating embeddings for documents and queries. The README reports a Top 1 recall of 0.77 on that dataset and says reproducing the result means your operations are correct. The reference section lists the functions: create, with BM25 parameters b defaulting to 0.75 and k defaulting to 1.2, refresh to update stats, drop to delete them, and the two to_svector conversions that emit either pgvecto.rs style or pgvector style sparse vectors. A comparison with pg_search notes that pg_bestmatch.rs only generates sparse vectors and leaves index based search to the vector extensions, while staying native to Postgres for full compatibility with transactions, filters, and JOIN operations.

The README reads like a focused tool: generate the stats, produce the sparse vectors, let pgvecto.rs or pgvector do the searching. The HNSW warning is the one real gotcha for anyone planning to index these vectors.

• Source 1 · Project repository
• Source 2 · README