Raudaschl/rag-fusion: multi-query generation plus Reciprocal Rank Fusion, with an evaluation harness attached

The problem: one phrasing of a query is one roll of the dice

A single vector search over an embedded corpus returns neighbours of one string. If the user writes "phone holder thing for car" and the index contains "magnetic vent mount", the embedding may or may not bridge that gap. The README frames the target case as a vocabulary mismatch between how users ask and how the corpus is indexed, naming lay versus technical names, jargon and paraphrase as the failure modes. The repository's one-line positioning statement claims the properly configured variant, described as hybrid_diverse+rerank, produces better retrieval rankings and better generated answers than baseline retrieval at proper sample sizes with confidence intervals, across difficulty buckets, even with a strong reranker. The same line states plainly that the vector-only fusion variant is a different story: roughly a wash on average and net-negative on rich queries at the answer level. That asymmetry is the most useful thing in the README, and it is the reason the project ships an evaluation harness rather than just a pipeline. The intended audience is someone with a retrieval problem where recall matters more than precision, not someone building a FAQ bot.

The four stages, as the flowchart lays them out

The pipeline diagram in the README has four boxes. An original query goes to an LLM, which generates multiple query variations. Each variation is run as a separate vector search through ChromaDB. The ranked lists from those searches are combined by Reciprocal Rank Fusion, which boosts documents that appear consistently across multiple query perspectives. The final stage produces a re-ranked document list, optionally synthesised into a natural-language answer by an LLM. The mechanism that matters is the third stage. RRF does not compare similarity scores across searches, because those scores are not commensurable when the queries differ. It combines ranks. A document that lands fifth for one rewrite and third for another accumulates more fused weight than a document that tops exactly one list. That is why the README describes the technique as casting a wider net: the cost is extra retrievals and extra LLM tokens, and the payoff is candidates that a single phrasing would have missed. Note that the flowchart shows vector search only. The hybrid BM25 plus vector variant that the README recommends for production appears in the evaluation code, not in the top-level diagram.

Getting it running: five packages, one env file, one command

The install line in the README is a single pip command: pip install openai chromadb python-dotenv tqdm tabulate rank_bm25. Copy .env.example to .env and replace the placeholder your-key-here with an OpenAI API key, since query generation calls GPT. Then python main.py runs the demo. The unit tests run without a key: python -m pytest test_main.py -v. That split is deliberate and worth noting, because it means CI or a local checkout can exercise the fusion logic without spending tokens. The evaluation path is a separate CLI, evaluate.py, described in the project structure as covering baseline plus fusion variants with an optional --rerank flag. The eval package contains the pieces you would need to extend it: dataset.py for NFCorpus download and loading, metrics.py for Precision, Recall, NDCG and MRR, retrieval.py for BM25, vector, hybrid and the RAG-Fusion variants, rerank.py for the cross-encoder stage, and query_cache.py, a disk-persisted cache for LLM query rewrites. That cache is the difference between a sweep being affordable and being absurd, since a pool-size and N-rewrites sweep multiplies the number of LLM calls.

The evaluation harness is the actual product

Most retrieval repositories ship a pipeline and a paragraph of assertion. This one ships a directory of drivers: sweep.py for pool size and rewrite count, steelman.py for pipeline ordering, truncation and difficulty tests, qualitative.py for verbose answer logs, answer_eval.py for LLM-judge answer scoring, bootstrap_ci.py for paired-bootstrap confidence intervals on per-query metrics, saved_queries.py for a binary recovery metric the README calls kohlrabi-class, and eval_with_ci.py for a retrieval-only headline table with intervals. The dataset is NFCorpus from BEIR: 3,633 medical and nutrition documents, 323 test queries with graded relevance judgments. The README's headline table is explicitly retrieval-only, with no cross-encoder reranking and no answer-quality evaluation, and it is labelled a quick visual rather than the production comparison. The fuller writeup lives under experiments/arxiv-2603-02153-replication/, which the README says contains n=200 paired-bootstrap confidence intervals, three rerankers, six fusion variants, an end-to-end LLM-judge answer evaluation, and a replication of arXiv 2603.02153v1. Treat the retrieval-only table as a smoke test and the replication directory as the evidence. The distinction between the two is stated clearly enough that you can decide which one to trust for your use case.

Where fusion loses: the wrong-tool list is in the README

The README lists poor-fit cases directly, which is rarer than it should be. FAQ chatbots and curated customer-support knowledge bases are poor fits, presumably because the corpus is small and the vocabulary is already aligned with the questions. Latency-critical retrieval is a poor fit: voice, autocomplete, sub-second p95 chat. That follows from the mechanism. Every query pays for at least one LLM call to generate rewrites before any retrieval happens, and the rewrite cache only helps on repeated queries. High-volume, margin-thin consumer search is a poor fit for the same reason. Code or identifier search is precision-dominated, so widening the candidate set works against you. Structured data, knowledge graphs and SQL-backed retrieval are out of scope entirely. The README also names the condition that decides whether the technique earns its compute: the downstream consumer must be able to handle topically-broad context, either a strong synthesis LLM or a UI that surfaces multiple candidates instead of one canonical answer. If your interface shows a single answer to a user who will not read a list, fusion's recall gain has nowhere to go. For mixed workloads the README recommends adaptive routing: run baseline plus rerank on every query, and fire fusion only when a cheap weakness signal trips. That is a design recommendation from the authors, not a shipped router; nothing in the project structure suggests a routing component exists in the repository.

What to compare it against, and what actually differs

The honest baseline is not another fusion library. It is a single vector search over ChromaDB with a cross-encoder reranker on top, which is what the README's baseline plus rerank configuration already is, and which the project's own comparison treats as the reference point. The difference in approach is where the extra compute goes. A plain reranked pipeline spends its budget on scoring the top candidates from one query with a more expensive model. RAG-Fusion spends it upstream, on generating query variants and retrieving more lists, and only then optionally reranks. The README's claim is that fusion still helps even with a strong reranker present, which is the interesting case, because it implies the gain comes from candidates the single query never surfaced rather than from better ordering of the same set. If that claim does not hold on your corpus, the reranked single-query pipeline is strictly cheaper. The other comparison worth making is against BM25 alone. The repository ships rank_bm25 and a hybrid retrieval path, and the recommended production configuration is hybrid, meaning lexical matching is part of the recipe. If your corpus has strong exact-term structure, a tuned BM25 setup with a reranker is a real alternative that costs no LLM calls at query time.

Maintenance surface, dependencies and the MIT licence

The dependency list is short and mostly boring in a good way: openai, chromadb, python-dotenv, tqdm, tabulate and rank_bm25. The last push recorded for the repository is 2026-04-26, and no releases were retrieved, so there is no versioned artifact to pin against. Installation is from source via pip on the listed packages. The practical maintenance cost sits in the eval and experiments directories rather than in main.py. The pipeline itself is a few hundred lines by the look of the structure; the harness has around a dozen modules, and the replication writeup carries raw results. Anything that changes the OpenAI client or the ChromaDB API touches query generation and vector search, which are the two external services the pipeline depends on. The licence is MIT, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are included. That is a statement about the licence text, not legal advice; if you are shipping this inside a product, have counsel read the actual LICENSE file and the OpenAI terms that govern the API calls, since the LLM dependency carries its own conditions independent of the repository licence.

Editorial conclusion

Adopt this if your retrieval problem is terminology mismatch over a specialist corpus and you can afford an LLM call per query for rewrites. Skip it for FAQ bots, autocomplete, or identifier search, where the README itself lists fusion as a poor fit. Before committing, run evaluate.py against your own corpus and check whether the hybrid variant holds up there, because the README states the vector-only fusion variant is roughly a wash on average and net-negative on rich queries at the answer level.

Official sources

1. Issues
2. License: MIT
3. Raudaschl/rag-fusion on GitHub
4. README