FlashRAG installs 22 packages before any extra, and its newest tag has no changelog entry
⚡FlashRAG: A Python Toolkit for Efficient RAG Research (WWW2025 Resource)
At a glance
- What is it?
- FlashRAG is a Python toolkit for reproducing RAG research: 36 pre-processed benchmark datasets, 23 algorithms, 7 reasoning methods. Its packaging is generous to a fault, and the release history has drifted away from the changelog.
- Who is it for?
- FlashRAG fits a research group that wants published benchmark numbers back on a known dataset rather than a team that just needs a retrieval pipeline this afternoon. The dataset collection, the 23 pre-implemented methods and the evaluation scripts are the reason to choose it; the dependency surface is the cost, and it is charged to everyone at import time.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 20 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- RUC-NLPIR/FlashRAG3,606 ★ · +26
| Repository | 2026-09-24 | 2026-11-15 |
|---|---|---|
| RUC-NLPIR/FlashRAG | 3.6k | 3.6k |
26 Stars in the last 52 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
install_requires points at the whole requirements list, so the extras only add
The README's navigation links to Installation and Quick Start, but the page's body stops inside its Chunkie changelog entry before either heading, so the install steps are not written on the page. What the repository does offer is setup.py, and it decides everything: requirements.txt is read at build time and handed to install_requires, with four extras declared on top.
extras_require = {
'core': requirements,
'retriever': ['pyserini', 'sentence-transformers>=3.0.1'],
'generator': ['vllm'],
'multimodal': ['timm', 'torchvision', 'pillow', 'qwen_vl_utils']
}
extras_require['full'] = sum(extras_require.values(), [])Because core is the entire requirements file, an extra adds to the base rather than trimming it. A reader who wants only pyserini and sentence-transformers for a retriever experiment still gets torch, spacy, fschat, streamlit, gradio and peft, because those sit in core. The full extra is the concatenation of all four lists, computed before it is inserted, so it stays a plain list. Two more details matter for packaging: package_data globs *.yaml and include_package_data is on, which is how component configs travel inside the installed package, and the version string is exec'd out of flashrag/version.py rather than written here. python_requires is >=3.9. The distribution is named flashrag_dev, not flashrag.
bm25s is held at one version, chonkie at a range, and transformers has no ceiling
requirements.txt holds twenty-two entries, and only three carry an upper bound. Two are precise:
bm25s[core]==0.2.1
chonkie>=1.0.2, <1.1.0The third is transformers>=4.40.0, a floor with nothing above it. torch, peft, fschat, spacy and the rest float entirely. For a toolkit whose stated purpose is reproduction, that split is worth noticing: the retrieval index layer is nailed to a single bm25s release and the chunking library is capped below 1.1.0, while the model stack, where API drift usually shows up first, is left open. Rebuilding an environment from this file resolves a different transformers and a different torch than the ones behind a paper's numbers, and the pinned bm25s holds retrieval behaviour steady in the middle of that churn. Chunkie is the chunker behind that cap; the changelog says it was integrated on 2025-01-07 and supports token, sentence and semantic chunking.
pyproject.toml adds nothing to the picture. It contains a single ruff lint table whose extend-select adds W291 and W293, the two trailing-whitespace rules, and no build-system or project metadata table, so setup.py and setuptools still own the build.
The features text names Faiss for vector indexes and no extra installs it
The feature list credits three names for speed: vLLM and FastChat for inference acceleration, and Faiss for vector index management. Two of the three arrive, in different ways. fschat is in core, so it is always installed. vllm is not in requirements.txt at all; it appears only in the generator extra, so a reader running a lexical pipeline against a hosted endpoint never pulls it. Faiss appears nowhere: not in requirements.txt, not in the retriever extra, not in the multimodal extra. The dense index path the page advertises depends on the reader sourcing Faiss themselves.
That is where the toolkit offers two genuinely different retrieval approaches. The lexical one, through rank_bm25 and bm25s, is installed by default and version-pinned, and the retriever extra adds pyserini with sentence-transformers>=3.0.1, which is the same sentence Transformers integration the roadmap counts as done. The dense one is named in prose and left to you. For a benchmark reproduction the difference matters more than the choice itself: the lexical path is deterministic and needs no GPU, while a vector index needs an embedding model and an index backend you have to add by hand.
Two web frameworks sit in core, so the library ships gradio and streamlit
FlashRAG-UI arrived in the changelog entry dated 2025-01-12, described as an interface for configuring the supported RAG methods and running evaluation scripts visually. The two frameworks it needs are listed in the base requirements, not in an extra: streamlit with no version constraint, and gradio>=5.0.0 with a floor and no ceiling. A user who only imports the retrieval code, on a headless training box, pays for both.
The repository layout splits along the same lines. webui/ sits next to flashrag/, tests/, scripts/ and docs/, and examples/ holds methods/, quick_start/, run_mm/, run_refiner.py and multi_turn.py, so the evaluation paths exist as files rather than only as interface state. indexes/, models/, datasets/ and results/ are top-level directories too, which means the toolkit expects to read or write indexes, checkpoints, corpora and result tables inside the working tree. Nothing in the packaging description says how any of those directories get populated. The root also carries a CITATION.cff, a LICENSE and a README_zh.md alongside the English README.
The changelog stops at 2025-11-06 and the newest tag is v0.4.0 from 2026-10-01
Three releases exist. v0.2.0, dated 2025-03-10, is named for multimodal RAG support. v0.3.0, dated 2025-08-18, is named for reasoning methods. v0.4.0, dated 2026-10-01, is named v0.4.0 and nothing else. The changelog's newest entry is dated 2025-11-06 and describes a web search retriever that needs a Serper API key, which lands between v0.3.0 and v0.4.0. Nothing on the page describes what v0.4.0 changed, and the release list and the changelog disagree about how recent the work is by eleven months.
The last push to the default branch is dated 2026-10-01, the same day as the v0.4.0 tag, so branch and tag moved together while the changelog stayed behind. Older entries are consistent with the tags: the reasoning pipeline on 2025-08-06 sits just before v0.3.0, and the multimodal entry on 2025-02-24 sits just before v0.2.0. Two other entries point into docs/original_docs rather than describing behaviour on the page, the RQRAG addition on 2025-01-11 and multi-retriever aggregation on 2025-01-07. The paper linked as the project homepage, arXiv 2405.13576, was accepted to the resource track of WWW 2025 in January 2025.
The open roadmap item is an api-based retriever that no extra provides
Five roadmap boxes are ticked: OpenAI models, instructions for each component, sentence Transformers integration, multimodal RAG, and reasoning-based methods. Three are open, and two read as unfinished work rather than planned features. One is "Enhance code adaptability and readability", which is a statement about the code rather than a plan. The other is "Add support for api-based retriever (vllm server)".
That item is worth reading twice, because vllm is already an extra. What is missing is retrieval served over HTTP from a vLLM server, a different capability from using vLLM as the generator backend. Until it lands, a distributed retrieval setup means running something outside the toolkit. The ticked box about instructions for each component points at documentation you reach through the docs/ tree rather than through the page, and the list itself is edited by hand, since two entries are misspelled: "Provdide instructions for each component" and "Inlcude more RAG approaches".
One F1 figure appears on the whole page, inside a changelog line
The toolkit advertises 36 pre-processed benchmark datasets, 23 pre-implemented RAG algorithms and 7 reasoning-based methods. The 23-algorithm milestone is the entry dated 2025-03-21, and the pipeline that ties the reasoning group together came on 2025-08-06, naming R1-Searcher and Search-R1 as the upstream work it covers. The only number attached to any of it sits in that same line: F1 scores close to 60 on multi-hop inference datasets such as HotpotQA. The multimodal entry from 2025-02-24 lists Llava, Qwen and InternVL as generators and Clip-architecture retrievers, with no figures either.
So the performance case rests on a single changelog sentence and a result table the page links without reproducing. The corpora are distributed off-repository, as a Hugging Face collection named FlashRAG_datasets and a ModelScope dataset named FlashRAG_Dataset, which means reproducing a published figure also means trusting a hosted copy of the corpus rather than a pinned digest in the tree. The results/ directory at the root is where the output side of that comparison is meant to land.
Editorial conclusion
FlashRAG fits a research group that wants published benchmark numbers back on a known dataset rather than a team that just needs a retrieval pipeline this afternoon. The dataset collection, the 23 pre-implemented methods and the evaluation scripts are the reason to choose it; the dependency surface is the cost, and it is charged to everyone at import time. Read setup.py before installing anything, because the extras you choose do not reduce the base list. Then check the changelog against the tag list: the newest release is v0.4.0 dated 2026-10-01 and the last documented change is dated 2025-11-06, so nothing on the page tells you what v0.4.0 contains.
Frequently asked questions
How do I install FlashRAG with only the retriever components?
The extras are declared in setup.py as core, retriever, generator and multimodal, and core is the entire requirements.txt. The retriever extra adds pyserini and sentence-transformers>=3.0.1 to the base list, so the twenty-two packages in core, including torch, spacy, fschat, streamlit and gradio>=5.0.0, are installed either way.
Which Python versions does FlashRAG support?
setup.py declares python_requires as >=3.9. The distribution is named flashrag_dev rather than flashrag, and the version is read out of flashrag/version.py at build time rather than written in setup.py.
What is in the FlashRAG v0.4.0 release?
The tag is dated 2026-10-01 and carries no description beyond its version number. The changelog's most recent entry is dated 2025-11-06 and covers a web search retriever that needs a Serper API key, so the page does not say what v0.4.0 changed.
Does a default FlashRAG install bring Faiss and vLLM?
Neither is in requirements.txt. vllm appears only in the generator extra, and Faiss, which the feature list names for vector index management, appears in no extra at all. The lexical path through rank_bm25 and bm25s is the one the base install covers.
Which benchmark results does FlashRAG publish?
One figure appears in the changelog for the reasoning pipeline: F1 scores close to 60 on multi-hop inference datasets such as HotpotQA. The page names 36 pre-processed datasets and 23 algorithms and links a result table without reproducing the numbers.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/ruc-nlpir-flashrag)Star history chart for your README
[](https://hysenlabs.com/projects/ruc-nlpir-flashrag)Was this useful?