Local PDF Chat RAG: a readable hybrid retrieval pipeline you can run without a cloud account
Transparent Python RAG reference with FAISS + BM25 hybrid retrieval, reranking, Gradio UI, and FastAPI.
At a glance
- What is it?
- weiwill88/Local_Pdf_Chat_RAG is an MIT-licensed Python reference implementation of a RAG pipeline with FAISS plus BM25 retrieval, optional reranking, a Gradio UI and a FastAPI service. It is built for reading and modifying, not for serving a company knowledge base.
- Who is it for?
- Adopt it if you want to read a RAG pipeline end to end, or if you need a small local document question-answering loop over PDF, TXT, Markdown, DOCX, XLS/XLSX or PPTX files and are willing to keep credentials in a local .env. Do not adopt it as a multi-tenant knowledge service: the README states plainly that authentication, tenant isolation, persistence, evaluation and deployment governance are not included.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 16 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What problem Local PDF Chat RAG solves, and for whom
Most RAG code you find online is either a notebook that ends at a single vector search, or a framework where the retrieval path disappears behind an abstraction layer. Local PDF Chat RAG takes the opposite position. The README describes it as "a transparent, runnable Python implementation for learning and inspecting RAG", and the repository layout backs that up: document loading, chunking, embeddings, the FAISS index, the BM25 index, hybrid retrieval, reranking and generation each sit in their own file under core/. The intended reader is a developer who wants to see where a query goes after it leaves the UI, and who wants to swap one stage without rewriting the rest.
The second audience is narrower but real. If you have a folder of PDFs, spreadsheets and slide decks and you want to ask questions against them on your own machine, the project ships both a Gradio web application and a FastAPI service for that. The supported formats are PDF, TXT, Markdown, DOCX, XLS/XLSX and PPTX. The README is explicit that this is not a production knowledge-base service, so treat the two audiences as one: people who will read the code before they trust the answer.
The pipeline as the repository lays it out
The README's flowchart is the clearest statement of the architecture. Documents go through parsing, then chunking. From chunking the path forks: one branch produces embeddings that go into a FAISS index, the other produces a BM25 index. Both indexes feed a hybrid retrieval stage, which feeds reranking, then context building, then LLM generation, which returns an answer plus sources.
That fork is the design decision worth noticing. Dense retrieval from FAISS handles paraphrase and semantic similarity; BM25 handles exact terms, identifiers and rare tokens that embeddings tend to blur. Running both and merging is the standard remedy for the failure mode where a vector search returns plausible passages that do not contain the term you asked about. The README also mentions that core/retriever.py covers "hybrid and recursive retrieval", so the retrieval module does more than a single weighted merge, though the README does not document the recursion policy or the weighting scheme. That is a gap: the mechanism is in the file, not in the documentation.
Reranking is described as optional and supports either a CrossEncoder or model-based relevance scoring. Optional is the right framing. A CrossEncoder rescores candidate passages by reading query and passage together, which is slower per candidate than the retrieval step that produced them, so the cost scales with how many candidates you pass in. The README does not state a default candidate count or a default rerank setting, so check core/reranker.py and config.py before assuming what is on.
Getting it running: environment, backends, ports
The quick start is four steps. Clone the repository, create a virtual environment with Python 3.10 or later, upgrade pip, and install requirements:
git clone https://github.com/weiwill88/Local_Pdf_Chat_RAG.git cd Local_Pdf_Chat_RAG python3.10 -m venv .venv source .venv/bin/activate python -m pip install --upgrade pip pip install -r requirements.txt
Then copy example.env to .env and configure at least one model backend. The README lists three: set SILICONFLOW_API_KEY, or set MAGICK_API_KEY along with its endpoint and model name, or run Ollama locally and pull the model named in .env. The topics list on the repository also mentions deepseek and ollama, which is consistent with an OpenAI-compatible API path. One detail is easy to miss and worth repeating: values beginning with Your_ are treated as placeholders and are not valid credentials, so a half-edited .env fails rather than silently calling out.
The two entry points are separate processes. python rag_demo.py starts the Gradio web UI, which first tries http://127.0.0.1:17995 and then ports 17996 through 17999 if that one is taken. python api_router.py starts the FastAPI service, with GET /api/status for runtime and provider configuration, POST /api/upload to upload and process a document, and POST /api/ask to question processed documents. Tests are run separately after installing requirements-dev.txt, with PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest. The README states the suite covers configuration and default backend selection, TXT and Markdown and unsupported-file loading behaviour, BM25 and hybrid-result merging, and a clean network-free failure when an API key is missing. That last test is the one that tells you the project treats missing credentials as an expected state rather than a crash.
Where it stops being the right tool
The README's own warning is the honest starting point: add authentication, tenant isolation, persistence, evaluation, security controls and deployment governance before using it with real business data. Read that as a list of things the repository does not implement, not as a roadmap. Nothing in the material describes user accounts, per-user document scoping, a database for documents or conversations, or an evaluation harness that scores retrieval quality. If two teams share one deployment, they share one index.
The second limitation is more structural. The project is described as local and the default backend story is Ollama, but two of the three documented backends are hosted APIs. Choosing SiliconFlow or the MAGICK endpoint means your document chunks leave the machine. The repository name and the local-first framing can create a false expectation here, so decide deliberately which backend you configure.
Third, persistence and re-indexing are undocumented in the supplied material. The README does not say whether the FAISS index and BM25 index survive a restart, where they are written, or what happens to the two indexes when a document is deleted. Because the indexes are separate structures built from the same chunks, any document removal has to touch both, and the README gives no procedure for that. If your corpus changes often, verify this before you build on it. The last push to main was 2026-08-31 and the repository snapshot records 5 commits in the last 90 days, so the code is moving slowly enough that you should expect to read it rather than wait for a fix.
How it compares with an embedded vector store like Chroma
The obvious alternative for a small local document question-answering tool is Chroma, which also runs in-process and stores embeddings locally. The difference is what sits between your query and the answer. Chroma gives you a collection, an embedding function and a similarity query, and leaves hybrid retrieval, reranking and context assembly to you or to a framework above it. Local PDF Chat RAG ships those stages as separate, named modules in core/, so the merge of dense and keyword results and the rerank step are visible code you can read and change rather than behaviour you configure by hoping.
That cuts both ways. Chroma has a persistence model and a client API that are documented as a product; Local PDF Chat RAG's index lifecycle is not documented in the material available here. If your priority is a stable storage layer with a documented API, Chroma is the safer base and you would add BM25 and a CrossEncoder yourself. If your priority is seeing how the retrieval stages fit together before you commit to a stack, this repository is the shorter path. The choice is about whether you want a component or a worked example.
Maintenance, releases and the MIT licence
The release history is short and legible. v2.0.0, dated 2026-03-18, is labelled as a modular refactor with bug fixes, which matches the current core/ layout. v2.1.0, dated 2026-08-12, is labelled as OSS readiness and bilingual documentation. The README is available in English and Simplified Chinese, and the repository carries contribution guidance, issue forms, a pull request template, GitHub Actions CI, and a documented private security-reporting process. CI compiles the Python sources and runs the credential-free tests on pull requests, which is the right scope for a project whose tests must not need API keys.
The licence is MIT, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is a summary of the licence text, not legal advice; read the LICENSE file and your own organisation's policy before shipping anything derived from it. MIT also means there is no warranty, so the maintenance burden lands on you. Practically, that burden is the cost of tracking changes in FAISS, the BM25 implementation, the embedding model, and whichever provider API you configure, plus the cost of keeping the two indexes consistent when your documents change. Because the project has no dependency-pinning story in the supplied material, check requirements.txt for version ranges before you assume an upgrade will be uneventful.
Who should take it, and what to check first
Take it if you are learning RAG and want a pipeline short enough to read in an afternoon, or if you want a local question-answering loop over your own PDFs and office documents and you are comfortable keeping credentials in a local .env file. The presence of a Gradio UI and a FastAPI service in the same repository means you can start with the browser and move to HTTP calls without changing the retrieval code underneath.
Leave it if you need multi-user access control, per-tenant document separation, durable storage with a documented schema, or retrieval quality metrics. None of those are in the material, and the README says so directly. Leave it too if your documents cannot leave your network and you were planning to use a hosted backend, because two of the three documented options are remote.
Before you commit, do three things in this order. Run the test suite with PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest to confirm the credential-free path works on your machine. Configure exactly one backend in .env and check GET /api/status reports it as active, since a placeholder value beginning with Your_ is rejected rather than used. Then read core/retriever.py and core/reranker.py to see how hybrid results are merged and whether reranking is on by default, because those two files determine answer quality far more than the choice of interface does.
Editorial conclusion
Adopt it if you want to read a RAG pipeline end to end, or if you need a small local document question-answering loop over PDF, TXT, Markdown, DOCX, XLS/XLSX or PPTX files and are willing to keep credentials in a local .env. Do not adopt it as a multi-tenant knowledge service: the README states plainly that authentication, tenant isolation, persistence, evaluation and deployment governance are not included. Before committing, verify that your chosen backend works from your network, then read core/retriever.py and core/reranker.py to confirm the hybrid merge and rerank behaviour match what you expect from your own documents.
Community notes