ollama_pdf_rag: A Local RAG Stack With Two Frontends and One FastAPI Contract
A full-stack demo showcasing a local RAG (Retrieval Augmented Generation) pipeline to chat with your PDFs.
At a glance
- What is it?
- The repository ships a Next.js UI, a Streamlit UI, a FastAPI backend and Jupyter notebooks over a shared ChromaDB and Ollama pipeline. The judgement: adopt it as a reference architecture or an air-gapped demo, not as a hardened document service.
- Who is it for?
- Adopt it if you want a working local PDF chat on your own machine and you are comfortable reading src/core/rag.py to understand what the retrieval step actually does. Do not adopt it as a multi-tenant document service: there is no authentication layer described anywhere in the README, and the vector store is a local ChromaDB directory under data/vectors.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 152 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap ollama_pdf_rag fills: PDFs that never leave the machine
Most document chat tools assume you will send page text to a hosted model. This project inverts that assumption. The README states that all processing happens on your machine and that no data leaves it, which is the entire reason the repository exists in this shape. The intended user is someone with a folder of PDFs and a machine capable of running Ollama locally, who wants question answering with citations rather than a summarization script. The repository is a demo, and the README calls it that in the project description. It is aimed at engineers evaluating a local RAG pattern, at people following the linked YouTube tutorial, and at anyone who needs a starting skeleton they can fork. It is not aimed at teams that need ingestion at scale or an access-control model, and nothing in the material suggests it was built for that.
How the pipeline is wired: FastAPI services, a ChromaDB directory, and two clients
The layout under src/ splits the RAG work into four modules: document.py handles PDF processing, embeddings.py produces vectors, llm.py configures the model, and rag.py assembles the pipeline. The README describes the retrieval as multi-query RAG with source citations, which means the question is expanded into several queries before retrieval, and the interface surfaces the retrieved chunks alongside the answer. Storage is ChromaDB, written to data/vectors, while uploaded PDFs land in data/pdfs. The API layer is separate: src/api/routers holds endpoints, src/api/services holds business logic, and src/api/main.py is the entry point. The Next.js app under web-ui talks to that API rather than importing Python, which is why the recommended setup runs two processes on ports 8001 and 3000. The Streamlit app under src/app is a second client over the same core. That is the architectural bet: one Python core, one HTTP contract, several frontends. The v3.0.0 release notes name exactly this split, calling it the Next.js UI, FastAPI backend and enhanced RAG.
Getting it running: the commands and the ports
The prerequisites are explicit. Install Ollama, then pull two models: ollama pull llama3.2 for chat and ollama pull nomic-embed-text for embeddings. Clone the repository, create a virtual environment with python -m venv venv, activate it, and run pip install -r requirements.txt. For the modern UI you also need the frontend dependencies: cd web-ui, pnpm install, pnpm db:migrate, then cd back. Note that pnpm db:migrate implies a database owned by the Next.js side, separate from ChromaDB, and the README does not spell out which database that is, so treat it as something to confirm before you run it. Starting the stack is two terminals: python run_api.py on port 8001 and cd web-ui && pnpm dev on port 3000, or the bundled ./start_all.sh. The Streamlit path is a single command, python run.py, on port 8501. Swagger UI is served at http://localhost:8001/docs. The API surface is small and worth reading before you build anything on top: POST /api/v1/pdfs/upload, GET /api/v1/pdfs, DELETE /api/v1/pdfs/{pdf_id}, POST /api/v1/query, GET /api/v1/models and GET /api/v1/health. Tests run with python -m pytest tests/ -v, and coverage with python -m pytest tests/ --cov=src.
Where this stack breaks: chunking, memory, and the empty retrieval case
The troubleshooting section is the most honest part of the README. It lists four failure modes: Ollama not responding, a model not found, no chunks retrieved, and port conflicts on 3000, 8001 or 8501. The third one matters most. When retrieval returns nothing, the documented fix is to re-upload the PDFs to rebuild the vector database. That means the index is not self-healing and there is no documented reindex command or migration path for the vector store; you rebuild by re-uploading. On CPU-only machines the README advises reducing chunk_size to 500-1000 in src/core/document.py, which tells you the default chunking is tuned for machines with more headroom and that large documents can exhaust memory. There is a Windows-specific failure documented as well, a DLL load error while importing onnx_copy2py_export, resolved by installing the Microsoft Visual C++ Redistributable. The wrong tool test is straightforward: if you need per-user document isolation, background ingestion of a large corpus, or an audit trail of who queried what, none of that appears in the material. The API has no authentication described, and the vector store is a local directory.
How it differs from LlamaIndex and from hosted document chat
LlamaIndex targets the same problem from the opposite direction. It is a library first: you assemble ingestion, indexing and query engines in your own application, and you choose the storage and serving layers yourself. This repository is an application first. The retrieval logic is already written into src/core/rag.py, the endpoints already exist, and the UIs already render citations. The trade is control for time. With LlamaIndex you would write the FastAPI routes, pick the vector store, and design the chunking strategy; here those decisions are pre-made and you inherit them, including the multi-query expansion and the ChromaDB choice. Against a hosted document chat product the difference is the constraint itself: no page text is sent to a third party, so you trade model quality and zero-ops for local control. The README makes no accuracy claim relative to hosted models, and it should not be read as one.
Maintenance surface and the MIT licence in practice
The cost of running this is the cost of four things staying in sync: Ollama and its pulled models, the Python dependencies in requirements.txt, the pnpm workspace under web-ui, and the ChromaDB directory in data/vectors. The release history shows the shape of that cost. v2.0 arrived in November 2024, v2.1.0 in January 2025 added testing infrastructure and reworked the project architecture, and v3.0.0 in December 2025 restructured around the Next.js and FastAPI split. Each of those is a structural change, not a patch, so a fork pinned to v2.x will not track cleanly onto v3.0.0. The testing story is a pytest suite plus pre-commit hooks, which covers the Python side; the README does not describe an equivalent test setup for the Next.js frontend. The licence is MIT, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are preserved. That is a statement about the licence text, not legal advice about your situation; if you embed this in a product, read the MIT terms yourself and check the licences of the models you pull, since model licences are separate from the code licence.
Editorial conclusion
Adopt it if you want a working local PDF chat on your own machine and you are comfortable reading src/core/rag.py to understand what the retrieval step actually does. Do not adopt it as a multi-tenant document service: there is no authentication layer described anywhere in the README, and the vector store is a local ChromaDB directory under data/vectors. Before you commit, verify three things on your own hardware: that ollama pull nomic-embed-text completes and the embedding model answers, that pnpm db:migrate in web-ui finishes against your database, and that a single PDF upload returns chunks rather than the empty retrieval the troubleshooting section warns about.
Community notes