RAGLite: DuckDB or PostgreSQL as the Retrieval Store, With Late Chunking Built In
🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL
At a glance
- What is it?
- RAGLite is a Python RAG toolkit that keeps its vector and keyword search inside DuckDB or PostgreSQL instead of a separate vector service. It bundles PDF-to-Markdown conversion, late chunking, hybrid search, an MCP server and an optional Chainlit frontend, and it deliberately avoids PyTorch and LangChain as dependencies.
- Who is it for?
- Adopt RAGLite if you already run PostgreSQL or want a single-file DuckDB store and you are willing to accept LiteLLM as your model abstraction layer. Do not adopt it if you need a managed vector service with independent scaling, or if you cannot install an accelerated llama-cpp-python wheel for local models.
- Can I use it commercially?
- Yes, with conditions. MPL-2.0 is a weak copyleft licence: you can use it inside commercial and closed-source software, but if you distribute changes to its own files, you must publish those changes under the same licence.
- Is it still maintained?
- Yes. The repository last received commits 30 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem RAGLite solves: keeping retrieval inside the database you already run
Most RAG stacks add a second data store. Documents live in PostgreSQL or a file share, embeddings live in a dedicated vector service, and the two drift apart because nothing keeps them in the same transaction. RAGLite takes the opposite position. The README states that you choose either DuckDB or PostgreSQL as a keyword and vector search database, and the hybrid search implementation uses the database's native facilities: FTS plus VSS on DuckDB, tsvector plus pgvector on PostgreSQL. There is no external vector service in the described architecture.
That choice defines the audience. If you are a Python developer building an internal document question-answering tool and you already operate PostgreSQL, RAGLite lets you add retrieval without provisioning another piece of infrastructure. If you are prototyping on a laptop, DuckDB gives you a single-file database with no server. The project also targets teams that care about dependency weight: the README says RAGLite uses only lightweight and permissive open source dependencies, explicitly naming PyTorch and LangChain as things it does not pull in. That is a real constraint on install size and on the number of transitive packages you have to audit, and it is stated as a design goal rather than a side effect.
How the pipeline is assembled: chunking, embedding, hybrid retrieval, adaptive generation
The documented flow starts at ingestion. PDFs are converted to Markdown on top of pdftext and pypdfium2, and the pandoc extra extends input support to filetypes other than PDF. Mistral OCR is offered as an optional higher-quality path for PDFs, images, DOCX and PPTX, with automatic image descriptions, but it requires installing the mistralai package separately.
Chunking is where the project spends most of its design effort. RAGLite performs multi-vector chunk embedding with late chunking and contextual chunk headings. Sentence splitting is handled by wtpsplit-lite, which the README describes as solving a binary integer programming problem to find an optimal split. Semantic chunking is described the same way, as another binary integer programming problem. This is an unusual mechanism: instead of a greedy character or token window, the split points are chosen by an optimiser. The practical consequence is that chunk boundaries are computed rather than fixed, and that computation has a cost that scales with document length. The README does not publish timing figures, so the only way to know whether that cost is acceptable is to run it on your own documents.
Retrieval is hybrid, combining keyword and vector search, and is followed by reranking. RAGLite delegates reranking to the rerankers library, with multilingual FlashRank as the default. Generation is adaptive: the README links adaptive retrieval, where the LLM decides whether to retrieve at all and what to retrieve based on the query. Two prompt-shaping details are documented: a prompt caching-aware message array structure intended to improve cost and latency, and Anthropic's long-context prompt format intended to improve output quality. Both are described in the README as improvements, without published measurements.
Query adapter: a closed-form correction fitted to your embeddings
The query adapter is the part of RAGLite that has no equivalent in most toolkits. The README describes it as an optimal closed-form linear query adapter, computed by solving an orthogonal Procrustes problem, and points at src/raglite/_query_adapter.py as the implementation. The distinction that matters is closed-form. This is not a trained neural adapter with a gradient descent loop and a checkpoint to version. It is a linear map derived analytically from paired query and document embeddings.
That has two consequences the README does not spell out. First, you need a set of query-document pairs to fit it, which means it belongs after you have some retrieval data, not before. Second, because the solution is closed-form, refitting is cheap relative to training, so the adapter can be recomputed as your corpus changes. The README lists it as step 4 of the usage overview, after inserting documents and running RAG, which is consistent with that reading. Whether the adapter helps on a given corpus is an empirical question, and the README presents the method rather than a result.
Getting it running: pip extras, LiteLLM model strings, and the llama.cpp escape hatch
The base install is a single command: pip install raglite. Optional capability comes through extras. pip install raglite[chainlit] adds the customizable ChatGPT-like frontend, which the README says can be deployed to web, Slack and Teams through Chainlit. pip install raglite[pandoc] adds conversion of input documents other than PDF to Markdown. pip install raglite[ragas] adds evaluation of retrieval and generation performance. Mistral OCR support is not an extra; it is enabled by installing mistralai directly.
Configuration starts with RAGLiteConfig, imported from raglite. The README shows a remote config example and states that you can use any LLM supported by LiteLLM, and that you can create a PostgreSQL database in a few clicks at neon.tech. For local models, RAGLite extends LiteLLM with llama.cpp support through llama-cpp-python, and the model identifier takes the form llama-cpp-python/<hugging_face_repo_id>/<filename>@<n_ctx>, where n_ctx is optional and sets the context size.
The README's most concrete operational warning concerns local inference. It recommends installing an accelerated llama-cpp-python precompiled binary and gives the environment variables that select it: LLAMA_CPP_PYTHON_VERSION, PYTHON_VERSION, ACCELERATOR (metal, cu121, cu122, cu123 or cu124) and PLATFORM (macosx_11_0_arm64, linux_x86_64 or win_amd64). The README flags that not every combination is available. The wheel URL is built from those variables, so a wrong pairing produces a failed install rather than a slower one. Acceleration is documented for Metal on macOS and CUDA on Linux and Windows.
Where RAGLite stops being the right tool
The database decision is also the main limitation. Because keyword and vector search run inside DuckDB or PostgreSQL, retrieval capacity is bounded by the database you chose. DuckDB is a single-node, file-oriented engine; the README presents it as a database option, not as a distributed one. A team that needs to scale retrieval independently of its application database, or that wants a managed vector service with its own operational story, is fighting the design rather than using it.
PostgreSQL has its own constraint. The hybrid path depends on tsvector and pgvector, so the PostgreSQL instance must have both available. Managed providers differ in what they expose, and the README does not enumerate which ones work. That is a verification step, not a documented guarantee.
The chunking pipeline is the second place to look carefully. Sentence splitting and semantic chunking are both framed as binary integer programming problems. Optimisation-based splitting is more expensive than a fixed window, and the README publishes no throughput numbers, no memory figures and no guidance on document size limits. For a corpus of long PDFs, ingestion time is the first thing to measure.
Finally, the dependency posture cuts both ways. Avoiding PyTorch keeps the install light, but it means the embedding and reranking options are those reachable through the chosen libraries rather than the full PyTorch ecosystem. If your organisation has standardised on a sentence-transformers model that is only distributed through PyTorch, the README does not describe a path for using it.
Alternatives and the actual difference in approach
The nearest alternative in spirit is a LangChain or LlamaIndex retrieval pipeline backed by a dedicated vector store such as a standalone vector database. The difference is architectural, not cosmetic. Those frameworks treat the vector store as a pluggable backend behind a retriever interface, which lets you swap in a managed service and scale it separately from your relational data. RAGLite collapses that layer into the database, so there is one system to operate and no synchronisation problem between documents and embeddings, at the cost of the scaling and backend flexibility the other design provides. The README names LangChain explicitly as a dependency RAGLite avoids, which confirms the comparison is intended.
For teams that want a hosted retrieval API, the trade is the reverse: you give up the ability to inspect and tune chunking, and you accept per-query pricing. RAGLite's chunking controls, late chunking and the query adapter are only meaningful if you are running the pipeline yourself.
A narrower alternative is to keep PostgreSQL and skip the toolkit, wiring pgvector and tsvector by hand. That is viable and removes a dependency, but you would be reimplementing the chunking optimiser, the reranker integration and the MCP server. The honest question is whether you want to own that code.
Interfaces, licensing and what upgrading costs
RAGLite ships two interfaces beyond the Python API. The README describes a built-in Model Context Protocol server that any MCP client, such as Claude desktop, can connect to, which means the retrieval index can be exposed to an assistant without writing a custom integration. Separately, the Chainlit extra provides a ChatGPT-like frontend deployable to web, Slack and Teams. Both are optional; neither is required for the library to work.
The project is licensed under MPL-2.0. That is a file-level copyleft licence: modifications to RAGLite's own source files must be made available under the same terms, while larger works that combine it with other code are not automatically covered. The README also emphasises permissive dependencies, which reduces licence review surface, but MPL-2.0 itself is not the same as MIT or Apache-2.0, and organisations with strict licence policies should check it before adoption. This is a description of the licence, not legal advice.
On maintenance, the release history shows v0.7.0 in March 2025, v1.0.0 in June 2025 and v1.1.1 in May 2026, with repository activity recorded as recently as August 2026. The gap between v1.0.0 and v1.1.1 is roughly eleven months, which suggests a stabilised API rather than rapid churn. The upgrade cost that actually matters is not the library version but the pieces around it: the LiteLLM provider list, the rerankers library, llama-cpp-python wheels and the pgvector extension all move on their own schedules, and the README's environment-variable matrix for llama-cpp-python is the kind of configuration that needs rechecking whenever those versions change.
Editorial conclusion
Adopt RAGLite if you already run PostgreSQL or want a single-file DuckDB store and you are willing to accept LiteLLM as your model abstraction layer. Do not adopt it if you need a managed vector service with independent scaling, or if you cannot install an accelerated llama-cpp-python wheel for local models. Before committing, verify two things against your own corpus: that the wtpsplit-lite and semantic chunking steps terminate in acceptable time on your longest documents, and that your PostgreSQL instance has the tsvector and pgvector extensions available, because the hybrid search path depends on both.
Community notes