neuml/rag: A Streamlit RAG Front End With Two Retrieval Modes

What neuml/rag Actually Ships

This is a Streamlit application, not a library. The README describes it as a Retrieval Augmented Generation (RAG) Streamlit application backed by txtai. That framing matters more than it first appears. You do not import neuml/rag into your own code and call functions on it. You run it, point it at an embeddings index, and interact with it through a browser. The two entry points given are a Docker image on Docker Hub and a Python virtual environment started with streamlit run rag.py.

The problem it addresses is narrow and concrete: RAG pipelines are easy to describe and fiddly to inspect. When an answer comes back wrong, the usual question is whether the retrieval step failed or the generation step failed. This application puts both in front of you. A vector query shows which documents were selected. A graph query draws the graph so you can see the traversal that produced the context. For an engineer evaluating whether retrieval quality or model quality is the bottleneck, that visibility is the point.

The audience is correspondingly narrow. If you are building a production retrieval service, this is not the component you embed. If you are building an internal tool, running a comparison, or trying to explain RAG behaviour to people who will not read a notebook, the Streamlit surface is the useful part.

Vector RAG and Graph RAG Are Two Different Query Paths

The README splits RAG into two categories. Vector RAG runs a vector search to find the top N matches for the user input, places those matches into an LLM prompt, and returns the answer. The worked example is Who created Linux?, which runs a vector search against the Embeddings index and hydrates a prompt with the matches. The number of matches is controlled by the CONTEXT variable, default 10.

Graph RAG replaces the vector search with graph path queries. Three forms are documented. The first is a graph query expansion, triggered by the gq: prefix, as in gq: Tell me about Linux. The documentation states it starts with a vector search to find the top n results, then expands those results using a graph network stored alongside the vector database. The second form is a graph path query, written as a chain of concepts such as linux -> macos -> microsoft windows. This finds the nodes closest to those concepts and runs a traversal to build a context of related nodes. The third form combines them: linux -> macos -> microsoft windows gq: Tell me about Linux runs a path query first, then restricts the graph query to that traversal.

The unit of the graph is worth noting. Each node is a section, meaning a paragraph, and node labels are generated with an LLM prompt that applies a topic label at upload time. So graph mode is not free at ingest: adding data costs an extra generation pass per section. The README does not state how long that takes or how it scales, and the TOPICSBATCH variable exists precisely because batching those LLM topic queries is something you may need to tune.

Getting It Running: Docker First, pip Second

The README recommends Docker, at least to get an idea of the application's capabilities. The command is a single line:

docker run -d --gpus=all -it -p 8501:8501 neuml/rag

Note the --gpus=all flag in the documented command. The README does not describe a CPU-only path for the container, so if you have no NVIDIA GPU the documented invocation is not the one to copy.

The manual path is two commands. Install dependencies with pip install -r requirements.txt, then start the app with streamlit run rag.py. The README recommends doing this inside a Python virtual environment. It does not pin a Python version in the material available here.

Configuration is entirely by environment variable. TITLE changes the application title. EXAMPLES is a semicolon-separated list of sample queries, defaulting to the four shown in the demo. LLM is a path to the model, defaulting to Qwen3-4B-Instruct-2507. EMBEDDINGS is the embeddings database path, defaulting to neuml/txtai-wikipedia-slim. MAXLENGTH caps generation length at 2048 for topics and 4096 for RAG. STRIPTHINK controls whether thinking text is stripped from responses and defaults to False. TEXTBACKEND selects the text extraction backend. DATA is an optional directory to index from. PERSIST is an optional directory to save index updates to. SAFEOPEN enables Textractor safeopen mode and defaults to True.

The # Prefix Is the Whole Ingestion Interface

There is no separate upload page described. Data enters the index through the query box. A query beginning with # is read by the application and loaded. The README gives two forms: # file path or URL, and # custom notes and text as a string here! So the string # txtai is an all-in-one AI framework would create a new entry in the Embeddings database.

This is elegant for a demo and awkward for anything else. It means the same input field is both a retrieval control and a write operation, distinguished by one character. There is no documented confirmation step, no batch import command, and no dry run. The DATA variable offers a directory to index from at startup, which is the more conventional route, but it is described only as an optional directory with no detail on file types or recursion in the material available.

The PERSIST variable is the one to watch. It is an optional directory to save index updates to. The word optional is doing real work here. If PERSIST is unset, the README does not state that uploads survive a restart, and the natural reading of optional save directory is that they do not. Anyone using the # syntax for real data should set PERSIST before adding anything they care about.

Where This Application Is the Wrong Tool

The graph mode has a cost that the README states plainly and that is easy to miss: node labels are generated with an LLM prompt at upload time. That means ingestion is not a pure indexing operation. It needs a working model, it needs generation capacity, and it produces labels that depend on that model's output. Change the LLM and the topic labels in an existing index may no longer match what a fresh ingest would produce. The documentation does not describe a re-labelling path.

Graph mode also assumes your corpus has a graph worth traversing. The path query syntax, linux -> macos -> microsoft windows, requires you to name concepts that exist as nodes. On a corpus of flat, unrelated documents, the traversal has little to walk. Vector RAG with a CONTEXT of 10 is the more sensible default there.

The larger limitation is architectural. Because this is a Streamlit app, there is no documented HTTP API, no Python entry point for embedding the retrieval logic in your own service, and no described authentication or multi-user handling. If your requirement is a RAG component inside a backend, you are looking at the wrong repository. The underlying txtai project is where that logic lives; this project is the interface on top of it.

How It Differs From a Framework Like LangChain

The obvious comparison is LangChain, and the difference is not feature count. LangChain is a library: you write Python, compose retrievers and chains, and the framework is invisible at runtime because your code is the application. neuml/rag inverts that. The application is given, and your configuration is the variable set.

That inversion has consequences. You cannot unit test a retrieval chain here, because there is no chain object to test. You cannot swap the retriever for a custom implementation, because retrieval is txtai's Embeddings and graph network. What you get instead is a running system in one Docker command, with a UI that renders the graph behind a graph query. The README notes that every Graph RAG query response shows a corresponding graph to help understand how the query works. For debugging retrieval behaviour, that is faster than instrumenting a LangChain pipeline.

The choice is therefore about what you are doing this week. Comparing retrieval strategies on a corpus, or demonstrating RAG to a non-engineering audience: neuml/rag. Building a service that other systems call: a library, with txtai underneath if you want the same graph expansion primitives.

Licence, Maintenance and Upgrade Surface

The project is Apache-2.0. That is a permissive licence and, unlike a copyleft licence, it does not require you to publish modifications you make to the application. It also includes a patent grant. This is a description of the licence text, not legal advice; if you are redistributing the application or the Docker image inside a commercial product, have counsel read the NOTICE and attribution requirements rather than relying on a summary.

The release cadence visible in the material is uneven. v0.12.0 and v0.13.0 landed about three weeks apart in November and December 2025, then v0.14.0 arrived roughly six months later in June 2026. That pattern suggests bursts of work rather than a fixed schedule, which is normal for a single-maintainer project but relevant if you need predictable upgrade windows.

The upgrade cost is concentrated in two places. First, the environment variables are the public interface, so a renamed or removed variable is a breaking change for anyone with a deployment script. Second, the default EMBEDDINGS and LLM values are Hugging Face references, and changing either changes retrieval and generation behaviour together. Pin both explicitly rather than relying on defaults if you want reproducible answers across upgrades. The DEFAULT branch is master, which is worth knowing when writing CI that clones the repository.

Who Should Run This

Run it if you have a document set, a GPU, and a question about whether graph traversal retrieves better context than a top-10 vector search on that specific set. The four default EXAMPLES queries give you a side-by-side starting point without writing code, and the rendered graph tells you why a graph query returned what it returned. That is a genuinely useful afternoon.

Do not run it if you need an API, if your corpus has no concept structure to traverse, or if you cannot afford an LLM pass over every section at ingest. The upload-time labelling requirement is the constraint most likely to disqualify a real corpus, and the README does not offer a way to skip it.

Before you commit, check three things. Whether your hardware matches the --gpus=all invocation in the documented Docker command. Whether PERSIST is set, since index updates go to an optional save directory and the README does not promise they survive without one. And whether the default Qwen3-4B-Instruct-2507 model is one you are willing to depend on for both answers and topic labels, because the same LLM setting drives both.

Editorial conclusion

Adopt neuml/rag if you want a working Streamlit interface for comparing vector RAG against graph path queries on your own data, and you are comfortable with txtai underneath. Do not adopt it if you need a Python library to embed in a larger service, or if you cannot accept an LLM call at data upload time to label topic nodes. Before committing, verify that the default EMBEDDINGS index (neuml/txtai-wikipedia-slim) and the default LLM (Qwen3-4B-Instruct-2507) fit your hardware, and check whether PERSIST is set, because without it index updates made through the # upload syntax are not written back to disk.

Official sources

1. Issues
2. License: Apache-2.0
3. neuml/rag on GitHub
4. README
5. Releases