NVIDIA RAG Blueprint: A NIM-Based Reference Pipeline You Decompose Yourself
This NVIDIA RAG blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline.
At a glance
- What is it?
- The blueprint is a reference RAG stack built from NVIDIA NIM microservices, a LangChain-based orchestrator and a cuVS-accelerated vector database. It is strongest when you already run NVIDIA inference infrastructure and want a starting point to fork, not a product to install.
- Who is it for?
- Adopt the blueprint if your inference already runs on NVIDIA NIM endpoints and you want a forkable reference for multimodal ingestion, hybrid retrieval and reranking rather than a framework to learn from scratch. Skip it if you need a vendor-neutral retrieval layer or cannot run the NIM microservices it depends on.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 13 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap the blueprint fills: an assembled RAG stack, not a library
Most RAG projects start as glue code. You pick an embedding model, a vector store, a reranker, an extractor for PDFs with tables, and a generation model, then spend weeks making them agree on formats. The NVIDIA RAG Blueprint is that glue, already written. The README describes it as a reference solution and foundational starting point for RAG pipelines built on NVIDIA NIM microservices, aimed at enterprises that need question answering grounded in their own data while meeting governance, latency and scalability requirements.
The intended reader is an engineer or platform team that has already decided to run NVIDIA inference endpoints and now wants a working pipeline to modify. The repository is explicit about this posture. It ships a pre-built reference UI and open-source code, and the README calls it a flexible starting point that developers can adapt and extend. That framing matters: this is a codebase to fork, not a package to depend on. The topics list (blueprint, nim, rag) and the build.nvidia.com homepage reinforce that the artefact is a demonstration of NIM composition as much as a standalone tool.
What the pipeline actually does between a question and an answer
The architecture splits into two layers the README names directly. The first is NVIDIA NIM microservices, which deliver the core AI functionality: large-scale inference with Nemotron LLM models for response generation, retrieval and reranking models, and specialised extractors for text, tables, charts and graphics. Optional NIMs extend this with OCR, content safety, topic control and multimodal embeddings. The second is the integration and orchestration layer, described as the glue that binds the system together.
Inside that layer sits the RAG Orchestrator Server, which the README says coordinates interactions between the user, retrievers, vector database and inference models, handling multi-turn and context-aware queries. It is LangChain-based. Alongside it is a vector database accelerated with NVIDIA cuVS, which stores and searches embeddings at scale.
The retrieval path is more than a single embedding lookup. The feature list includes hybrid search with dense and sparse retrieval, reranking to improve accuracy, multi-collection searchability, and query processing steps for decomposition and dynamic filter expression creation. Ingestion handles multimodal content (text, tables, charts, infographics and audio) with custom metadata support, and points to the NeMo Retriever extraction documentation for the supported file type list. Generation can optionally use multimodal and vision language models, document summarisation with multiple strategies, optional reflection, and programmable guardrails.
A second pipeline exists for harder questions. Agentic RAG adds a LangGraph plan-and-execute flow alongside the standard retrieve-then-generate chain, with scope discovery, parallel sub-tasks, synthesis, optional verification, and streaming stage events surfaced in both the UI and the API. It can be turned on per request with agentic: true on /v1/generate, or deployment-wide with ENABLE_AGENTIC_RAG, or selected in the reference UI under Pipeline, Agentic. That per-request switch is the most interesting design detail in the README: it means the expensive path is opt-in rather than the default, which keeps latency predictable for simple queries.
Running it: Docker, Kubernetes and the configuration surface
The README gives three deployment shapes: local Docker with NVIDIA hosted endpoints, local Docker without them, and Kubernetes. It also mentions NIM Operator support and a Python library mode, plus OpenAI-compatible APIs. The repository does not reproduce the full command sequence in the material available here, so the exact docker compose invocation and environment file names should be read from the repository itself rather than guessed at.
What is documented as configuration is the toggle set. ENABLE_AGENTIC_RAG turns the plan-and-execute pipeline on for the whole deployment. The agentic flag on /v1/generate turns it on for one call. The reference UI exposes the same choice as a Pipeline selector. Beyond that, the README describes the system as decomposable and configurable, with a pluggable vector database, which implies the storage backend is a configuration decision rather than a hard-coded dependency.
Two operational notes are worth flagging before you start. First, the blueprint is built around NIM microservices, so a local Docker run without NVIDIA hosted endpoints still needs those services reachable somewhere. Second, telemetry and observability are listed as features, which means there is something to configure or disable depending on your environment. Neither detail is spelled out in the material provided, so treat the repository's deployment documentation as the source of truth.
Where the blueprint constrains you
The dependency on NIM microservices is the central constraint. Response generation is documented against a specific Nemotron model, retrieval and reranking against specific llama-nemotron embed and rerank models, and extraction against a set of Nemotron page, table, graphic and OCR NIMs. Swapping in a different generation model is not described as a first-class path in the README. If your organisation standardises on a different inference provider, the orchestration code may still be useful, but you are rewriting the parts that made the blueprint worth adopting.
The second limitation is documentation depth in this material. The README is a feature inventory. It lists hybrid search, reranking and query decomposition without stating how they are tuned, what the defaults are, or how the components fail. There is no discussion of what happens when extraction misreads a table, or how the agentic planner behaves when scope discovery returns nothing. Those are the questions that decide whether a RAG system is usable in production, and they are not answered here.
Third, the blueprint carries a substantial service footprint. A minimal configuration already involves an orchestrator, a vector database, an embedding model, a reranker and a generation model, with OCR, content safety, topic control and multimodal embeddings as optional additions. Each is a separate microservice with its own resource profile. Teams without GPU capacity or an existing NIM deployment should treat that as a hard boundary, not a tuning problem.
How it compares with building on LangChain or LlamaIndex directly
The obvious alternative is a hand-rolled stack on LangChain or LlamaIndex. The orchestrator here is LangChain-based, so the difference is not the framework. It is what comes pre-wired. A direct LangChain build gives you complete freedom over embedding and generation providers, but you write the extraction pipeline, the hybrid retrieval and reranking logic, the multi-turn session handling, the evaluation harness and the reference UI yourself. The blueprint hands you those as starting code, at the cost of a NIM-shaped dependency graph.
A second comparison is with managed RAG services, where ingestion, retrieval and generation are one hosted product. Those remove operational work but also remove the ability to inspect or modify the retrieval logic. The blueprint sits between the two: more assembled than a framework, more transparent than a managed service, and deliberately incomplete. The README's own phrasing, a foundational starting point, is accurate in both directions.
Evaluation, release cadence and the cost of staying current
The repository includes evaluation scripts built on the RAGAS framework. That matters because a RAG pipeline without a measurement loop is guesswork, and having a starting harness in the repository lowers the cost of checking whether a change to chunking, reranking or the agentic planner helped. What the scripts measure, and against what dataset, is not described in the material here.
The release history shows v2.6.0 in June 2026, v2.6.1 in early August, v2.6.2 in late August, and a push in early September. That is a fast minor-release cadence. For a fork-based project, this is a real maintenance consideration: every release is a rebase against your modifications, and the more of the orchestration layer you have changed, the more expensive that becomes. The blueprint's decomposable design helps here, since components can be updated independently, but the LangChain-based orchestrator is likely to be the file you touch most and the file that changes most.
On licensing, the repository is Apache-2.0, which permits commercial use and modification with the usual notice and attribution conditions. That licence covers the code in this repository. It does not automatically cover the NIM microservices or the models behind them, which are distributed under their own terms on build.nvidia.com. Verify those separately before shipping anything; the Apache-2.0 header on the blueprint is not a blanket grant for the runtime it orchestrates.
Who this is for, and what to check before you fork it
The blueprint fits teams already committed to NVIDIA inference infrastructure who want a working multimodal RAG pipeline as a starting point rather than a blank repository. The per-request agentic toggle and the pluggable vector database are the two design choices that make it adaptable rather than rigid, and the RAGAS evaluation scripts give you something to measure against from day one.
It does not fit teams that need provider neutrality, that cannot run the required NIM microservices, or that want a supported product with a stable API surface. The fast release cadence and the breadth of the service footprint both work against the second group.
Three things are worth verifying before you commit engineering time. First, read the deployment documentation in the repository to confirm the exact Docker and Kubernetes commands, since the README does not reproduce them here. Second, check the licence terms for each NIM you intend to use, separately from the Apache-2.0 code licence. Third, decide early whether you are forking or tracking upstream, because with releases landing roughly every two to six weeks, that decision determines how much of your adaptation work survives the next version.
Editorial conclusion
Adopt the blueprint if your inference already runs on NVIDIA NIM endpoints and you want a forkable reference for multimodal ingestion, hybrid retrieval and reranking rather than a framework to learn from scratch. Skip it if you need a vendor-neutral retrieval layer or cannot run the NIM microservices it depends on. Before committing, verify which NIMs your deployment actually requires, whether your vector database is on the pluggable list, and what the Apache-2.0 licence covers versus the model licences attached to each NIM.
Community notes