graphrag-local-ollama: Microsoft GraphRAG with Ollama as the LLM and embedding backend

What problem the fork actually removes

Microsoft's GraphRAG builds a graph-based text index in two stages. It derives an entity knowledge graph from source documents, then pre-generates community summaries for groups of closely related entities. The published paper frames the motivation as global sensemaking: questions like "What are the main themes in the dataset?" are query-focused summarization, not retrieval, and a naive RAG baseline answers them poorly because no single chunk contains the answer. GraphRAG's answer is to summarize communities of entities ahead of time and then summarize those partial responses again at query time.

The cost of that design is model calls. Indexing a corpus means an extraction pass over every chunk plus a summarization pass over every community, and the reference implementation expects a hosted API. TheAiSingularity/graphrag-local-ollama targets that specific bill. Its README describes itself as "an adaptation of Microsoft's GraphRAG, tailored to support local models downloaded using Ollama," and the feature list names LLM and embedding extraction as the two things being localized. So the person this is for is someone with a private or previously unseen document collection who wants the graph-index approach without per-token spend, and who is willing to run the models on their own machine.

It is not a reimplementation. The repository is Python and its query entry points are still invoked as python -m graphrag.query, which tells you the package structure and CLI surface are inherited rather than replaced.

Where Ollama sits in the GraphRAG pipeline

GraphRAG has two distinct model roles, and the fork has to satisfy both locally. The first is the LLM that does entity and relationship extraction during indexing and then answers queries. The second is the embedding model that turns text units and entities into vectors for the vector store. The README lists llama3, mistral, gemma2 and phi3 as the supported LLMs and states that embeddings are also extracted through Ollama, which means both roles resolve to a locally running Ollama server rather than to a remote endpoint.

The query side exposes five methods, and they differ in how much of the index they touch. Global search uses the pre-generated community summaries. Local search works from entities. Basic search is the lightweight path: it embeds the query with the Ollama embedding model, retrieves the top-10 most similar entities from the vector store, builds context from those entities plus their relationships and source text units, and answers. Basic search explicitly needs no community reports, so it works after a minimal index run. DRIFT is the iterative one: a primer phase decomposes the question into scored sub-questions using community reports as context, a search loop answers each sub-question against the entity graph and generates follow-up questions from those answers under a priority queue with a depth limit, and a reduce phase synthesizes the intermediate answers. Lazy is the fifth.

That is a real architectural distinction worth noting. Basic search is vector similarity over entities, which is closer to conventional RAG than to graph traversal. DRIFT is the opposite end, spending multiple model calls per query to explore. Global sits in between, reading summaries that were paid for at index time. Choosing a method is choosing when you pay.

LazyGraphRAG moves the cost from indexing to the first query

The most consequential setting in the repository is lazy_graph_rag, a boolean in settings.yaml. Set it to true and community summarization is skipped at index time; summaries are generated on demand when a query arrives. The README claims indexing becomes roughly 99 percent faster under this mode. That figure comes from the project's own documentation and I have not reproduced it; treat it as a claim about the indexing phase specifically, not about end-to-end work, because the summarization still happens, just later.

The README is candid about the trade-off in its own note: first-query responses are slower than standard global search because summaries are computed at query time, and for large datasets the overall cost is still much lower. That is the correct framing. LazyGraphRAG does not delete work, it defers it and amortizes it across queries. If you index once and query once, you have gained nothing and lost latency. If you index once and query many times, the deferred summaries get cached in effect by repeated use, which is where the saving lives.

There is a second-order effect the README also flags. DRIFT's primer phase uses community reports as context. If you indexed with lazy_graph_rag: true, the primer step has limited context, though the README says the search loop still operates normally. So the lazy flag is not orthogonal to query quality. It degrades the mode that depends most on pre-computed summaries. If DRIFT is your intended query method, lazy indexing works against you.

Two ways to run it, and what each one assumes

The Docker path is the short one. Copy .env.example to .env, run docker-compose up --build, drop .txt documents into ./input/, and indexing runs automatically on container start. Queries go through docker-compose run graphrag python -m graphrag.query --root /app --method global "What are the main themes?". The README notes that GPU acceleration requires adding deploy.resources.reservations.devices to the ollama service in docker-compose.yml. Without that edit you are on CPU, and the extraction pass over a large corpus is where you will feel it.

The Web UI path assumes a Python environment. Install requirements-ui.txt, run python app.py, and open http://localhost:7860. The UI has four tabs: Index for uploading .txt files and watching live log output, Query for the five search methods, Graph for an interactive knowledge-graph visualizer that requires pyvis, and Settings for editing model names, chunk size and the LazyGraphRAG toggle. The visualizer dependency is worth catching early, since the Graph tab is inert without pyvis installed.

Input formats beyond plain text are handled: .txt and .csv as before, plus JSON arrays of objects with id, title and text fields, and JSONL. The README also claims full UTF-8 support with CJK-aware chunking covering Chinese, Japanese, Korean, Arabic and Cyrillic. Chunking that respects CJK boundaries matters more than it sounds, because naive character-count splits cut mid-token and degrade the extraction step's output.

The local model is the weak link, not the fork

GraphRAG's index quality depends on the extraction LLM reliably emitting structured entity and relationship output over long chunks. That is a demanding task. A hosted frontier model does it consistently; a quantized 7B or 8B model running through Ollama may not, and the failure is quiet. You get a thinner graph, fewer entities per community, and community summaries built on less material. Nothing in the README describes validation or a retry path for malformed extraction output, so the practical failure mode is an index that looks complete and answers global questions vaguely.

The README itself carries a line under the contributing section: "Need support for llama integration." Read literally that suggests the llama integration is incomplete or in progress, which is a strange thing to say when llama3 is listed among the supported models. I cannot resolve that from the supplied material, and it is exactly the kind of ambiguity worth checking against the repository before you plan around it.

There is also no tagged release. The repository has no releases retrieved, so installation means tracking main, and the last push date is the only signal about how current the code is. For a tool that sits underneath an index you may want to keep for months, that is a real operational consideration. Pin a commit hash yourself if you care about reproducibility, because the project does not do it for you.

How this differs from running GraphRAG against a hosted API

The obvious alternative is Microsoft's upstream GraphRAG with a hosted model provider. The difference is not cosmetic. Upstream assumes network calls to a paid endpoint for both the LLM and the embeddings, which gives you consistent extraction quality and no local hardware requirement, at the cost of sending your documents to a third party and paying per token for every indexing run and every query. This fork inverts all three: documents stay on the machine, marginal cost per token approaches zero, and the ceiling on quality becomes whatever your GPU and your chosen Ollama model can sustain.

A second alternative, and the one worth weighing if your questions are mostly factual lookups rather than corpus-wide themes, is plain vector RAG. The fork's own Basic search is effectively that mode: embed the query, pull the top-10 similar entities, answer from their relationships and source text units, with no community reports involved. If Basic search is all you need, the graph construction and community summarization stages are overhead you are paying for and not using. The paper's argument for GraphRAG applies to global sensemaking questions; it does not apply to "Who is Alan Turing?"

So the honest comparison is three-way. Hosted GraphRAG buys extraction reliability with money and data exposure. Plain vector RAG buys speed and simplicity by giving up corpus-level summarization. This fork buys privacy and near-zero marginal cost by accepting that your local model's extraction quality is the binding constraint.

Licence, maintenance, and what to check before you commit

The repository is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is the permissive end of the spectrum and it is compatible with the upstream project it adapts, but I am not giving legal advice: if you are redistributing a modified version or bundling it into a product, read the licence text and the upstream GraphRAG licence yourself rather than relying on a summary.

Maintenance cost here is mostly model and hardware cost, not code churn. Ollama models are pulled and updated independently of this repository, so a model tag that behaves one way today can change under you. The repository has no releases to pin against, which pushes the burden of version control onto you: record the commit you installed, record the model tags and quantization you pulled, and record the settings.yaml values, because chunk size and the lazy_graph_rag flag both change what the index contains. Rebuilding an index after changing any of those three is not optional.

The thing to verify first is extraction quality on your own documents, not the setup. Run the Docker path, index a small representative sample, and inspect the resulting entities and relationships before you spend hours on the full corpus. If the graph is thin, the problem is the model, and no amount of query-method tuning will fix it. If the graph is dense and the community reports read coherently, then the fork is doing its job and the remaining question is whether your hardware can carry the full collection.

Editorial conclusion

Adopt it if you have documents you cannot send to a hosted API and a machine with enough VRAM or unified memory to hold both an instruct model and an embedding model at once. Do not adopt it if you need a pinned, reproducible index over a corpus that changes weekly: the README documents no incremental re-index path, and the LazyGraphRAG flag changes what the index contains. Before committing, verify two things on your own hardware: that your chosen Ollama model can actually emit the entity and relationship JSON the graph extraction step expects, and that the community report step completes without truncating on your chunk size. The repository carries the MIT licence and has no tagged releases, so you are tracking the main branch.

Official sources

1. Issues
2. License: MIT
3. README
4. TheAiSingularity/graphrag-local-ollama on GitHub