Graph-R1 trains a language model to loop on a knowledge hypergraph

The method is a loop, and the loop is the contribution

The paper behind this repository argues that GraphRAG solves the data silo problem and improves knowledge retrieval efficiency, but that a disconnect between graph structured knowledge and language modalities still limits performance. Graph-R1 is the proposed answer: an end to end reinforcement learning framework aimed at improving reasoning on graph capabilities of large language models. The mechanism is described in three steps. The system constructs a knowledge hypergraph using n-ary relation extraction, which is the point where hypergraph comes from, a hyperedge can hold more than two entities and so a relation with several arguments is a first class object rather than a set of pairs. Within reinforcement learning an explicit reward mechanism lets the model iteratively execute a think, generate query, retrieve subgraph and rethink cycle. So the model is trained to write its own retrieval queries, act on the graph, and revise, with the reward computed over the whole trajectory rather than only the final text. The stated application areas are knowledge intensive fields such as healthcare, finance and law.

The pipeline is four long running stages

Everything the repository does runs as background processes with logs written beside them, and the sequence matters. First the dataset is converted to parquet format by a script_process.py invocation per data source. Second, and optionally, contexts are extracted and the knowledge hypergraph is built by script_build.py, which the documentation says uses GPT-4o-mini as the extractor, which is why an OpenAI key has to be set in openai_api_key_txt. Third, a retrieval server is started on port 8001 with script_api.py, and that server is what the trained agent queries during training. Fourth, training is launched with one of three shell scripts. The whole thing is designed to run unattended, since each stage is wrapped in nohup and redirected to a log file, and the sixth step of the quick start is to kill the retrieval server on port 8001 when you are done. The hypergraph step is optional because pre-built hypergraphs can be downloaded and pointed at from expr/, which saves both the API cost and the wall clock time of the extraction pass.

The environment asks for Python 3.11, torch 2.4 and flash-attn

The install sequence is fixed and specific, and a few of the pins matter more than others.

conda create -n graphr1 python==3.11.11
conda activate graphr1
pip3 install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu124
pip3 install flash-attn --no-build-isolation
pip3 install -e .
pip3 install -r requirements.txt

The Python version is pinned exactly rather than as a minimum, and the torch build is pinned to a CUDA 12.4 wheel index, so the environment is a specific combination rather than a range. flash-attn is installed with no build isolation, which is how its extension build normally has to be done. The editable install of the project itself is followed by requirements.txt, and the setup.py in the repository shows why the second command is not redundant: the install_requires list there pulls in accelerate, datasets, hydra-core, peft, ray, transformers, vllm pinned at 0.6.3 or lower, wandb and tensordict below 0.6, while the requirements file adds the development set on top, including FlagEmbedding, faiss-cpu, fastapi, uvicorn, spacy and a set of vector and graph database clients such as neo4j, pymilvus and graspologic. That is a heavy dependency surface, and it is the surface of a training stack rather than of a library.

Three RL algorithms, one launcher per algorithm

Training is launched through three separate scripts, one for each algorithm the paper compares: run_grpo.sh, run_rpp.sh for REINFORCE++ and run_ppo.sh. The quick start runs GRPO with Qwen2.5-3B-Instruct as both the policy and the model argument, against 2WikiMultiHopQA, and the commented variants show the same command for the other five datasets.

nohup bash -u run_grpo.sh -p Qwen/Qwen2.5-3B-Instruct -m Qwen2.5-3B-Instruct -d 2WikiMultiHopQA > result_run_Qwen2.5-3B-Instruct_2WikiMultiHopQA_grpo.log 2>&1 &

The hardware requirement is stated in the heading of that step: 4 x 48GB GPUs. That is the single most important practical fact about this repository, and it is the reason the project reads as a reproduction package rather than a usable component. The three algorithms are run so their results can be compared, and the experimental section of the README presents results across different RL algorithms, across different GraphRAG datasets and across different parameter scales of the model, each as a set of figures rather than as numbers in text. The six datasets are 2WikiMultiHopQA, HotpotQA, Musique, NQ, PopQA and TriviaQA, downloadable from a linked TeraBox share with the data path set inside datasets/.

The repository bundles a fork of verl

The top level directory listing is worth reading carefully, because it explains the shape of the code. There is a verl/ directory, and the setup.py in the repository carries the Apache 2.0 header of Bytedance and points its url at github.com/volcengine/verl, which is the upstream project the fork is based on. The version number is read at install time from a file inside the verl package rather than being hard coded. Alongside that fork sit the Graph-R1 specific pieces: an agent/ directory, a graphr1/ package, eval.py, and the three scripts that orchestrate the pipeline, script_process.py, script_build.py and script_api.py. Evaluation and inference are not documented in the top level README at all, they are delegated to the evaluation/ and inference/ folders with their own README files, and requirements.txt carries a comment saying it records the full set of dependencies for development. The repository licence is MIT while the bundled verl code carries its own Apache 2.0 header, so a fork of the fork has two licences in play and the boundary between them is worth checking before you redistribute anything.

Where this stops being the right tool

There are clear cases for stopping here. If you want a GraphRAG question answering service in production, this is not it, because the deliverable is a training pipeline and the README does not describe a supported serving configuration, and the retrieval server on port 8001 exists to serve the agent during training rather than to be exposed. If you cannot run four 48GB GPUs, the training stage is out of reach, and the alternative of using a smaller policy model would take you away from the paper's setup. If you have no budget for an OpenAI key, the hypergraph construction stage needs one, unless you take the pre-built graphs. And if what you need is a plain graph retrieval library, the dependency list here is dominated by reinforcement learning tooling, ray and vllm among them, none of which a retrieval use case needs. The realistic use is as a reference implementation to read and adapt, and the honest caveat is that the last push to the main branch was on 2026-04-30 with no tagged releases, so what you get is a snapshot of a research artifact rather than a versioned library.

Editorial conclusion

Graph-R1 suits research groups replicating the paper's setup, or engineers who want a reference implementation of an agentic GraphRAG training loop to adapt. It does not suit teams that want a working retrieval service, since the deliverable here is training scripts and a paper, and it does not suit anyone without multi GPU hardware or an OpenAI key for the extraction step. Before using it, check that you can run four 48GB GPUs, set the openai key in openai_api_key_txt, and read the inference and evaluation READMEs in the repository, since the top level document stops at the training commands.

Frequently asked questions

Official sources

1. Issues
2. LHRLAB/Graph-R1 on GitHub
3. License: MIT
4. Project website
5. README