DR Tulu trains a deep research agent with rubrics that keep changing

Three codebases that answer to one model

DR Tulu is the official repository for DR Tulu, Reinforcement Learning with Evolving Rubrics for Deep Research, and the claim in the README is that DR Tulu-8B is the first open deep research model trained for long-form DR tasks, and that it matches OpenAI DR on long-form DR benchmarks.

The repository holds three main components, and the split mirrors the three stages of building such a model.

The agent/ directory is the agent library, named dr-agent-lib, with an MCP-based tool backend, high-concurrency async request management and a flexible prompting interface for developing and training deep research agents. It also contains the evaluation scripts for benchmarking DR agents.

The rl/ directory holds the RL training code, based on Open-Instruct, for training deep research agents with GRPO and evolving rubrics.

The sft/ directory holds the SFT training code, based on LLaMA-Factory, for supervised fine-tuning of deep research agents.

The top level README deliberately does not duplicate the setup instructions and points to the README files in each subdirectory instead. For a repository this size that is the right call, and it is also a warning: the three directories are separate code bases with separate dependencies, and a user who expects one pip install to cover everything will not get it.

The project is provided by The Allen Institute for Artificial Intelligence, with the code developed in collaboration with student researchers at the University of Washington, Carnegie Mellon University and MIT.

The agent talks to search services through an MCP backend

The mechanism that makes the demo work is a tool backend rather than a direct HTTP call. The agent library is described as having an MCP-based tool backend, and the quick start makes that concrete by requiring API keys for three separate services.

   export SERPER_API_KEY="your_key"  # https://serper.dev/
   export S2_API_KEY="your_key"      # https://api.semanticscholar.org/
   export JINA_API_KEY="your_key"    # https://jina.ai/reader/

Each key maps to a different capability. Serper is the web search service, Semantic Scholar is the academic search API, and Jina is the reader used for fetching and reading page content. The configuration in the evaluation section confirms the split, because the search agent and the browse agent are configured separately, and the browse tool is selected by name with a value of jina.

The tool layer is a separate process. The MCP server is launched on its own port with a module invocation.

   python -m dr_agent.mcp_backend.main --port 8000

That separation has a practical consequence for anyone operating this. The model server and the tool server scale differently, the tool server is the part that consumes the third party API keys and therefore the part that hits rate limits, and an evaluation that runs twenty concurrent examples is putting twenty concurrent requests through whatever Serper and Jina allow.

The flexible prompting interface and the high-concurrency async request management are the two other pieces the README highlights, and they matter for the same reason: the agent makes many tool calls per research question, and the wall clock cost of an evaluation is dominated by how many of those calls can be in flight at once.

The interactive demo needs one or two GPUs and a conda environment

The quick start is the fastest way to see the model work, and it is a four-step sequence.

First, the environment, created in the agent directory and installed in editable mode.

   cd agent/
   conda create -n dr_agent python=3.10 -y && conda activate dr_agent
   uv pip install -e .

The Python version is pinned to 3.10 by the conda command, and the package is installed with uv rather than pip, which suggests a workflow built around uv for the rest of the agent tooling.

Then the API keys, shown above.

Then the demo itself.

   uv run --extra vllm python scripts/launch_chat.py --model rl-research/DR-Tulu-8B

The --extra vllm flag pulls in the vLLM extra when the environment is resolved, and the model identifier is a Hugging Face repository name. The README states that the demo auto-launches the required services, the MCP server and vLLM, and then starts an interactive chat interface, so you ask research questions and watch the model search and synthesise answers in real time.

The hardware requirement is 1 to 2 GPUs, and the setup is described as taking just a few steps. The model identifier in the launch command is the same one used in the evaluation instructions, which is worth noting because it means the demo and the benchmark runs use the same checkpoint rather than a locally rebuilt one.

Advanced options live in the agent README, and the model and data are published as a Hugging Face collection, with an interactive demo also available on the web at dr-tulu.org.

Evaluation puts two vLLM servers and the MCP server on one node

The evaluation instructions are the most demanding part of the repository, and they start with a hardware statement: launch the required servers on the same node, which requires two GPUs.

   # Launch VLLM servers
   CUDA_VISIBLE_DEVICES=0 vllm serve rl-research/DR-Tulu-8B --dtype auto --port 30001 --max-model-len 40960
   CUDA_VISIBLE_DEVICES=1 vllm serve Qwen/Qwen3-8B --dtype auto --port 30002 --max-model-len 40960
   
   # Launch MCP server
   python -m dr_agent.mcp_backend.main --port 8000

Two models are served, not one. The first is the model under test, and the second is Qwen3-8B on port 30002, which the evaluation harness needs alongside it. The context length is set to 40960 for both, which is the long-context setting a deep research agent needs to hold a gathered evidence trail.

On top of that sits the MCP server on port 8000, launched as shown in the previous section. Three processes, two GPUs, one node.

The task list in the run script names the benchmarks covered: healthbench, deep_research_bench, research_qa, genetic_diseases, simpleqa, 2wiki and webwalker. The README also refers to Deep Research Bench, SimpleQA and HealthBench as examples of what can be benchmarked, and notes that SQA-CS-V2 and Deep Research Bench require additional conversion scripts, with the details in the evaluation README.

That is a heavy footprint for what looks like a single evaluation, and it is the main practical obstacle to reproducing the paper's numbers.

The evaluation loop caches, caps tool calls and writes jsonl

The generation step is driven by a workflow script with a set of flags that show how the runs are shaped.

   cd agent/
   
   # Example: Run on all benchmarks
   for task in healthbench deep_research_bench research_qa genetic_diseases simpleqa 2wiki webwalker; do 
       python workflows/auto_search_sft.py \
           generate-dataset $task \
           --num-examples final_run \
           --max-concurrent 20 \
           --use-cache \
           --config workflows/auto_search_sft.yaml \
           --config-overrides "use_browse_agent=true,search_agent_max_tool_calls=10,browse_tool_name=jina" \
           --output eval_output/auto_search_sft/$task.jsonl
       
       python scripts/evaluate.py $task eval_output/auto_search_sft/$task.jsonl
   done

The --config-overrides argument is the interesting part. It is a comma separated list of key value pairs that is applied on top of the YAML configuration, and the three shown turn on the browse agent, cap the search agent at 10 tool calls, and select the jina browse tool. A YAML file for a research run is long, and being able to change three values from the command line is what makes iterating on an evaluation practical.

--max-concurrent 20 is the concurrency knob, and it maps directly onto the API keys you configured earlier, since each concurrent example is making its own tool calls. --use-cache reuses previous results, which matters for a run that is otherwise re-requesting the same pages and the same search results on every iteration.

The scoring step is a separate script that reads the generated file.

       python scripts/evaluate.py $task eval_output/auto_search_sft/$task.jsonl

Generation and evaluation are decoupled, and the intermediate artefact is a jsonl file per task under eval_output. That separation is what lets you rescore a generation without re-running the agent, and it is the right design for anyone who wants to try a different judge on the same outputs.

Evolving rubrics replace a fixed scalar reward

The title of the work is the mechanism: Reinforcement Learning with Evolving Rubrics for Deep Research, and the RL code is described as training deep research agents with GRPO and evolving rubrics.

A rubric is a written specification of what a good answer should contain, and the interesting claim is that the rubric is allowed to change during training rather than being frozen at the start. For a task with no single right answer, a fixed scalar reward model is the weak link: it compresses a long, structured research report into one number and then optimises against that number, which is how a model learns to produce answers that look good to the reward model rather than answers that are good.

The repository does not restate the algorithm in detail, and the README points to the RL README for the full setup including reward model training and policy optimization. So the honest summary is that the reward specification is the contribution and the code that implements it is in rl/open-instruct, on top of the Open-Instruct codebase.

That dependency is worth naming, because it shapes what you inherit. The RL directory is not a standalone trainer, it is Open-Instruct with the DR agent and rubric machinery added, and the SFT directory is LLaMA-Factory with the same. Both are mature upstream projects, which means the setup instructions, the hardware expectations and the failure modes are largely theirs.

cd rl/open-instruct/
# See rl/open-instruct/README.md for detailed instructions

The SFT side follows the same pattern.

cd sft/llama-factory/

A team reproducing the training pipeline is therefore running three known code bases and gluing them together, not debugging one system.

Apache 2.0, no releases, and a published citation

The licence is Apache-2.0, with the LICENSE file at the root of the repository, which is the permissive choice with an explicit patent grant and no copyleft obligation. The model weights and data are published separately, through the Hugging Face collection linked from the README, and the licence on the code does not by itself tell you the terms of the weights.

Maintenance is active but the release history is thin. The last push was on 2026-10-09 and the repository is not archived. The repository has no GitHub releases, so there is no tag to pin and no release note feed.

What there is, instead, is a dated release notes list in the README, and the order in it is worth noticing. November 19, 2025 is the initial code release, November 25, 2025 brought the interactive CLI demo code along with additional documentation for evaluation, training and new RL checkpoints, and February 9, 2026 added a free interactive demo at dr-tulu.org.

The citation is a BibTeX entry for an arXiv preprint, arXiv:2511.19399, with the author list included in the README, and named contacts for questions are given for Rulin Shao, Akari Asai, Shannon Shen and Hamish Ivison, alongside opening a GitHub issue.

The top level tree has an app/ directory alongside agent/, rl/ and sft/, plus script/ and assets/, which is consistent with the web demo being part of the same codebase rather than a separate deployment.

Editorial conclusion

Use this repository if you want to run or evaluate an open deep research agent with tool use, or if you are studying reinforcement learning with evolving rubrics, since the RL code sits on top of Open-Instruct and the agent code is a separate installable library. Do not expect a one-command reproduction, because the evaluations assume two GPUs on one node, an MCP backend and a set of third party API keys. Before starting, read the README in each of the agent, rl and sft directories, since the top level file deliberately only points at them.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. rlresearch/dr-tulu on GitHub