# ARPO and AEPO: agentic reinforcement learning code from a paper team

> The RUC-NLPIR/ARPO repository carries the code, datasets and checkpoints for two agentic reinforcement learning algorithms, Agentic Reinforced Policy Optimization and Agentic Entropy-Balanced Policy Optimization, the first accepted at ICLR 2026 and the second at WWW 2026 as an oral. It is a research release with no stated licence, and the last push was on 2026-10-11.

**RUC-NLPIR/ARPO** — [ICLR 2026] Agentic Reinforced Policy Optimization (ARPO)

- Repository: https://github.com/RUC-NLPIR/ARPO
- Stars: 1,143 · Forks: 64
- Language: Python
- License: not declared
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/ruc-nlpir-arpo

## One repository, two algorithms, and a series that keeps growing

The ARPO repository is a research release rather than a library, and the note at the top of the readme says what is inside it: the codebase, datasets and checkpoints for two reinforcement learning algorithms, Agentic Reinforced Policy Optimization and Agentic Entropy-Balanced Policy Optimization, with the promise that the agentic RL series will continue to iterate and expand.

The top-level listing matches that description. There is an `ARPO/` directory and an `AEPO/` directory, an `evaluation/` directory, an `assets/` directory, and a `LLaMA-Factory/` directory, which is the training framework the project builds on. The primary language is Python.

The timeline in the news section explains the shape of the work.

On July 25, 2025 the full ARPO codebase was released, supporting multi-tool agentic reinforcement learning training for the Qwen2.5, Qwen3 and Llama3 models, along with all the model checkpoints from 3B to 14B and the SFT, RL and evaluation datasets. The same entry records the engineering work behind it, extensive tool-call acceleration and memory optimisation during RL training, with the claim that training Qwen3-14B on a single node with a batch size of 128 takes only ten minutes per step, and that a dynamic cache mechanism saves tool call results in real time.

On October 16, 2025 the second algorithm was proposed, AEPO, focused on entropy-balanced agentic RL, with the claim that it consistently outperforms ARPO on datasets like GAIA, HLE and AIME, and with the full codebase and models released alongside.

The papers were accepted at ICLR 2026 for ARPO and at WWW 2026 as an Oral for AEPO, and both are on arXiv, 2507.19849 for ARPO and 2510.14545 for AEPO.

The most recent entry is from October 12, 2026 and points elsewhere, to a separate Agent-Reflex project about teaching language agents to act reflectively under noisy environments. That is the clearest signal of the direction: the group is building a series, and this repository is the entry point to one part of it.

## What the models and datasets actually are

The released artefacts are what make a research repository usable by anyone else, and the listing is specific.

The model checkpoints are on Hugging Face under the `dongguanting` account, with collections for both ARPO and AEPO. The commented-out block in the readme names the individual models: Qwen2.5-3B-ARPO, Qwen2.5-7B-ARPO, Llama3.1-8B-ARPO, Qwen3-8B-ARPO-DeepSearch and Qwen3-14B-ARPO-DeepSearch. Later entries add the larger QwQ-32B based models, AEPO-32B and ARPO-32B, released on December 20, 2025, with reported scores of 53.4 and 12.8 for AEPO and 51.5 and 11.2 for ARPO on GAIA and HLE respectively.

The datasets are on Hugging Face too, and the names describe their role: ARPO-SFT-54K for supervised fine-tuning, ARPO-RL-Reasoning-10K for reinforcement learning on reasoning, and ARPO-RL-DeepSearch-1K for the deep search setting.

Three things are worth noticing about the naming. The `-DeepSearch` suffix marks a distinct experimental setup rather than a different size of the same model. The split between SFT and RL datasets reflects the two-stage training approach, where a base model is first adapted to the tool-calling format and then trained with reinforcement learning. And the model sizes span 3B to 32B, which tells you the project was aimed at models small enough to train on a single node, which is the practical constraint that shaped the tool-call acceleration work.

The DeepSearch datasets being separate is also what connects this to the wider agentic RL literature, where the hard part is not learning to call a tool but learning to use a tool when the answer requires many steps.

## The engineering claims are about training throughput, not model quality

The one detailed technical statement in the readme is the July 25 entry about tool-call acceleration and memory optimisation, and it is worth parsing carefully because it is the difference between a usable codebase and an unusable one.

Training Qwen3-14B in one node with a batch size of 128 takes only ten minutes per step, and the project maintains a dynamic cache mechanism to save tool call results in real time.

For agentic reinforcement learning, the step time is dominated by something other than the usual forward and backward pass. A roll-out in an agentic setting involves the model calling tools, waiting for the results, and calling again, often many times, and each tool call is a round trip through the environment. A step that takes ten minutes is dominated by those round trips rather than by the matrix multiplications, and that is why a tool-call cache and memory optimisation are the two levers the project pulled.

The dynamic cache is the more interesting of the two, and its purpose is stated: to save tool call results in real time. In an RL roll-out the same tool call with the same arguments is often repeated, both within one trajectory and across trajectories in the batch, and caching the result turns a network round trip into a dictionary lookup. A batch of 128 times a long agentic trajectory is a lot of repeated calls, so the saving compounds.

This is the kind of detail that separates a repository you can run from one you cannot. A research codebase that ships without its training infrastructure requires you to rebuild the throughput, and multi-day training runs make that difference between a week and a month of work.

The readme also claims the project has a dynamic cache mechanism maintained for this purpose, and given the tool-call acceleration claim in the same entry, the two are presumably the same work seen from two angles.

## What is missing: a licence, and a stable interface

Two things stand out as absent from the readme, and both matter if you are considering adoption rather than citation.

The first is a licence. The project metadata records it as unknown, and the readme's licence section, which is not present in the visible text, does not state one. For a repository of research code with published models and datasets, that is a real obstacle. The code may be free to read and the model weights may be under a Hugging Face licence, and the datasets derived from public corpora may carry their own terms, and none of that is summarised anywhere in the repository. Anyone wanting to build on this has to ask, and the arXiv papers are the wrong place to ask.

The second is a stable interface. This is a research codebase, and the news section shows a project where the algorithms are being revised and extended over time, with AEPO described as an improvement on ARPO and a third project, Agent-Reflex, already appearing in October 2026. The directory names `ARPO/` and `AEPO/` are the natural place to look for the entry points, and the `evaluation/` directory is where the evaluation harness lives, but the readme does not give you a command to run, a configuration file to edit, or a list of scripts.

Compare that with the parts of the readme that are thorough: the release timeline is dated to the day, the model names are exact, the dataset names are exact, and the benchmark scores are given as pairs against GAIA and HLE. The presentation is precise about the artefacts and silent about the mechanics, which is a coherent choice for a paper release, and a poor one for a project people want to build on.

The `LLaMA-Factory/` directory in the tree is a strong hint about what the entry point is, since LLaMA-Factory is the training framework, and the presence of `evaluation/` as a separate top-level directory suggests the evaluation harness is separable from the training code. Both are reasonable places to start reading, but neither is documented in the readme.

## Where this repository is the wrong tool

The first case is a team that wants a production reinforcement learning service or a stable library to depend on. This is a paper release, and its interface will move as the group publishes the next algorithm in the series. The readme says the series will continue to iterate and expand, and a dependency on code that is being restructured around new papers is a dependency you will be maintaining yourself.

The second case is a team without the hardware. The released checkpoints go from 3B to 32B parameters, the training claim is a single node at a batch size of 128, and the datasets are RL datasets with tens of thousands of examples. Reproducing the training runs needs serious GPU memory, and fine-tuning the 32B models needs more. The checkpoints are the realistic entry point for most people, and using them is a different project from training.

The third case is a team that needs a specific algorithm rather than this family. If you want a particular reinforcement learning method for agentic training, this repository gives you ARPO and AEPO, plus the artefacts around them, and nothing else.

The fourth case is a team with a licence compliance requirement. With no stated licence on the code, and separate terms on the models and datasets, a legal review has nothing to review. That is a concrete blocker rather than a formality, and it is the first thing to resolve.

A note on what this repository is good for, then. It is a complete, dated record of two published algorithms with their weights and their data, produced by a group with the resources to train 32B models and to publish at ICLR and WWW. For a researcher in agentic reinforcement learning, that combination is unusual and valuable, and the GAIA and HLE numbers give you something to compare against. The gap is entirely in the packaging.

## Conclusion

This repository fits a researcher who wants to reproduce or build on agentic reinforcement learning with multi-tool agents, and who has the GPU budget to train or fine-tune a 3B to 32B model. It does not fit a team looking for a production reinforcement learning service or a stable library API, since the code is a paper artefact rather than a maintained package and the licence is not stated. Before you use anything from it, read the AEPO directory alongside the ARPO one, because the project says the series will keep expanding, and check the model and dataset collections on Hugging Face rather than assuming a checkpoint exists for the base model you have.

## FAQ

### What algorithms does the RUC-NLPIR/ARPO repository contain?

The codebase, datasets and checkpoints for two algorithms, Agentic Reinforced Policy Optimization (ARPO) and Agentic Entropy-Balanced Policy Optimization (AEPO). The ARPO paper was accepted at ICLR 2026 and the AEPO paper at WWW 2026 as an Oral, with both on arXiv.

### What models and datasets are released with ARPO?

Checkpoints on Hugging Face from Qwen2.5-3B up to QwQ-32B based models, including Llama3.1-8B-ARPO and the Qwen3 DeepSearch variants, plus datasets named ARPO-SFT-54K for supervised fine-tuning, ARPO-RL-Reasoning-10K for reinforcement learning and ARPO-RL-DeepSearch-1K for the deep search setting.

### What engineering work went into ARPO's training pipeline?

The readme records extensive tool-call acceleration and memory optimisation during reinforcement learning training, with the claim that training Qwen3-14B on one node at a batch size of 128 takes ten minutes per step, supported by a dynamic cache mechanism that saves tool call results in real time.

### How do I run the ARPO training code?

The readme does not give a run command or a configuration file. The repository layout points to the `ARPO/` and `AEPO/` directories for the two algorithms, `evaluation/` for the evaluation harness, and `LLaMA-Factory/` for the training framework the work is built on.

## Sources

- [Issues](https://github.com/RUC-NLPIR/ARPO/issues)
- [README](https://github.com/RUC-NLPIR/ARPO/blob/main/README.md)
- [Releases](https://github.com/RUC-NLPIR/ARPO/releases)
- [RUC-NLPIR/ARPO on GitHub](https://github.com/RUC-NLPIR/ARPO)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/ruc-nlpir-arpo