# slime-agentic: agentic RL reproductions built on the slime framework

> slime-agentic is a repository of reproductions of agentic reinforcement learning methods, each built on the slime post-training framework and each shipping a trained checkpoint on Hugging Face. The three methods implemented so far are AgentFlow, MemAgent and ToolOrchestra, and all three are GRPO training runs with a carefully chosen loss mask.

**LMIS-ORG/slime-agentic** — A project implementing various agentic RL  based on the Slime post-training framework

- Repository: https://github.com/LMIS-ORG/slime-agentic
- Stars: 544 · Forks: 40
- Language: Python
- License: Apache-2.0
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/lmis-org-slime-agentic

## A reproduction repository, not a framework

The repository describes itself as reproductions of various agentic RL training methods built on top of the slime framework, and the README is precise about why that matters. Slime exposes flexible hooks for custom generate functions and reward functions, making it straightforward to integrate multi-step agent rollouts into RL training pipelines.

That is the whole architectural bet of the project. A multi-step agent rollout is not a single prompt and a single completion, it is a sequence of model calls interleaved with tool executions and observations, and the training loop has to decide which tokens count as the policy's output and which are context. Slime provides the hooks for that, and this repository fills them in with three specific algorithms.

The implementations live under the agentic/ directory, and the top level of the repository shows the shape of a full training stack rather than a small research script. There is a slime/ package and a slime_plugins/ package, train.py and train_async.py as entry points, a tools/ directory, a scripts/ directory, a docker/ directory, a build_conda.sh, a requirements.txt, a pyproject.toml and a setup.py. There is also a docs/ directory and a tests/ directory, and a Chinese README at README_zh.md.

The licensing situation is worth separating from the framework's. The repository is Apache-2.0, and the setup.py at the root is the slime package's own setup, with name slime, version 0.2.2, author the slime Team, and packages found for slime* and slime_plugins*. The Python floor is 3.10, and the classifiers list 3.10 through 3.12 and the environment as GPU NVIDIA CUDA.

## AgentFlow, and what the loss mask decides

AgentFlow is the first of the three, and it reproduces the core idea of the AgentFlow paper, arXiv 2510.05592: extending single-step LLM inference into a multi-turn Planner, Executor, Verifier agent loop, and applying RL signals with GRPO to the Planner's generation trajectory. The stated benefit is that the model improves its tool-use and reasoning capabilities without requiring manually annotated intermediate steps.

The architecture diagram in the README shows the mechanism, and the annotations on it are the interesting part because they mark where the loss applies.

```text
Input question
  │
  ▼
Planner.plan()              ← Analyze the problem and devise a solution strategy (loss_mask=1)
  │
  └─► for step in range(max_steps):
        │
        ├─ Planner.generate_next_step()             ← Select next tool and sub-goal (loss_mask=1)
        ├─ Executor.generate_tool_command()
        │  + execute_command()                       ← Invoke tool (excluded from sequence)
        ├─ Verifier.verificate_context()             ← Decide whether to continue (excluded)
        └─ Memory.add_action()                       ← Record execution result
  │
  ▼
Planner.generate_final_output()   ← Summarize results and produce final answer (loss_mask=0)
  │
  ▼
Rewarder.compute_reward()         ← LLM-as-Judge: compare model answer with ground truth
```

The loss_mask is doing the real work. The Planner's planning call and its per-step next-tool selection are trained on, because those are the policy decisions. The Executor's tool command generation and the Verifier's continuation decision are excluded from the sequence, and so is the tool execution itself, because those are not the policy being improved. The final answer summarisation is marked loss_mask=0, which is the opposite of what you might expect and is worth thinking about: the answer that gets the reward is not the text that gets the gradient.

The reward comes from a Rewarder that is an LLM-as-Judge comparing the model answer with the ground truth, which is the standard way to score a free-form answer against a reference without a task-specific checker.

The tools available are two: a base_generator that is a general-purpose text generation tool answering sub-tasks directly via the LLM, and a python_coder for Python code generation and execution for math computation and algorithmic problem solving. The training configuration is GRPO with a KL divergence constraint using the low_var_kl variant, the model is Qwen2.5-7B and is described as replaceable, the dataset is DAPO-Math-17K, the inference engine is SGLang with the Planner and Executor on separate ports, and the evaluation is AIME 2024.

## MemAgent, and reward amortised across turns

MemAgent is the second method, and it is the one with a different training objective. It reproduces the core idea of the MemAgent paper, arXiv 2507.02259: compressing arbitrarily long documents into a fixed-size recurrent memory via a chunk-by-chunk LLM update loop, then answering questions from memory alone.

The diagram shows the loop.

```text
Input: question + long document
  │
  ▼
memory = "No previous memory"
  │
  └─► for chunk in split(document, chunk_tokens):
        │
        └─ LLM(problem, memory, chunk) → updated memory   (loss_mask=1)
  │
  ▼
LLM(problem, memory) → final answer in \boxed{}           (loss_mask=0)
  │
  ▼
Reward: exact-match / F1 against ground truth
         (distributed evenly across all memory-update turns)
```

The objective is described as a Multi-Conversation training objective, and the README explains the mechanics. Each memory-update turn is an independent training sequence, and the reward is evenly amortised across all turns in the conversation via custom_convert, matching the Multi-Conv RL objective in the MemAgent paper. The reward itself is exact-match or F1 against ground truth, which is a much cheaper and more deterministic scorer than the LLM judge AgentFlow uses.

The inversion of the loss mask is the same idea as in AgentFlow. Here the memory-update turns carry loss_mask=1, because writing the memory is the skill being trained, and the final answer carries loss_mask=0, because the answer is read out of the memory rather than produced by the trained behaviour. The model never sees the full context, which is the point: the memory is fixed size, so the training signal has to teach retention rather than recall.

The setup is GRPO with the same low_var_kl constraint, Qwen2.5-7B as the replaceable model, the HotpotQA dataset from BytedTsinghua-SIA, and SGLang with YaRN enabled at a context length of 131072 tokens. The evaluation is RULER-HQA across context lengths from 7K to 448K tokens, five runs with the best score reported, and the README states the model is trained on a 7B base and consistently outperforms all baselines including much larger models across all context lengths.

## ToolOrchestra, and routing as a trained behaviour

ToolOrchestra is the third method, and it covers the routing case. It reproduces the core idea of the ToolOrchestra paper, arXiv 2511.21689: an Orchestrator-Expert multi-agent framework for RL training, where a central Orchestrator LLM learns to route tasks to the best specialised expert model and the corresponding tools through multi-turn tool calls. GRPO is applied to the Orchestrator's decision trajectory, again without manually annotated intermediate steps.

The architecture is a loop with three exits. The Orchestrator decides which tool to call and that decision is trained. Then parse_tool_call parses a tool call from the model output, the tool is called with the result excluded from the loss, and call_expert routes to specialist models running on separate ports, also excluded. An answer terminates the loop and produces the GenerationOutput.

That output is described in the README as token_ids plus log_probs for all turns concatenated, with a loss_mask where the Orchestrator output is 1 and the tool result is 0. The distinction from AgentFlow is that the trained decision here is a routing decision, which means the reward is a measure of whether the right specialist was chosen, and the expert models themselves are never trained.

The retrieval side is a FAISS retrieval service on port 8000, and the specialised experts run on their own ports, so the topology is several inference servers rather than the two that AgentFlow needs for its Planner and Executor.

The training configuration is again GRPO with low_var_kl, the model here is Qwen3-8B and is replaceable, and the dataset is τ²-Bench. The reported comparison on τ²-Bench gives a baseline of 0.278 for Qwen3-8B against 0.388 for the trained ToolOrchestra, an improvement of +0.110.

The three methods together make a coherent set: a tool-use loop, a memory compression loop, and a routing loop. Each one isolates a different place where a multi-step agent makes a decision, and each one marks exactly that decision with the loss mask.

## Reproductions, weights, and the licensing of the setup

Every method releases its trained weights on Hugging Face under the LMIS-ORG organisation: LMIS-ORG/AgentFlow_Slime_Agentic_Qwen2.5_7B, LMIS-ORG/MemAgent_Slime_Agentic_Qwen2.5_7B and LMIS-ORG/ToolOrchestra_Slime_Agentic_Qwen3_8B. Publishing the checkpoints is what makes these reproductions useful to a third party, since a reader can inspect the model rather than trust the number in the table.

The reported results are worth reading with the right expectations. AgentFlow goes from 10.0% on AIME 2024 for the baseline to 30.0% for the trained model, an improvement of +20.0%. ToolOrchestra goes from 0.278 to 0.388 on τ²-Bench. Both are single figures from single benchmarks in the README's tables, and the tables are accompanied by result images rather than run logs, so a reader cannot see the variance from the documentation alone. The MemAgent entry is the one that states its protocol most explicitly, five runs with the best score reported, which is a choice that flatters the number.

The dependency list in requirements.txt describes the stack the training runs need. It includes accelerate, blobfile, datasets, httpx with http2, mcp with cli, memray for debugging, numba, omegaconf, pillow, pylatexenc, pyyaml, qwen_vl_utils for VLM work, ray with the default extra, ring_flash_attn, sglang-router at 0.2.3 or newer, tensorboard, transformers and wandb.

Two details in that list are worth flagging. sglang-router is a separate process from SGLang itself and is what routes requests to the different model servers that these architectures need, which is why the per-method configurations mention separate ports. And memray is a memory profiler, included for debugging, with a comment in the file noting it is lightweight and could move to a development mode when using pyproject.toml.

The build and lint configuration is also documented, in pyproject.toml. Isort uses a black-compatible profile with a line length of 119 and a target of Python 3.10, black is configured with a line length of 119, and ruff selects E, F, B and UP while ignoring E402 for module-import-not-at-top-of-file and E501 for line length. The pytest configuration adds --verbose, --pyargs, --durations=0 and --strict-markers, with testpaths pointing at ./tests and a norecursedirs list that excludes external, examples, docs, scripts and tools.

## Where this repository is the wrong place to start

Three cases point elsewhere. The first is a team that wants an agentic RL training framework rather than reproductions of existing methods. Slime itself is the separate project, hosted at the THUDM organisation, and this repository is a set of implementations sitting on top of it. If you want to build a new method, you need slime and its hooks, and this repository is a worked example of how those hooks are used rather than a framework to extend.

The second is a team that expects the results to be independently verifiable from the repository. The README publishes summary numbers and images, and the configuration lists, but the training runs themselves need a cluster with SGLang, Ray, a CUDA-class GPU and a distributed setup with the Planner, Executor, experts or retrieval service on separate ports. Reproducing a number is a matter of assembling that environment, and the per-method directories are where the launch instructions live.

The third is a team that wants a stable, released version. The repository has no GitHub releases, the version in setup.py is 0.2.2 for the slime package, and the last push to the repository was on 2026-04-11, which is more than six months before the current date. That is a specific fact to plan around rather than a reason to avoid the code, since the trained weights on Hugging Face are versioned artefacts in their own right, but it does mean the training code is not moving quickly.

The alternative worth weighing is running the original implementations from each paper's own repository, and the difference in approach is the point of this project rather than a deficiency in it. Reproducing a paper from its authors' code gives the authors' exact result; reproducing it on a common framework with the same loss-mask discipline and the same GRPO configuration gives a result that is comparable across methods, because the same training stack produced all three. The check that matters is therefore whether you want a working system or a controlled comparison, because a controlled comparison is only controlled if every arm runs on the same stack, which is what this repository provides and what three separate codebases do not.

## Conclusion

slime-agentic fits a research group that wants to reproduce published agentic RL results on a common training stack rather than from each paper's own code, and who want the trained weights to inspect. It does not fit a team that needs a training framework, because this is a set of reproductions sitting on top of the separate slime project, and it does not fit anyone expecting new methods, since the repository's stated job is reproduction. Before you start, read the per-method directory, agentic/agentflow and agentic/memagent, because that is where the training parameters and launch instructions live, and check the requirements list, since SGLang, Ray and a CUDA-class GPU are implied by the setup rather than spelled out in the README.

## FAQ

### What is slime-agentic?

It is a repository of reproductions of agentic reinforcement learning methods built on top of the slime post-training framework, which exposes hooks for custom generate and reward functions so multi-step agent rollouts can be integrated into RL training pipelines. All implementations live under the agentic/ directory.

### Which agentic RL methods does slime-agentic implement?

Three at present: AgentFlow, a Planner-Executor-Verifier loop trained with GRPO on the Planner trajectory; MemAgent, which compresses long documents into a fixed-size recurrent memory with a Multi-Conversation objective; and ToolOrchestra, an Orchestrator-Expert framework where the Orchestrator learns to route tasks to specialist models and their tools.

### What do the loss_mask annotations in slime-agentic mean?

They mark which tokens in a multi-turn rollout belong to the policy being trained. In AgentFlow the Planner's plan and next-step selection are masked in while tool execution and the final answer are masked out, and in MemAgent the memory-update turns are masked in while the final answer read from memory is masked out.

### Where can I find the trained weights for the slime-agentic reproductions?

They are published on Hugging Face under the LMIS-ORG organisation, as LMIS-ORG/AgentFlow_Slime_Agentic_Qwen2.5_7B, LMIS-ORG/MemAgent_Slime_Agentic_Qwen2.5_7B and LMIS-ORG/ToolOrchestra_Slime_Agentic_Qwen3_8B.

### What does the slime-agentic training stack require?

The requirements list includes ray, sglang-router, ring_flash_attn, transformers, accelerate, wandb, datasets and mcp, and the setup.py requires Python 3.10 or newer and classifies the environment as GPU NVIDIA CUDA. The per-method configurations use SGLang as the inference engine with different model components on separate ports.

## Sources

- [Issues](https://github.com/LMIS-ORG/slime-agentic/issues)
- [License: Apache-2.0](https://github.com/LMIS-ORG/slime-agentic/blob/main/LICENSE)
- [LMIS-ORG/slime-agentic on GitHub](https://github.com/LMIS-ORG/slime-agentic)
- [README](https://github.com/LMIS-ORG/slime-agentic/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/lmis-org-slime-agentic