uni-agent: putting reinforcement learning behind any agent harness

The problem uni-agent attacks is training, not agent frameworks

Most agent frameworks are built to make an agent work at inference time. Reinforcement learning needs something different: thousands of trajectories where every turn, tool call and reward stays attached to the session that produced it. Uni-Agent exists to close that gap.

The README states three goals. Bring any existing agent harness into reinforcement learning, unify diverse agent tasks through one extensible interface, and run agents concurrently at scale while collecting traceable trajectories as training-ready data for both SFT and RL.

The audience is research and engineering teams that have already built or borrowed an agent harness and hit the wall between a single successful demo and a training pipeline. That includes people running SWE-bench style tasks, terminal tasks and computer use tasks, all of which need long horizons, sandboxed execution and reproducible scoring.

It is not an agent library in the sense of providing an agent loop for you. The examples directory lists harnesses such as claude_code_swe_task, claude_code_kernel_task, mini_swe_agent, deepseek_harness, agent_aware_router, mem_agent and deepeyes, so the design assumes the harness already exists and the work is on the training side.

The Gateway is where a harness becomes trainable

The mechanism that makes black-box harnesses usable for training is the Uni-Agent Gateway. A harness that can point its model endpoint at an OpenAI- or Anthropic-compatible address can point it at the Gateway instead, and the README summarises the contract as request string in, training tokens out.

That is the whole trick. Rather than asking a harness to expose internal state, the Gateway intercepts the model calls and records the tokens that would have been generated. A harness such as Claude Code or Mini-SWE-Agent can then be driven through the same pipeline as a white-box agent written inside Uni-Agent, because the training signal comes from the token stream rather than from harness internals.

The consequence is that a new harness can be added without writing a training integration for it. If it speaks an OpenAI- or Anthropic-compatible API and points at the Gateway, it becomes a candidate for the same rollout, evaluation and RL path. The examples folder includes a gateway example and a blackbox_recipes directory, which is where you would look for how this is wired in practice.

The limit of the approach is worth stating plainly. Because the training signal is the token stream rather than the harness state, the Gateway sees what the model produced, not necessarily every internal decision the harness made. Harnesses with side channels, such as their own memory or router state, have to express that through the request and response surface if they want it to be trained on.

Agent, Tool, Task and Sandbox are kept deliberately separate

Uni-Agent also supports white-box agents, built from four abstractions the README names explicitly: Agent, Tool, Task and Sandbox. The claim is that agent logic, tools, task environments, sandbox backends and rewards can each be customised independently while reusing the same evaluation and training runtime.

That separation is the architectural opinion of the project. Agent logic decides what to do, Task decides what counts as a correct outcome and supplies the reward, Sandbox decides where code runs, and Tool defines the actions available. Because each is pluggable, a team can change the reward function or swap the sandbox backend without rewriting the agent loop, and a team that wants a different agent implementation can keep the same tasks and evaluation code.

The examples layout supports that reading. There are separate directories for the gateway, for inference, for black-box recipes, for individual harnesses and for the quickstart, plus a quickstart directory whose training subfolder holds the runnable RL scripts. The uni_agent directory holds the framework itself, with tests and a docs directory alongside, and pyproject.toml at the root with ruff and mypy configuration for linting and type checking.

One wrinkle in the layout: the repository also carries a verl directory and a .gitmodules file, and the ruff configuration excludes the verl path from linting while mypy excludes it too. The project is packaged to work with recent verl features, which the README names explicitly, including colocate_async.

A thousand stateful sessions, with traces kept apart

The scaling claim is specific. The README describes running 1,000 or more long-horizon, stateful sessions with distributed workers, pooled Gateway sessions, isolated sandboxes and asynchronous scheduling. The emphasis on stateful matters, because a long-horizon agent accumulates history, and the session is the unit that carries that history.

The other half of the claim is traceability. Every trajectory, log and reward stays associated with the correct session, which the project says makes evaluation, RL training and data synthesis reliable. In practice this is the hard part of concurrent rollout: if a log line or a reward gets attached to the wrong session, the training signal is corrupted in ways that are hard to notice.

Settlement of the sandbox backends is the price of isolation. The README acknowledges that large-scale parallel interaction and verification rely on remote sandbox backends, and names veFaaS from Volcengine as a serverless backend for launching sandboxes elastically at scale, and Modal for spinning up isolated, reproducible sandbox environments. The remote sandboxes are what make a thousand concurrent sessions possible, and they are also the component most likely to be the largest line item and the biggest operational dependency in a real deployment.

A second task format is supported through Harbor. The README states that Uni-Agent integrates Harbor as an additional task format, so all Harbor tasks can run through the same inference pipeline. That is convenient for teams already writing tasks in that format.

Install and first use is a documentation walkthrough, not a one-liner

The project does not publish an install command on the repository page, so the setup steps live in the documentation. The README lays out a four-step path: install Uni-Agent with support for the latest verl features such as colocate_async, launch a sandbox and run code locally or with cloud services, run agent inference at scale for benchmarking and trajectory generation, then train an agent with RL using reproducible scripts.

The packaging metadata does pin the runtime constraints. The project requires Python 3.10 up to but not including 3.13, and the distribution is named uni-agent under the Apache-2.0 licence:

name = "uni-agent"
requires-python = ">=3.10,<3.13"

A first real use therefore starts at the installation page in the documentation, then the sandbox launch page, which is where you find out whether you are running sandboxes locally or through one of the cloud services. Nothing in the repository page suggests a local-only path works for the thousand-session case, since the acknowledged backends are remote serverless platforms.

If you want to look at the framework rather than run it, the docs directory and the examples tree are the place to browse, and CONTRIBUTING.md describes how to get started with a contribution. The repository also carries a .pre-commit-config.yaml and a .codecov.yml, so a pull request goes through local hooks and coverage reporting.

RL recipes are published as scripts with result tables attached

The training side ships as recipes under examples/quickstart/training, and the README states that the project publishes runnable recipes with complete configurations, benchmark settings, result tables and learning curves, each intended as a tested starting point.

The objectives named are GRPO and GSPO-style, with partial rollout support and fully async training recipes across several tasks, models and datasets. The result table compares base scores with RL scores under two settings. Under Fully Async at 100 turns and 128K, Qwen3-30B-A3B with ReAct on R2E-Gym moves from 22.2 to 36.8, and Qwen3-Coder-30B-A3B on R2E-Gym moves from 46.2 to 52.0. Under Colocate Async at 200 turns and 128K, Qwen3.5-4B with ReAct on SWE-reBench goes from 52.9 to 59.8, and the same model with Claude Code goes from 40.2 to 46.2.

One run is documented in more detail. The Qwen3.5-4B entry uses train_qwen3p5_dense.sh on 64 H100 GPUs, and its SWE-Bench Verified score peaks at step 50. That is a useful reality check on the cost of this kind of work: the small-looking 4B model run is not a laptop experiment.

The inference side has its own table, covering SWE-Bench Verified with ReAct and Claude Code agents, SWE-Bench Multilingual, and Terminal-Bench v2.0 and v2.1, with the model, turn limit and context length given per row. The README frames these as a selection of representative results, with detailed settings and reference results in the Inference and Verification section of the documentation.

Version 0.1.0 and an Apache licence say little about stability

The release history is short. A release candidate, v0.1.0rc1, was published on 2026-08-26, and v0.1.0 followed on 2026-09-23. The last push to the repository was on 2026-10-20, and the repository is not archived, so work continues after that tag.

A project at v0.1.0 with a young release history should be read as an early framework with an active surface area rather than a stable dependency. The public API is likely to move, and the recipes, which are the thing you would copy, will change with it. Pinning to a tag rather than tracking main is the safer habit here, though the documentation does not say whether the recipes are expected to stay compatible across versions.

The licence is Apache-2.0, with the licence file referenced in the packaging metadata and a LICENSE file at the repository root. That is a permissive licence with an explicit patent grant and a requirement to preserve notices, which is generally convenient for both research use and commercial use. That is an observation about the terms, not advice on how to deploy it.

There is also a roadmap, referenced as the Uni-Agent 26Q3 Roadmap in the issues tracker, and a citation entry the README asks users to include. Both point to the same thing: the project is at the stage where it wants academic and industry users to anchor on it, and where published numbers matter more to the maintainers than API stability.

Editorial conclusion

uni-agent suits teams that already have a working agent harness and now need trajectories, evaluation and RL training at the same scale as their inference, and it is a reasonable base for people building white-box agents from the Agent, Tool, Task and Sandbox abstractions. It is the wrong tool for a single-agent application with no training budget, and for anyone who cannot provision remote sandbox backends such as veFaaS or Modal. Before committing, read the installation page for the verl features such as colocate_async, confirm your Python version falls inside >=3.10,<3.13, and reproduce one of the published recipes under examples/quickstart/training before writing your own.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. verl-project/uni-agent on GitHub