RPent

What RPent provides

RPent stands for Recursive Physical Agent and is an open framework for building embodied agents that evolve through repeated interaction with the physical world. The README states it does not prescribe a single foundation model. Instead it offers a recursive agent framework that brings together heterogeneous capabilities such as perception, reasoning, memory, execution, and self evolution into one physical agent. The design principles are service oriented, standardized, and composable, which means capabilities deploy as reusable services connected through unified interfaces and composed into different agents. The project positions itself as infrastructure for the physical world rather than a traditional robot control framework, where intelligence is not only deployed but continuously built and expanded. The first publication from the project is described as a paper on steering frozen vision language action models into reliable manipulation primitives through memory guided agents. The feature matrix shows the moving parts: an agentic planner that can use Claude Code, Codex, or a custom planner; action primitives spanning vision language action models such as Pi0.5, RLDX-1, and LingBot-VLA plus a WAM entry; simulators including LIBERO-PRO, RoboCasa, and RoboTwin; and real world targets such as Franka and SO-101. That spread shows the framework is meant to bridge simulation and hardware. The source lives in the rlinf-rpent repository on GitHub and is implemented in Python, with the license recorded as Apache-2.0 in the project metadata.

Supported backends

The README enumerates which backends RPent can drive today. On the planner side, Claude Code, Codex, and a custom planner are marked supported, and the CLI lets you pick the planner and the model, with api and codex as additional options in the configuration docs. On action primitives, the vision language action column lists Pi0.5, RLDX-1, and LingBot-VLA as supported, with a WAM entry for DreamZero noted without a checkmark. The simulator column marks LIBERO-PRO, RoboCasa, and RoboTwin as supported, which covers the main training and evaluation environments. The real world column lists Franka and SO-101, again with the lighter marking that suggests partial or in progress support. The installation pulls a full end to end stack by default, which the README describes as the openpi Pi0.5 vision language action model plus the LIBERO-PRO and RoboCasa365 simulators and SAM 3.0 on the RLinf runtime. If you do not need the whole stack, narrower install extras are documented. The project also builds on the simulators, vision language action models, and training infrastructure of RLinf, and on agent SDKs from the broader open source community such as pydantic-ai, the Claude Agent SDK, and the OpenAI Codex SDK. That dependency list tells you RPent is a coordination layer over existing models and environments rather than a model trainer itself.

Running a task

Getting a task running starts with a pip install of the cloned repo using the full extra, then downloading LIBERO-PRO assets with a command that skips existing files. The README shows an environment setup with an Anthropic base URL and API key, a downloaded Pi0.5 checkpoint, a SAM 3.0 checkpoint, and a LIBERO type of pro. A sample run targets the libero robot with a suite, task, and seed, using the claude_code planner and a specific model. The CLI has many options: robot and suite are required, while task, seed, planner, model, max turns, max tokens, reasoning effort, and several endpoint overrides are optional. The reasoning effort setting ranges from none to xhigh and the docs note that disabling reasoning cut average runtime from about 13.2 to 7.9 minutes in one evaluation, with higher effort possibly raising success rate. There is an interactive mode for steering the agent from a terminal, a dashboard mode that streams reasoning and camera views, and an explore mode for resettable attempts with memory distillation and automatic merge. The evaluation default can use a local memory profile against a corpus of global, suite, and task memories. The docs also cover RoboCasa and RoboTwin as separate entrypoints. Taken together, the CLI is the main surface, and the README points to per feature docs for planners, LIBERO usage, and the other simulators.

Editorial conclusion

The framework is written in Python, released under the Apache-2.0 license, and installs with a single pip command from the source repository.