UniLab: contract-driven robot reinforcement learning that survives a change of simulator

One task definition, many simulators and accelerators

UniLab is configurable infrastructure for robot reinforcement learning, and its stated aim is a single task-authoring surface for locomotion, manipulation and motion tracking. The problem it addresses is concrete: in most robot learning stacks the task environment ends up welded to one simulator, so swapping MuJoCo for another backend, or moving training onto a different accelerator, means rewriting the environment lifecycle.

The approach is to describe a task with Hydra, assemble it from manager terms, select a physics backend, and train or evaluate through one CLI. The same task-facing contract connects CPU, GPU and external-worker simulation to the learner runtime.

Who it is for is fairly narrow and worth stating plainly. This is research and platform infrastructure for teams that run many robot learning jobs, not a tutorial library. The evidence-graded approach is telling: backend and task maturity are graded, and the documentation directs readers to a support matrix to choose a tested combination rather than promising uniform behaviour everywhere. The project has been accepted to CoRL 2026, with a paper on arXiv at 2605.30313, and it also serves as a shared task and training surface that robot-specific repositories consume.

Manager terms assembled in Hydra owner YAML

The core authoring idea is stated as configure, don't code. Actions, observations, rewards, terminations, events, commands, curricula and metrics are manager terms, and they are assembled in Hydra owner YAML.

That has a concrete consequence for how a team works. A variant built from existing terms needs no new environment class, and the documentation goes further, noting that such a variant often needs no Python code at all. If your experimental axis is a different reward function, a different termination condition, or a curriculum schedule, you express it as configuration and the environment lifecycle stays put.

The dependency list backs this up. Hydra and OmegaConf are runtime dependencies, not optional extras, and the package pulls in rsl-rl-lib for the reinforcement learning side, gymnasium for the environment interface, tensordict for tensor handling, and lark for parsing. wandb is there for experiment tracking and tensorboard is included as well, so both tracking paths are wired in from the start rather than left to the user.

The limitation sits next to the flexibility. Configuration-driven assembly works when the variation you need can be expressed as a term or a composition of terms. A task that needs new physics behaviour, a new sensor model, or a bespoke observation encoding still requires writing code against the contracts, and the documentation does not describe a plugin or extension point for third-party terms.

The SimBackend contract and an evidence-graded backend matrix

Registered simulators meet a public contract named SimBackend. You choose a backend with the --sim flag, and when a matching task owner exists, task authoring and train or eval stay consistent while backend-specific details remain explicit. The README shows the same task and algorithm moving between two backends, which is the whole point of the abstraction.

The backends visible in the examples are motrix and mujoco, and the Docker image description names the scoped tensor Manager runtime with MuJoCo and MJWarp as physics backends. The extras in the build and Makefile targets line up with this, since the sync targets install a mujoco extra and the container build adds a mjwarp extra.

What the contract buys and what it does not is the interesting part. A backend that satisfies SimBackend can be dropped in, but the task and backend combination still has to have been exercised, and the support matrix records the evidence level for each pairing. That is an honest way to handle a multi-backend stack, and it is also a real constraint: you are limited to combinations somebody has tested, and the grading means the quality of support is uneven by design.

Sim-to-sim deployment has its own documentation page, including a backend swap guide, which is the scenario the abstraction exists for: train on one backend, then evaluate the same task on another.

Solver and learner devices kept independent

One of the more opinionated design choices is that the solver and the learner devices stay independent. CPU-parallel, native or external-worker simulation can feed an accelerator learner without the simulation having to become a CUDA-resident simulator first. The learner itself can run on CUDA, ROCm, MPS or XPU.

This matters because physics simulation is often the bottleneck and because moving a simulator into a GPU memory space is not always possible. Under the UniLab arrangement, a team with a strong GPU but a simulator that only runs on CPU can still use the accelerator for the learner, and a team with external workers can feed a single learner process. That is a different arrangement from frameworks that assume the environment lives on the same device as the policy.

Platform coverage is documented for Windows, Apple Silicon macOS, Linux CUDA, AMD ROCm and Intel XPU. The build files show how each is handled. The ROCm path is a separate dependency file, pyproject.rocm.toml, with its own lock file uv.rocm.lock, and the Makefile has a sync-rocm target that copies both over the standard ones before syncing and copies the lock file back afterwards. The XPU path installs torch separately with a pinned version and the xpu backend:

uv pip install torch==2.7.0 --torch-backend xpu

The same reasoning appears in the dependency ranges. torch is constrained as a range rather than an exact pin so that ROCm users can substitute a ROCm torch build, while uv.lock pins the tested CUDA builds through tool.uv.sources. That is a deliberate choice to keep one codebase serving several accelerator vendors, and it is also where upgrade risk concentrates.

Quick start with uv, make setup and a downloaded checkpoint

The supported source workflow uses uv, and the README gives the shortest path to a policy demo:

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/unilabsim/UniLab.git
cd UniLab

make setup
# Downloads the checkpoint and assets from Hugging Face on first run.
uv run demo dance

The Makefile shows what setup does. The sync target runs uv sync with the mujoco extra, and setup then runs a completion install step through the package's own command:

uv sync --extra mujoco

uv run --no-sync unilab-complete install

The comment in the quick start is the detail to plan around. The checkpoint and assets come from Hugging Face on the first run, and the pyproject build configuration excludes downloaded robot meshes from the wheel, listing the bot asset directories and the g1 and go2 asset and texture paths as source-exclude entries. The intent is stated in the file: downloaded robot meshes stay out of the wheel, kept in sync with the robot asset specification in src/unilab/assets/hub.py. In other words, a machine without network access to that host cannot complete setup, and the assets are fetched rather than shipped.

For Windows, macOS, CUDA, ROCm, XPU, optional backends and headless rendering, the documentation points to a separate installation guide and a quick demo guide rather than trying to cover those paths in the README.

One command shape for training, evaluation and algorithm changes

Training and evaluation share a command line where the algorithm, task and simulator stay visible as separate flags:

# Train and replay one task with Motrix.
uv run train --algo ppo --task go2_joystick_flat --sim motrix
uv run eval --algo ppo --task go2_joystick_flat --sim motrix --load-run -1

# Use the same task-facing command with another configured backend.
uv run train --algo ppo --task go2_joystick_flat --sim mujoco

# Replay-based off-policy path.
uv run train --algo sac --task g1_walk_flat --sim mujoco

Two mechanisms are worth reading out of these lines. The --load-run -1 argument on eval means the most recent run, which is what makes the train then replay loop usable without tracking run identifiers by hand. And the same --task with a different --sim is the demonstration of the backend contract: the task does not change, only the physics runtime behind it.

The replay-based off-policy path uses SAC and FlashSAC, where simulation data collection overlaps with learner updates. The paper reports 3 to 10 times end-to-end gains on representative configurations, and the documentation itself points to the Why UniLab page for the scope and the measurements behind those figures. Treat that range as a report about specific configurations rather than a general expectation.

Resume, W&B, Hydra overrides, playback, backend setup and the full command matrix live in the training guide, the backend guide and the support matrix rather than in the README, so the README gives you the shape of the interface and the guides give you the surface area.

Ecosystem, packaging and what an upgrade costs you

UniLab is built to be consumed by robot-specific repositories that ship their own recipes while sharing the same task, backend and RL contracts. The downstream examples listed are MicroDuck RL, EngineAI RL, Unitree RL, Wuji, Legged Manipulation, Sharpa and Cricket-Gym, which tells you the intended shape of the ecosystem: robot repositories on top, one contract layer below.

Packaging is straightforward and worth knowing for CI. The project name on PyPI is unilab, the version in pyproject.toml is 1.3.5, and the build backend is uv_build with a constrained uv_build requirement. Python support runs from 3.10 up to but not including 3.14, and the classifiers list 3.10 through 3.13. The licence is Apache-2.0 with the licence text shipped as a LICENSE file.

The container path is a Linux NVIDIA/CUDA image based on nvidia/cuda 12.8.0 with cuDNN on Ubuntu 22.04, and it installs uv, syncs with the mujoco and mjwarp extras, and defaults to showing the training help:

FROM nvidia/cuda:12.8.0-cudnn-runtime-ubuntu22.04

For development, the Makefile offers formatting through ruff, typing through mypy and pyright, and test tiers that separate fast tests from slow ones and a benchmark smoke test. The check target runs formatting, typing and the test lint pass together.

On upgrades, the release numbering is dense rather than semantic. v1.3.5 was published on 2026-10-12 and v1.3.4 on 2026-10-10, with the last push on 2026-10-20, so the project is moving quickly and a pinned version will age within days. The practical cost of moving forward is that task configurations and the support matrix can shift between patch releases, and the matrix is the artifact to re-read when a combination you relied on changes.

Editorial conclusion

Adopt UniLab if you are running robot reinforcement learning across more than one simulator or accelerator and have hit the point where task code and simulator code are tangled. Skip it for a single-backend research script, since the Hydra and contract machinery buys nothing there, and treat the support matrix as binding rather than advisory. Before starting, check the evidence grade of the exact backend and task pair you intend to use, confirm that your Python version falls inside the >=3.10,<3.14 range, and plan for the checkpoint and robot asset download from Hugging Face on the first run.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Motphys/UniLab on GitHub
4. README
5. Releases