# robotic_world_model is an Isaac Lab extension for model-based RL on legged robots

> This repository adds Robotic World Model and Uncertainty-Aware Robotic World Model environments and training pipelines to Isaac Lab, letting a robot policy be trained on a learned neural dynamics model instead of a physics simulator. It suits a robotics researcher reproducing either paper, and it is tied to a specific Isaac Lab and Python stack.

**leggedrobotics/robotic_world_model** — Repository for our papers: Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics and Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots

- Repository: https://github.com/leggedrobotics/robotic_world_model
- Website: https://sites.google.com/view/roboticworldmodel
- Stars: 693 · Forks: 55
- Language: Python
- License: Apache-2.0
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/leggedrobotics-robotic-world-model

## This is a paper release, not a general purpose RL library

The repository description gives the scope away. It is a repository for papers, namely Robotic World Model, a neural network simulator for policy optimization in robotics, and Uncertainty-Aware Robotic World Model, which makes offline model-based reinforcement learning work on real robots. The title of the README is more precise about the artefact itself: it is a Robotic World Model Extension for Isaac Lab. That is the frame to keep in mind for every decision about it. The code exists so that the two papers can be run, reproduced and extended on top of a simulator the authors already use, and it is organised around the environments and training pipelines those two methods need. The authorship is given as Chenhao Li, Andreas Krause and Marco Hutter, with the affiliation listed as the ETH AI Center, the Learning and Adaptive Systems Group and the Robotic Systems Lab at ETH Zurich. The project page lives at a Google Sites URL, and the papers are on arXiv at 2501.10100 and 2504.16680.

## The core idea is training a policy on learned dynamics instead of a simulator

The overview lists what the extension enables, and one line is the whole point. Joint training of policies and neural dynamics models in Isaac Lab is the online mode, where a simulator is still in the loop while the model learns. Training of policies with learned neural network dynamics without any simulator is the offline mode, where the learned model replaces the physics engine entirely. Beyond that it enables evaluation of model-based against model-free policies, visualization of autoregressive imagination rollouts from the learned dynamics, and visualization of trained policies in Isaac Lab. The second paper is what makes the offline mode practical. The Uncertainty-Aware variant adds an ensemble whose disagreement is used to decide when the learned model is trustworthy, which is the mechanism that allows a policy trained purely on imagined rollouts to work on real hardware. The two papers are linked from the README with their arXiv identifiers and separate project pages.

## The online and offline training paths are two different algorithms

The policy training section is explicit that there are two options, and the difference is not a flag but a different algorithm with a different assumption. Option 1 trains the policy in imagination online, where additional environment interactions are continually collected using the latest policy to update the dynamics model, which the README ties to RWM and MBPO-PPO from the first paper. Option 2 trains the policy in imagination offline, where no additional environment interactions are collected and the policy has to rely on the static dynamics model, tied to RWM-U and MOPO-PPO from the second paper. That distinction is the whole contribution of the second work. An online imagination loop keeps correcting the model with fresh data, so model error stays bounded. An offline one cannot, so the model has to carry its own uncertainty and the policy has to avoid the parts of the state space where the model is unsure. If you are choosing between them, the deciding question is whether you can afford simulator time at training, not which one is newer.

## The model heads are configured, not coded

The world model pretraining section explains that Robotic World Model is a model-based reinforcement learning algorithm that learns a dynamics model and a policy concurrently, and the configuration surface is where you see that structure. Model inputs and outputs are configured under ObservationsCfg_PRETRAIN in the AnymalDFlatEnvCfg_PRETRAIN class, and the available components map to distinct heads. SystemStateCfg is the state input and output head. SystemActionCfg is the action input. SystemExtensionCfg is a continuous privileged output head, for example rewards. SystemContactCfg is a binary privileged output head for contacts, and SystemTerminationCfg is a binary privileged output head for terminations. The split between ordinary state and privileged outputs is what lets a single model be used both as a policy input and as a source of training signal, and the binary heads are a compact way to predict event-style quantities such as contact and termination.

## Ensemble size and forecast horizon are the knobs that matter

The architecture and training hyperparameters are configured under RslRlSystemDynamicsCfg and RslRlMbrlPpoAlgorithmCfg in the AnymalDFlatPPOPretrainRunnerCfg class, and four options are listed. The ensemble size is the ensemble size for uncertainty estimation, which is the parameter that makes the uncertainty-aware variant work, since disagreement between ensemble members is the signal for where the model does not know the dynamics. The history horizon is the stacked history horizon, so the model sees a window of past states rather than a single frame. The architecture config is the architecture configuration. The system dynamics forecast horizon is the number of autoregressive prediction steps, which is how far ahead the model is asked to predict on its own. That last one is a budget parameter as much as a modelling one, because a longer autoregressive rollout lets a policy plan further but compounds the model's own error, and the visualization script exists precisely so you can watch that compounding happen.

## Installation replaces the RSL RL library Isaac Lab ships with

The install sequence is four steps, and the second one is the part people miss. First you install Isaac Lab following the official installation guide, and the README notes this is not needed for offline policy training. The guide recommends the Conda installation because it simplifies calling Python scripts from the terminal. Second, you install model-based RSL RL following the official installation guide of the model-based RSL RL fork, in order to replace the rsl_rl_lib that comes with Isaac Lab. Third you clone this repository outside your Isaac Lab directory, and fourth you install the extension into the Python environment where Isaac Lab is installed:

```bash
python -m pip install -e source/mbrl
```

The editable install of source/mbrl is the package name the tooling expects, since the pyproject configuration sets the first party package to mbrl and the local folder to config. The verification step, also optional for offline training, runs the standard training entry point:

```bash
python scripts/reinforcement_learning/rsl_rl/train.py --task Template-Isaac-Velocity-Flat-Anymal-D-Init-v0 --headless
```

If that task name resolves and the headless run starts, the extension is wired into Isaac Lab correctly.

## Pretraining and visualization are separate entry points

The pretraining run uses its own task name and its own script, and the distinction matters because the two phases produce different artifacts. The pretraining command trains a PPO policy from scratch while the induced experience during training is used to train the dynamics model, so one run produces both a policy and a dynamics model:

```bash
python scripts/reinforcement_learning/rsl_rl/train.py \
  --task Template-Isaac-Velocity-Flat-Anymal-D-Pretrain-v0 \
  --headless
```

Visualization is a separate script that rolls the learned model out autoregressively in imagination, conditioned on the actions from the learned policy:

```bash
python scripts/reinforcement_learning/rsl_rl/visualize.py \
  --task Template-Isaac-Velocity-Flat-Anymal-D-Visualize-v0 \
  --checkpoint <checkpoint_path> \
  --system_dynamics_load_path <dynamics_model_path>
```

The dynamics model path should point at the pretrained dynamics model checkpoint, for example a model_<iteration>.pt file, inside the saved run directory. That checkpoint is the input to everything downstream, so the two scripts have to agree on the run directory layout, and the visualization step is the honest way to find out whether the model you trained is one you want to build a policy on.

## Where this repository is the wrong tool

Three situations argue against it. The first is a project that does not already run Isaac Lab. The extension is an Isaac Lab extension, the configuration classes are Isaac Lab environment configurations, and the training entry points live under the Isaac Lab scripts directory, so adopting it means adopting the simulator stack and its Python environment as well. The second is a general purpose reinforcement learning need. The environments, the configuration classes and the runner are shaped around a legged locomotion task on the ANYmal platform, and the code exists to support two named methods rather than to be a general framework, so a different task means writing new configurations against a model-based algorithm rather than reusing a toolkit. The third is a stack that does not match the pins. The badges name IsaacSim 4.5.0, IsaacLab 2.1.0, Python 3.10, Linux 64 and Windows 64, and the pyright configuration sets the same Python version and platform. The repository is not archived and the last push was on 2026-04-08, with no GitHub releases, so there is no tagged artefact to fall back on, and the code is versioned against a simulator release that will move. If you want the ideas rather than the code, the two papers and their project pages are the durable artefacts, and the repository is the implementation detail behind them.

## Conclusion

Adopt robotic_world_model when you want to reproduce or extend the Robotic World Model and Uncertainty-Aware Robotic World Model results and you already run Isaac Lab, and skip it if your goal is a general purpose reinforcement learning toolkit, because the code is organised around one specific pair of papers. Verify first that you are on the pinned IsaacSim 4.5.0 and IsaacLab 2.1.0 with Python 3.10 on Linux 64 or Windows 64, and confirm the model-based RSL RL fork is installed in place of the rsl_rl_lib that ships with Isaac Lab, since the training entry points depend on it.

## FAQ

### What is robotic_world_model?

It is a Robotic World Model Extension for Isaac Lab that adds environments and training pipelines for Robotic World Model, Uncertainty-Aware Robotic World Model and related model-based reinforcement learning methods, published alongside the two papers on arXiv.

### How do I install the robotic_world_model extension?

Install Isaac Lab, then install the model-based RSL RL fork to replace the rsl_rl_lib that comes with Isaac Lab, clone this repository outside your Isaac Lab directory, and run python -m pip install -e source/mbrl in the environment where Isaac Lab is installed.

### What is the difference between the online and offline policy training options?

The online option continually collects additional environment interactions with the latest policy to update the dynamics model, as in RWM and MBPO-PPO. The offline option collects no further interactions and the policy has to rely on the static dynamics model, as in RWM-U and MOPO-PPO.

### Which model outputs can I configure in the pretraining environment?

Under ObservationsCfg_PRETRAIN you can choose SystemStateCfg for the state input and output head, SystemActionCfg for the action input, SystemExtensionCfg for a continuous privileged output head such as rewards, SystemContactCfg for a binary privileged output head for contacts, and SystemTerminationCfg for a binary privileged output head for terminations.

### Is robotic_world_model still maintained?

The repository is not archived and the last push was on 2026-04-08, and it publishes no GitHub releases. The badges pin IsaacSim 4.5.0, IsaacLab 2.1.0 and Python 3.10 on Linux 64 and Windows 64, and the licence file is at the top of the repository.

## Sources

- [Issues](https://github.com/leggedrobotics/robotic_world_model/issues)
- [leggedrobotics/robotic_world_model on GitHub](https://github.com/leggedrobotics/robotic_world_model)
- [License: Apache-2.0](https://github.com/leggedrobotics/robotic_world_model/blob/master/LICENSE)
- [Project website](https://sites.google.com/view/roboticworldmodel)
- [README](https://github.com/leggedrobotics/robotic_world_model/blob/master/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/leggedrobotics-robotic-world-model