rl_sar for Simulated and Real-World Robot Policies

rl_sar's Simulation-to-Hardware Scope

rl_sar addresses a specific handoff problem: a reinforcement learning policy has been trained elsewhere, but an engineer needs executable components for simulation and physical deployment. The name combines “simulation and real,” matching the repository's purpose of supporting movement from virtual robot environments toward hardware operation. It is intended for engineers working with quadruped, wheeled, or humanoid robots rather than users looking for a general reinforcement learning tutorial.

The repository is primarily C++. Its support table includes locomotion and dance policies for robots such as Unitree A1, Go2, Go2W, B2, B2W, G1, GR1T1, GR1T2, L4W4, Lite3, D1, and Tita. Coverage is configuration-specific rather than universal. Go2 has documented Gazebo, MuJoCo, and real-world support, while B2 and B2W have real-world cells marked with a warning symbol rather than a check. Tita is marked for Gazebo but not MuJoCo or physical deployment. The GR1T1 and GR1T2 entries are restricted to Ubuntu 20.04.

These distinctions matter because a robot directory does not establish a complete simulation-to-hardware path. The policy source also differs. Listed training systems include `legged_gym`, `himloco`, `robot_lab`, `robomimic`, and `whole_body_tracking`. A user cannot assume that every policy works with every runtime supported by the repository.

The framework is presented as an execution and deployment layer, not a general policy trainer. The README specifically directs IsaacLab users who want to train policies with IsaacSim to the separate robot_lab project. rl_sar is therefore more relevant to engineers who already possess a compatible trained model and robot configuration than to researchers looking for one package that covers training, simulation, and hardware.

Policy Files, ROS Packages, and Build Layout

The visible repository places policy resources at `policy/` and C++ implementation under `src/`. Before execution, the README instructs users to copy a trained `pt` model into `rl_sar/src/rl_sar/policy/<ROBOT>/<CONFIG>`. Robot and policy variants are then represented by combinations such as `go2/himloco` and `go2w/robot_lab`. Configuration is divided between `<ROBOT>/<CONFIG>/config.yaml` and `<ROBOT>/base.yaml`.

That organization separates a model artifact from the runtime settings and robot package around it. The project supports `libtorch` and `onnxruntime`, so the selected inference runtime must match the available policy artifact and build. The README excerpt does not describe policy network architecture, observation construction, action scaling, or the inference loop in enough detail to support more specific claims.

ROS Noetic and ROS2 Foxy or Humble provide package-based builds. Gazebo launch files accept a robot name through `rname`, while CMake-only mode supports deployments where simulation is unnecessary. In that mode, the README places executables in `cmake_build/bin` and libraries in `cmake_build/lib`. MuJoCo support is also enabled through a CMake option rather than the normal ROS workspace build.

Other top-level entries include `scripts/` and `docker/`. Based on the repository layout, they separate automation and container resources from the C++ package, but the README excerpt does not explain how to use them. That silence should not be interpreted as either documented support or documented incompatibility.

Installing the Ubuntu Build Dependencies

The documented Ubuntu route covers Ubuntu 20.04 and 22.04. Clone the project recursively because the repository includes a `.gitmodules` file and the published command initializes nested dependencies:

git clone --recursive --depth 1 https://github.com/fan-ziqi/rl_sar.git

This creates a shallow working tree. It may reduce local history but does not omit submodule initialization. The project then requires native compilation and communication dependencies:

sudo apt install cmake g++ build-essential libyaml-cpp-dev libeigen3-dev libboost-all-dev libspdlog-dev libfmt-dev libtbb-dev liblcm-dev

For ROS2 on Ubuntu 20.04 or 22.04, the package names are tied to the active distribution. The README uses `$ROS_DISTRO` rather than prescribing one distribution globally:

sudo apt install ros-$ROS_DISTRO-teleop-twist-keyboard ros-$ROS_DISTRO-ros2-control ros-$ROS_DISTRO-ros2-controllers ros-$ROS_DISTRO-control-toolbox ros-$ROS_DISTRO-robot-state-publisher ros-$ROS_DISTRO-joint-state-publisher-gui ros-$ROS_DISTRO-gazebo-ros2-control ros-$ROS_DISTRO-gazebo-ros-pkgs ros-$ROS_DISTRO-xacro

This set covers control, state publication, Gazebo integration, and teleoperation components used by the documented path. ROS Noetic has a separate package list containing `ros-noetic-*` controller and Gazebo packages. The chosen ROS distribution, robot configuration, and build mode must agree before compilation.

The README also names IsaacGym, IsaacSim, MuJoCo 3.2.7, `libtorch`, and `onnxruntime` as supported components, but the setup excerpt does not give complete installation commands for all of them. Users must obtain those dependencies from their respective projects rather than assuming the Ubuntu package command installs the entire stack.

Compiling ROS, Hardware, and MuJoCo Targets

The standard build starts at the repository root and compiles the ROS workspace:

./build.sh

The script also accepts package names for a narrower compilation:

./build.sh package1 package2

Those names are examples rather than documented robot-specific package identifiers. The README does not map every `<ROBOT>/<CONFIG>` pair to its package names, so users should replace the placeholders with names listed by the selected package or build configuration.

For hardware-only deployment, the project provides a CMake path that disables ROS:

./build.sh -m

The README says this mode places executables in `cmake_build/bin` and libraries in `cmake_build/lib`. It is the clearest documented separation between the full simulation workspace and a hardware-oriented build. It does not by itself validate motor limits, sensor calibration, emergency handling, or robot-specific control settings.

MuJoCo support uses another CMake option:

./build.sh -mj

This flag builds with MuJoCo support under CMake. It does not expand robot compatibility. The README labels MuJoCo support as partial, and several rows in the support table lack MuJoCo support. macOS is also limited to MuJoCo simulation and marked experimental.

If `catkin build` reports that neither `empy` nor the `em` Python module can be found, the README prescribes selecting the system Python interpreter:

catkin config -DPYTHON_EXECUTABLE=/usr/bin/python3

A separate README notice says Python support is temporarily suspended and directs users who need it to release `v2.3`. That restriction should be resolved before selecting a release for a Python-dependent workflow.

First Gazebo Launch for a Compatible Robot

A first use should select a robot and policy combination with documented Gazebo support. Before running it, copy the trained `pt` file into `rl_sar/src/rl_sar/policy/<ROBOT>/<CONFIG>` and review the matching `config.yaml` and `base.yaml` files. The README does not document automatic placement, model conversion, or validation of a mismatched policy.

For a ROS2 build, source the generated workspace and launch the Gazebo environment with the robot name passed through `rname`:

source install/setup.bash
ros2 launch rl_sar gazebo.launch.py rname:=<ROBOT>

For ROS Noetic, source the development workspace and use `roslaunch`:

source devel/setup.bash
roslaunch rl_sar gazebo.launch rname:=<ROBOT>

The expected result is the Gazebo environment for the selected robot. The README then instructs users to open another terminal and start the control program. Its complete ROS2 executable name is truncated after `ros2 run rl_`, so it cannot be reproduced without guessing. For ROS1, the complete documented control command is:

rosrun rl_sar rl_sim

This two-terminal arrangement reflects a separate simulation environment process and a separate policy-control process. Before starting either process, the model path, robot name, joint configuration, and inference runtime need to match. The README does not document watchdog behavior, invalid-action handling, or a rollback procedure for an unstable simulation policy.

Uneven Robot Coverage and Real Deployment Limits

The principal limitation is uneven support across the compatibility matrix. MuJoCo is described as partially supported, and several robots lack either MuJoCo or physical deployment entries. Some real-robot cells use a warning symbol rather than a confirmation. A supported robot may also require a particular policy source, such as `himloco` for Go2 or `robot_lab` for Go2W and D1.

The repository spans several generations of tooling. The README names ROS Noetic, ROS2 Foxy, ROS2 Humble, Gazebo Classic, MuJoCo 3.2.7, IsaacGym, and IsaacSim. It does not provide a complete matrix mapping every robot, policy source, ROS distribution, simulator, and inference runtime. Dependency compatibility therefore requires checking individual configurations rather than relying on the project name or robot list alone.

Physical deployment adds requirements that the README does not fully specify. It does not document an emergency-stop procedure, a staged hardware checklist, actuator limit configuration, calibration, watchdog behavior, or recovery after a bad action. The repository disclaimer says users must implement proper safety measures and bear the resulting risks. The absence of details in the README does not prove those features are absent from the source, but it prevents a safety claim based on the published setup instructions.

rl_sar is the wrong choice for a team that needs uniform hardware support across many robot models. It is also unsuitable for a user expecting a maintained Python training workflow. Its strongest fit is a project that already has a trained, supported policy and needs the documented deployment or simulation plumbing.

robot_lab, Standalone MuJoCo, and the Full ROS Build

The README names robot_lab as the alternative for policy training with IsaacLab and IsaacSim. The difference is architectural. robot_lab is designated for training, while rl_sar is presented as the framework for simulation verification and physical deployment of resulting algorithms. A team may therefore use both projects when it needs training followed by deployment.

Compatibility is not automatic. The README states that the order of joints in the robot_lab `joint_names` configuration must match `xxx/robot_lab/config.yaml` in rl_sar. This ordering boundary is important because a model trained with one joint sequence may not be valid when runtime configuration uses another sequence. The README does not provide a conversion or validation command for this mismatch.

For development that does not need physical hardware, MuJoCo can provide a narrower CMake route. However, `./build.sh -mj` only adds support for combinations marked in the support table. It is not a general MuJoCo replacement for every robot listed by the project.

The full ROS build is more appropriate when Gazebo and ROS control integration are required. It adds ROS distribution packages, controller components, and workspace setup steps. The README does not claim that this larger build improves policy quality or runtime performance.

The practical choice depends on workflow. Use robot_lab for the documented IsaacLab training path, use rl_sar to consume compatible trained models, and select ROS or MuJoCo according to the exact robot support row. No performance comparison between these paths is published in the README.

Release History, Updates, and License Cost

The repository is not archived. Its last push was on 2026-08-16, and it includes releases v3.2.0 on 2025-09-19, v4.0.0 on 2025-10-29, and v4.1.0 on 2026-03-19. These dates confirm recent repository activity and versioned releases, but they do not resolve the README's separate warning that Python support is temporarily suspended.

Updating an existing checkout uses two documented commands:

git pull
git submodule update --init --recursive --recommend-shallow --progress

The first updates the repository, and the second synchronizes nested dependencies. Upgrading can still require manual compatibility work because policy directories, robot configurations, ROS packages, and inference artifacts are connected. A user moving between releases should compare the selected robot's configuration and verify that the trained model still matches it.

The project uses the Apache-2.0 license and includes a `LICENSE` file. Organizations should account for that license when incorporating the code, while checking the separate terms of dependencies such as ROS, MuJoCo, `libtorch`, and `onnxruntime`. This identifies licensing work to review without providing legal advice.

The most expensive upgrades are likely to occur where a project changes ROS distribution, robot model, policy source, or simulator. The README gives a repository update procedure but no automatic migration procedure for those combinations. Version selection should therefore follow the selected robot's supported configuration rather than simply choosing the newest tag.

Editorial conclusion

rl_sar suits engineers who already have a compatible trained policy and need a documented path into Gazebo, MuJoCo, or supported hardware. It is less suitable for users seeking an active Python training stack, unrestricted robot coverage, or uniform real-world support. Before adoption, verify the support table for the exact robot and policy, confirm the required ROS and inference dependencies, then test the selected policy first in simulation. For IsaacLab training, the README directs users to robot_lab, so verify that project's output is compatible before committing to rl_sar for deployment.

Frequently asked questions

Official sources

1. fan-ziqi/rl_sar on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases