# unitree_rl_mjlab and the train-play-deploy loop for Unitree robots

> unitree_rl_mjlab is a reinforcement learning implementation for Unitree robots built on mjlab and MuJoCo, supporting the Go2, A2, As2, G1, R1, H1_2 and H2. The value is that it closes the loop from a policy trained in simulation to an ONNX policy deployed on a physical robot, and the cost is that almost everything about the environment is exposed as a command-line parameter.

**unitreerobotics/unitree_rl_mjlab** — This is a repository for reinforcement learning implementation for Unitree robots, based on Mujoco.

- Repository: https://github.com/unitreerobotics/unitree_rl_mjlab
- Stars: 701 · Forks: 227
- Language: C++
- License: Apache-2.0
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/unitreerobotics-unitree-rl-mjlab

## mjlab's API with MuJoCo underneath

Unitree RL Mjlab is a reinforcement learning project built upon mjlab, using MuJoCo as its physics simulation backend, and it currently supports the Unitree Go2, A2, As2, G1, R1, H1_2 and H2.

The lineage matters for anyone who has used Isaac Lab, because mjlab is described as combining Isaac Lab's proven API with MuJoCo physics to provide lightweight, modular abstractions for RL robotics research and sim-to-real deployment. If your team's existing policies, environment classes and training scripts are written against the Isaac Lab conventions, mjlab is a familiar shape, and the swap underneath is a physics engine change rather than a rewrite.

Lightweight is the substantive claim. Isaac Lab's own selling point is GPU-accelerated parallel simulation, and MuJoCo is a different tradeoff, with a smaller footprint and a different set of performance characteristics. The README links the mjlab documentation for detail and the MuJoCo Warp repository as the physics implementation.

The primary language listed for this repository is C++, which fits a project that ships a compiled deployment binary for the robot alongside the Python training code.

## The three stages and what each one produces

The workflow is stated as a single line: Train, then Play, then Sim2Real.

Train is where the agent interacts with the MuJoCo simulation and optimises policies through reward maximisation. Play is where trained policies are replayed to verify expected behaviour. Sim2Real is where trained policies are deployed to physical Unitree robots for real-world execution.

That third stage is what makes this a robotics repository rather than a research codebase, and it has a concrete artefact. During training, policy.onnx and policy.onnx.data are also exported for deployment onto physical robots. So the training run produces not just a checkpoint but a deployable pair of files, and the deployment path consumes exactly those.

The structure of the repository reflects the split. There is a src/ directory with the task implementations, scripts/ with the train, play and data conversion entry points, deploy/ with the robot-side code, simulate/ alongside it, and doc/ with the setup guide and supporting documentation. There is also a README_zh.md, so the project is documented in two languages.

The pipeline is not just conceptual. The training command, the play command and the deployment build are all concrete scripts in this repository, and the intermediate artefact format is fixed, which means the loop from a reward function to a running robot is closed rather than assembled from parts.

## Velocity tracking and the parameters that shape it

The first training task is velocity tracking, and the command is short:

```bash
python scripts/train.py Unitree-G1-Flat --env.scene.num-envs=4096
```

The first positional argument names the task. The available velocity tracking tasks are Unitree-Go2-Flat, Unitree-G1-Flat, Unitree-G1-23Dof-Flat, Unitree-H1_2-Flat, Unitree-A2-Flat and Unitree-R1-Flat. The Flat suffix refers to flat-ground scenes, and the 23Dof variant is the 23 degree of freedom G1 configuration.

Scaling to multiple GPUs is a flag rather than a different script:

```bash
python scripts/train.py Unitree-G1-Flat \
  --gpu-ids 0 1 \
  --env.scene.num-envs=4096
```

The parameter list is where the framework earns its keep, because the environment is fully configurable from the command line. The env.scene group covers simulation scene configuration such as num_envs, dt, ground type, gravity and disturbances. env.observations covers the observation space, for example joint state, IMU and commands. env.rewards lists the reward terms used for policy optimisation. env.commands covers task commands such as velocity, pose or motion targets, and env.terminations the termination conditions for each episode. On the agent side there is agent.seed for reproducibility, agent.resume to continue from the last saved checkpoint, agent.policy for the network architecture, and agent.algorithm for the reinforcement learning algorithm configuration with its hyperparameters.

Checkpoints are written to a predictable path, logs/rsl_rl/<robot>_(velocity | tracking)/<date_time>/model_<iteration>.pt, so a training run and its artefact are easy to line up later.

## Motion imitation needs a converted file first

The second task type is motion imitation, where a Unitree G1 is trained to mimic reference motion sequences, and it has an extra preparation step that the velocity task does not.

Reference motions are CSV files, and they have to be converted to npz format before training. The conversion script takes the input file, an output name, an input and output frame rate, and the robot variant:

```bash
python scripts/csv_to_npz.py \
  --input-file src/assets/motions/g1/dance1_subject2.csv \
  --output-name dance1_subject2.npz \
  --input-fps 30 \
  --output-fps 50 \
  --robot g1 # g1 or g1_23dof
```

The frame rate arguments are worth noticing. The example goes from 30 to 50, so the conversion resamples the motion, and the output frame rate is the one the policy will be trained against. Getting this wrong means the policy learns a timing that does not match the reference.

The npz files land in src/motions/g1/..., and the training command then points at the converted file:

```bash
python scripts/train.py Unitree-G1-Tracking-No-State-Estimation --motion_file=src/assets/motions/g1/dance1_subject2.npz --env.scene.num-envs=4096
```

Two tracking tasks are available, Unitree-G1-Tracking-No-State-Estimation and Unitree-G1-23Dof-Tracking-No-State-Estimation. The name is the point: the policy does not receive an estimate of the robot's state, so it has to work from what it can observe. The README points to the BeyondMimic documentation, in the whole_body_tracking repository, for the detailed motion preprocessing and registry setup.

## Replaying a policy and exporting it for the robot

Visualisation is the play script, and it takes the same task name plus a checkpoint path:

```bash
python scripts/play.py Unitree-G1-Flat --checkpoint_file=logs/rsl_rl/g1_velocity/2026-xx-xx_xx-xx-xx/model_xx.pt
```

The motion imitation version adds the motion file, since the playback has to follow the reference:

```bash
python scripts/play.py Unitree-G1-Tracking-No-State-Estimation --motion_file=src/assets/motions/g1/dance1_subject2.npz --checkpoint_file=logs/rsl_rl/g1_tracking/2026-xx-xx_xx-xx-xx/model_xx.pt
```

The path shape tells you how the logging directory is organised, with a robot and mode component and a timestamp, so a run is unambiguous.

The deployment side of the loop is where the repository becomes more than a training framework, and the steps are laid out step by step. Before anything, you install the communication tools, cyclonedds and unitree_sdk2. The robot is started in a suspended state and you wait until it enters zero-torque mode, then press L2 and R2 on the controller to enter debug mode with joint damping enabled. The PC connects to the robot over Ethernet with the address 192.168.123.222 and netmask 255.255.255.0, and you use ifconfig to find the Ethernet device name.

Then the compiled part. The exported policy files go into deploy/robots/g1/config/policy/velocity/v0/exported, and the build is a plain CMake invocation:

```bash
cd deploy/robots/g1
mkdir build && cd build
cmake .. && make
```

## The package pins its dependencies exactly

The setup.py in the repository root is short and says something important about how the project expects to be used. The minimum dependencies required prior to installation are pinned:

```python
INSTALL_REQUIRES = [
    "mjlab==1.2.0",
    "mujoco-warp==3.5.0",
]
```

The package itself is named unitree_rl_mjlab at version 0.0.1, with packages set to src and install_requires pointing at that list. The comments in the file mark it as the installation script for the package, and the file is described as an installation script for the unitree_rl_mjlab python package.

Exact pins rather than version ranges is a deliberate choice for a framework that is coupled to a specific mjlab and a specific physics build, and it does mean you cannot move either dependency without expecting friction. The MuJoCo Warp version in particular tracks a fast-moving GPU physics implementation, and the training and deployment binaries have to agree about it.

The install documentation is delegated. The README's installation section says only to refer to setup.md at doc/setup_en.md for installation and configuration steps, and the top-level entries confirm that doc/ is where the documentation lives. So the practical install sequence, including the GPU and simulator prerequisites, is in that document rather than in the README, and the version pins above are the part of the requirement set the README does surface.

## Where the limits are

Three constraints are worth stating plainly, and the README is honest about the third.

The first is compute. The default example trains with 4096 parallel environments, which is a GPU-scaled configuration for a mass-parallel simulator. The whole point of a lightweight MuJoCo backend is that it is cheaper than Isaac Lab, not that it is free, and the README includes a multi-GPU flag with a list of GPU ids. A team without a discrete GPU is not the audience.

The second is scope. The supported robot list is Unitree-specific, and the tasks are locomotion-style velocity tracking and whole-body motion imitation. There is no perception stack, no manipulation, and no simulator abstraction that would let you swap MuJoCo for something else. That is a coherent scope for a vendor-backed robotics stack, and it is a narrow one for a general RL framework.

The third is the human factor in deployment, and the README is unusually explicit about it. The robot has to be started suspended and left until it enters zero-torque mode, the controller has to be held in a specific combination to enable debug mode with joint damping, and the machine has to be on a hardcoded Ethernet address. A person is standing next to a powered robot during the process. The documentation also recommends a simulation deployment before touching the real robot, and that recommendation should be taken literally rather than treated as a formality.

## Conclusion

unitree_rl_mjlab is a good fit for a robotics team that already owns Unitree hardware and wants to train a locomotion or tracking policy without building a simulation stack from scratch, because it packages the train, play and sim-to-real stages into one repository with per-robot task names and a deployment directory. It is a poor fit for a team whose budget excludes GPU hardware, since the default example asks for 4096 parallel environments, and a poor fit if you need a simulator other than MuJoCo. Before adopting, read doc/setup_en.md, since the README delegates all installation detail to it, and pin the mjlab and mujoco-warp versions the setup script requires, since the package declares mjlab==1.2.0 and mujoco-warp==3.5.0 exactly rather than as ranges.

## FAQ

### How do I train a velocity tracking policy with unitree_rl_mjlab?

Run python scripts/train.py with a task name and the environment count, for example python scripts/train.py Unitree-G1-Flat --env.scene.num-envs=4096. Available tasks are Unitree-Go2-Flat, Unitree-G1-Flat, Unitree-G1-23Dof-Flat, Unitree-H1_2-Flat, Unitree-A2-Flat and Unitree-R1-Flat.

### What do I need to prepare before motion imitation training?

Reference motion CSV files have to be converted to npz format with python scripts/csv_to_npz.py, giving an input file, output name, input and output frame rates, and the robot variant. The converted files are stored at src/motions/g1/ and are then passed to the training script with --motion_file.

### How is a trained policy deployed to a Unitree robot?

During training policy.onnx and policy.onnx.data are exported for deployment. You place them into deploy/robots/g1/config/policy/velocity/v0/exported, build with cmake .. && make in a build directory under deploy/robots/g1, and connect the PC to the robot over Ethernet at 192.168.123.222 with netmask 255.255.255.0 after enabling debug mode.

### Which simulator and framework versions does unitree_rl_mjlab require?

The setup script pins mjlab==1.2.0 and mujoco-warp==3.5.0 as the minimum dependencies, and the project is built on mjlab using MuJoCo as the physics backend. The README directs readers to doc/setup_en.md for the full installation and configuration steps.

## Sources

- [Issues](https://github.com/unitreerobotics/unitree_rl_mjlab/issues)
- [License: Apache-2.0](https://github.com/unitreerobotics/unitree_rl_mjlab/blob/main/LICENSE)
- [README](https://github.com/unitreerobotics/unitree_rl_mjlab/blob/main/README.md)
- [unitreerobotics/unitree_rl_mjlab on GitHub](https://github.com/unitreerobotics/unitree_rl_mjlab)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/unitreerobotics-unitree-rl-mjlab