Microduck RL trains an 800 g biped against a voltage level servo model
RL training environments for Microduck (mjlab)
At a glance
- What is it?
- pollen-robotics/microduck_rl holds the mjlab environments that train policies for Pollen Robotics' 800 g biped, and it models every Dynamixel XL330 servo down to its voltage control law. The runtime that executes the exported ONNX files lives in a separate repository.
- Who is it for?
- Adopt microduck_rl if you own a Microduck or plan to train for one, because the voltage level M6 actuator and the per joint backlash hinges are sim2real decisions you would otherwise rebuild from scratch. It is the wrong tool for a general locomotion stack, since every environment is bound to one robot and one set of MJCF models.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 11 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on October 23, 2026, and from our analysis. They are not legal advice.
Editorial analysis
What the training repository leaves to the runtime
pollen-robotics/microduck_rl is the training half of a two repository setup, and that split matters before anything runs. Environment registration, PPO training and ONNX export all happen inside the Python package `mjlab-microduck`, which plugs itself into `mjlab` through an entry point. The runtime in pollen-robotics/microduck executes the exported files on hardware, so nothing in this repository drives a servo.
The plant is a biped of about 800 g and 25 cm tall, policies train at 50 Hz, and every environment shares a 61-dimensional observation contract. That shared contract is what lets the runtime hot swap a walking policy for a recovery or trick policy at any moment instead of running one merged controller, and `scripts/infer_policy.py` rehearses the same swap on a laptop.
Coverage runs past locomotion. The registry holds walking with velocity and head pose commands, one policy that walks and recovers from a fall, standing up from face down, face up or sitting, a commanded sit and stand, crouching to touch the ground with the mouth tip, kicking a 70 mm / 15 g ball, a forward roll over the head, and a family of roller tasks built around passive wheels under the feet.
The M6 voltage law replaces the XML PD controller
Every task drives the joints with the BAM M6 model for the Dynamixel XL330 rather than an ideal PD controller. The model keeps the voltage control law, back EMF, and Coulomb, Stribeck and load dependent friction, and the implementation sits in `FrictionDRBamActuator` under `src/mjlab_microduck/actuator/`. Simulation wires it in through `bam.mujoco.MujocoController`, so the same component that randomises battery conditions also decides how a servo resists a command.
Four quantities vary per environment: battery voltage, voltage sag under load, command delay, and friction magnitude. At 4096 environments a single run spans that spread instead of averaging it away, which is the reason for randomising per environment at this scale.
Three flags pull the spread back to one value for debugging. `--vin` pins battery voltage, `--vin-drop-gain` pins sag under load, and `--kp-fw` pins the forward gain. A fourth, `--no-bam`, removes the model altogether and falls back to the PD actuators declared in the MJCF, which is how you tell a reward problem apart from an actuator problem.
Fourteen servos each carry a passive backlash hinge
Every main task has a twin that trains against gear play. Insert `-Backlash` before `MicroDuck` in a task id and you get, for example, `Mjlab-Velocity-Flat-Backlash-MicroDuck`. Each of the 14 servo joints then gets an extra unactuated hinge named `passive_<joint>_backlash` holding plus or minus 1 degree of play, 2 degrees across the full range.
The part that carries weight is where the encoder sits. On the real robot the encoder reads the output side of the play, so `BacklashEncoderBamActuator` and the `joint_pos` and `joint_vel` observations read through the hinge as `qpos[servo] + qpos[backlash]`. Firmware PD emulation and observations therefore follow the lag the hardware actually has, rather than a joint limit standing in for it.
Observation and action dimensions do not change, so exported ONNX files and the runtime need no changes. A backlash twin exports through exactly the same path as its base task, and the construction is in `src/mjlab_microduck/tasks/backlash.py`.
uv sync pulls about 2 GB of CUDA wheels on ARM boxes
The local path needs a CUDA GPU, because training runs through MuJoCo Warp, and uv handles the environment. On ARM machines the first sync is the trap. `uv sync` pulls about 2 GB of CUDA wheels, uv's default 30 s HTTP timeout can abort that download partway through, and the documented fix is to export `UV_HTTP_TIMEOUT=600` before the first sync.
git clone https://github.com/pollen-robotics/microduck_rl
cd microduck_rlA first training run follows, which the README puts at roughly 1 to 2 hours for a usable gait at 4096 environments:
uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096The same README block chains `uv run play` to watch the trained policy in the viewer, `uv run scripts/export.py` to write the ONNX file, and `uv run scripts/infer_policy.py` to drive it from the keyboard in CPU MuJoCo. The first two take a `--wandb-run-path` argument. Resuming later repeats the train command with checkpoint flags:
uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096 \
--agent.run-name resume --agent.load-checkpoint model_29999.pt --agent.resume TrueThe registry lists tasks, the README table only samples them
`uv run list-envs` prints the live registry, and that registry, not the README table, is the list of what can actually run. `pyproject.toml` registers `mjlab_microduck = "mjlab_microduck.tasks"` under the `mjlab.tasks` entry point, so the environments reach the training CLI because the installed package declares them.
The naming pattern encodes the terrain choice. Flat and Rough variants exist for Velocity, VelStand, StandUp, SitStand and GroundPick, while BallKick and Roulade are flat only and the roller family adds a slope variant. A Rough variant is a separate task with its own terrain sampling and its own policy, not a cosmetic suffix.
Keyboard control in `scripts/infer_policy.py` covers the same range from the other side. Velocity commands take the usual keys, G triggers a ground pick, Y a sit and stand, R a roll, and K or L the two kicks. `--debug`, `--save-csv` and `--record` exist for sim2real comparisons, and handing the script four exported files in one launch rehearses the runtime hot swap:
uv run scripts/infer_policy.py --walking walk.onnx --standing stand.onnx \
--sitstand sitstand.onnx --roulade roulade.onnx --new-cmd-obsA full collision model ships, and no task loads it
MJCF models sit in `src/mjlab_microduck/robot/microduck/`, one `config_mjcf_*.json` per model, exported from Onshape with `onshape-to-robot`. Four XML files ship, and their names mislead in both directions.
`robot_walk.xml` strips trunk and head contacts, on the stated reasoning that falling is cheap for the main walking task. `robot_groundcontact.xml` keeps a curated collision set for the parts that touch the floor and lets the body lie on the ground, and it serves VelStand, StandUp, SitStand, GroundPick, BallKick and Roulade. That file used to be called `robot_allcollisions.xml`, and the name now belongs to a different file, the true full collision model where every part has a collision geom.
No task loads that full collision model, and the repository does not record why. The environments in the registry run on `robot_walk.xml`, `robot_groundcontact.xml` or `robot_groundcontact_rollers.xml`, so contact coverage in every trained policy is bounded by the curated set. Anyone reusing these models for another robot should pick the collision set first and treat the file name as noise.
Running PPO without a local GPU or without bam
Two alternatives live inside the project rather than beside it. For hardware, `--hf-jobs` on any train command runs the job on Hugging Face Jobs instead of the local machine, which is the documented answer for people without a CUDA GPU, with details in `scripts/hf/README.md`. The path that stays local on any machine is inference after export, since `scripts/infer_policy.py` runs the ONNX policy in CPU MuJoCo with the keyboard.
For physics, `--no-bam` drops the M6 voltage model and falls back to the PD actuators declared in the MJCF. Task ids, observations and the ONNX path stay the same, so nothing downstream breaks, but the actuator fidelity the project treats as most of the sim2real gap is simply gone. Treat it as a diagnostic setting, not a way to train for hardware.
`uv run publish` is a third route off the machine. It takes `--onnx`, `--repo`, `--kind` and `--duration-s`, and the documented example shares a policy as an episodic entry lasting 4 seconds. The README does not document how a published policy gets rolled back.
torch is pinned to 2.9.1 because the CUDA index moves faster
The dependency block in `pyproject.toml` is mostly pins that exist to stop the environment drifting rather than to fix a version problem. `requires-python` is bounded by `>=3.12, <3.13`, because BAM pins the same range, and a floating upper bound once let Hugging Face Jobs resolve 3.13.14 on 2026-07-21.
`scipy>=1.16` is a direct dependency only because `mjlab==1.3.0` imports scipy from its terrain module without declaring it; without that line a fresh `uv sync` cannot even import `mjlab_microduck`. `torch==2.9.1` is exact rather than a floor, and the comment gives the reason: the entry binds torch to the CUDA index on aarch64, where that index carries builds newer than the PyPI pin, and a floating `>=` would move 2.9.1 to 2.13.0 for mjlab and rsl_rl in one step.
Two naming traps sit in the same list. The BAM distribution is published as `better-actuator-models` while the import stays bam, so a grep for the short name in package metadata finds nothing. A script name also collides with the `train` command of mjlab on purpose, which is why the quickstart calls training through `uv run train`. The project ships under `Apache-2.0` with its licence text stored as a file in the repository.
Editorial conclusion
Adopt microduck_rl if you own a Microduck or plan to train for one, because the voltage level M6 actuator and the per joint backlash hinges are sim2real decisions you would otherwise rebuild from scratch. It is the wrong tool for a general locomotion stack, since every environment is bound to one robot and one set of MJCF models. Before trusting a policy, check what it trained against: robot_walk.xml carries no trunk contacts, robot_allcollisions.xml is loaded by no task, and only Python 3.12 resolves in pyproject.toml.
Frequently asked questions
How do I install and train a Microduck RL policy?
Clone the repository and drive it through uv, since training needs a CUDA GPU for MuJoCo Warp. On ARM boxes such as DGX Spark / GB10 or Jetson, export `UV_HTTP_TIMEOUT=600` before the first `uv sync`, because that sync pulls about 2 GB of CUDA wheels and uv's default 30 s timeout can abort it.
Which Python versions does microduck_rl support?
Only 3.12. The `requires-python` bound is `>=3.12, <3.13` because BAM pins the same range, and a floating upper bound once let Hugging Face Jobs resolve 3.13.14 on 2026-07-21.
What does a Backlash task change in microduck_rl?
It puts plus or minus 1 degree of gear play, 2 degrees in total, into each of the 14 servo joints through an unactuated `passive_<joint>_backlash` hinge. Encoder and observations read through that hinge, while observation and action dimensions stay unchanged, so ONNX export and the runtime need no changes.
Do I need a GPU to use microduck_rl?
Training does, because it runs through MuJoCo Warp, and `--hf-jobs` sends any train command to Hugging Face Jobs instead of the local machine. Inference after export runs in CPU MuJoCo, where `scripts/infer_policy.py` drives the ONNX policy from the keyboard.
What does --no-bam change in microduck_rl?
It removes the BAM M6 model for the Dynamixel XL330 and falls back to the PD actuators declared in the MJCF. Three other flags narrow the randomisation rather than removing it, pinning battery voltage with `--vin`, sag with `--vin-drop-gain`, and the forward gain with `--kp-fw`.
Which MJCF model does the Microduck walking policy use?
`robot_walk.xml`, which strips trunk and head contacts. Ground tasks use `robot_groundcontact.xml` and the roller family uses `robot_groundcontact_rollers.xml`, while `robot_allcollisions.xml`, the true full collision model, is loaded by no task.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/pollen-robotics-microduck-rl)Was this useful?