UMR: retargeting human motion to humanoids through learned point cloud correspondence

The mapping problem UMR is built to remove

Retargeting human motion onto a humanoid normally starts with a hand-built mapping: this human joint drives that robot joint, with offsets and scaling chosen per skeleton pair. That mapping is brittle. Change the source skeleton or the robot and the work starts over. UMR takes a different position. It treats the moving exterior body surface as the interface, so the correspondence is learned between surfaces rather than declared between joints.

The README frames this as two stages: Point Cloud Correspondence Learning, which learns ordered source-robot surface correspondence in aligned canonical poses, and Correspondence-Guided Retargeting, which optimizes robot motion using matched surface positions, orientations, contacts, and kinematic constraints. The claim that matters for adopters is the reuse rule: a learned correspondence is reused by motions with the same source template and target robot. Learn once per source and robot pair, then retarget many sequences without redoing the mapping.

The intended audience is robotics and graphics engineers who already have motion data with surface information and a humanoid model they want to drive. It is not a general animation tool and it does not pretend to handle sources that only expose joint rotations with no surface.

How the two stages connect at runtime

The pipeline is a Python entry point driven by JSON configuration. The visible data flow is: a motion source adapter produces surface samples in an aligned canonical pose, correspondence learning produces ordered source-robot matches, and the retargeting optimizer then solves for robot motion against those matches plus contacts and kinematic constraints.

Configuration is split in two. A robot config under robot_configs/ carries the robot name, the MJCF path, and a tpose_qpos entry. A defaults file at the repository root carries per-source settings; the top level lists humanoid_retarget_defaults.json plus variants for bones_seed, batch, batch_bones_seed, hsi_hoi_grail, hsi_hoi_standard, humanoid_character, and nr. That naming is the clearest map of what the code supports: each suffix corresponds to an adapter family in sample_data/.

The adapter guides are per-directory READMEs, for example sample_data/bones-seed/README.md and sample_data/grail/README.md, and each documents the expected local layout. This is the part a new user should read first, because the pipeline expects data already arranged the way the adapter expects, not an arbitrary file dropped into sample_data/.

Installing UMR and running the LAFAN1 example

The README gives a conda-based install targeting Python 3.12 and a CUDA 12.1 PyTorch build. Create the environment, install torch from the PyTorch index, then install the project requirements.

conda create -n umr python=3.12 pip -y
conda activate umr
python -m pip install --index-url https://download.pytorch.org/whl/cu121 torch==2.4.1
python -m pip install -r requirements-umr.txt

SMPL-X body models are not distributed with the repository. The README says to download them from the official SMPL-X provider after accepting its terms, and that both .pkl and .npz are supported. Place at least the neutral model at one of these paths:

smpl/SMPLX_NEUTRAL.pkl
# or
smpl/SMPLX_NEUTRAL.npz

Add SMPLX_MALE and SMPLX_FEMALE in either format when a sequence needs those genders. The README notes the NPZ path has been tested with both neutral SMPL-X motion and female OMOMO motion.

With the model in place, the quick start retargets the included LAFAN1-derived sequence. The default configuration uses sample_data/lafan1_smplx/dance1_subject2.npz, builds or reuses the learned correspondence, runs retargeting, and opens the MuJoCo viewer.

python scripts/humanoid_retarget_pipeline.py \
  --config robot_configs/humanoid_retarget_unitree_g1_example.json

To move to your own robot, copy the example config in robot_configs/ and update its name and MJCF path. Prepare the robot T-pose in UMR Studio: load the robot asset folder, select its MJCF, adjust it into a T-pose, and click Copy T-pose Config. Paste the copied tpose_qpos into the new config and rerun the pipeline with it. The README states no manual human-robot mapping is required.

Where the surface formulation stops being enough

The clearest limitation is stated by the README itself, in the OmniContact note. An internal development version of UMR produced the Unitree G1 retargeting data released by OmniContact, but OmniContact ships source motions as BVH while UMR consumes SMPL-X. The internal BVH-to-SMPL-X converter is not included, so the current release does not directly support those BVH files. That is a concrete gap between what the method has been used for and what this repository can do.

For LAFAN1 the README points elsewhere: use lafan_to_smplx to convert BVH motion to SMPL-X before retargeting. So BVH is not categorically out, but it is out of scope for this codebase. Anyone whose pipeline is BVH-native should budget for an external conversion step and for the mismatch between the converted result and what the adapter expects.

The second constraint is licensing. SMPL-X body-model files are not distributed here and must be obtained from the official provider after accepting its terms. The GRAIL example applies its bundled G1-SMPL-X template and pose-corrective overlay to the user-provided neutral SMPL-X model at runtime, and the README says the derived baked SMPL-X weights are not distributed. So a GRAIL run depends on a model you supply under someone else's terms.

The third is a design trade-off rather than a bug. Correspondence is reused only when the source template and target robot match. A new robot means a new T-pose in UMR Studio and a new correspondence build. The per-pair cost is real even though the per-motion cost is not.

UMR against direct joint-space retargeting

The alternative most teams already have is direct joint-space retargeting, the approach behind tools like MimicKit, which the README lists as the source of the Humanoid Character sample motion. In that style, you specify which source joint maps to which robot joint, apply offsets and scale factors, and optionally run a solver to clean up foot contact and balance. It is cheap to start and easy to reason about, because every output joint traces back to a declared input joint.

The difference in approach is where the correspondence lives. Joint-space retargeting encodes it once, by hand, in a mapping table. UMR learns it from aligned canonical poses, so the same formulation covers sources whose surface is sampled rather than whose joints are named. That is why the README can list BONES-SEED/SOMA, GRAIL, OmniContact, LAFAN1/SMPL-X, OMOMO, Humanoid Character, AdaPT body+racket, and NR FBX/BVH as motion sources under one formulation.

The cost is the setup you cannot skip: a canonical T-pose for the robot, a correspondence build, and a config per robot. If your source is a single skeleton and your target is a single robot, and you never expect to change either, direct joint-space mapping is less machinery for the same job. UMR pays off when the pair count grows or when the source has no clean joint naming to map from.

Maintenance, licence, and upgrade exposure

The repository is not archived, and the last push was on 2026-09-14. There are no retrieved releases, so there is no versioned artifact to pin and no changelog to read. Installation follows the README rather than a package index, which means upgrades are a git pull plus a re-run of pip install -r requirements-umr.txt, and any change to the pipeline scripts or to the defaults JSON files lands immediately in your working tree. The defaults files are named per adapter, so an upstream rename of one of them will break the corresponding command line rather than fail silently.

The licence for the repository itself is not stated in the README, so treat redistribution as unresolved until you check the repository directly. What is clear is the third-party position: SMPL-X body models carry their own terms from the official provider, and the README is explicit that neither the models nor the derived baked SMPL-X weights ship here. The GRAIL example builds on a user-provided neutral SMPL-X model at runtime. If your product depends on redistributing retargeted output, the SMPL-X terms are the constraint to read, not this repository's.

There is also a version-coupling cost. The install pins torch==2.4.1 from the cu121 index. Moving to a newer CUDA or torch release is not documented as supported, so plan an environment per project rather than upgrading in place.

Editorial conclusion

UMR is worth adopting if you already have surface-level motion in SMPL-X or one of the adapter formats and a humanoid MJCF you can pose in UMR Studio, because the learned correspondence removes per-motion human-robot mapping work. It is the wrong tool if your source is raw BVH with no SMPL-X conversion path, since the README states the internal BVH-to-SMPL-X converter is not included, or if you cannot accept the SMPL-X licence terms, because the body models are not distributed here. Before committing, verify three things: that a neutral SMPL-X model loads from smpl/SMPLX_NEUTRAL.pkl or smpl/SMPLX_NEUTRAL.npz, that your robot T-pose pastes cleanly into tpose_qpos in a copied config under robot_configs/, and that the pipeline reaches the MuJoCo viewer on the included sample_data/lafan1_smplx/dance1_subject2.npz sequence.

Frequently asked questions

Official sources

1. hanyang9/UMR on GitHub
2. Issues
3. README