# InterMimic: one policy for whole-body human-object interaction, on humanoids > InterMimic is a research codebase from a CVPR 2025 highlight paper that trains a single policy for full-body human-object interaction, working out of the box for both SMPL-X human bodies and the Unitree G1 robot. Its practical cost is a heavy simulation stack split across two NVIDIA environments. **Sirui-Xu/InterMimic** — [CVPR 2025 Highlight] InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions - Repository: https://github.com/Sirui-Xu/InterMimic - Website: https://sirui-xu.github.io/InterMimic/ - Stars: 546 · Forks: 34 - Language: Python - License: MIT - Published: 2026-10-22 · Updated: 2026-10-22 · Language: en - Canonical page: https://hysenlabs.com/projects/sirui-xu-intermimic ## One policy, many interactions InterMimic is a unified framework for whole-body control in physics-based human-object interactions, and the Overview section states the claim compactly. It features one unified policy, spanning diverse full-body interactions with dynamic, heterogeneous objects, and it works out of the box for both SMPL-X and Unitree G1 humanoids. The single-policy claim is the research contribution, and the way it is phrased gives you the shape of the method. One policy rather than a per-object or per-task policy, and the objects are heterogeneous and dynamic, which is the hard case, because a policy that works for a chair and a box and a rolling ball has to generalise across contact patterns rather than memorise one. The project is the work of Sirui Xu, Hung Yu Ling, Yu-Xiong Wang and Liang-Yan Gui, with the University of Illinois Urbana-Champaign and Electronic Arts named in the header, and the paper is a CVPR 2025 Highlight, arXiv 2502.20390, with a project page, a YouTube video and a Bilibili mirror linked alongside the code. The repository layout matches a two-simulator research project. There is an InterAct/ directory, an isaacgym/ directory and an isaaclab/ directory, plus assets/, an environment.yml and a requirement.txt. The primary language is Python, the licence is MIT, the default branch is main, and the last push was on 2026-04-21. The repository publishes no GitHub releases, so the code and the checkpoints in the tree are the distribution. ## Two simulator environments, and the friction that comes with them The Getting Started section is the most useful part of the README, and it is long, because the setup is the hard part. There are two supported environments and they are set up separately. The Isaac Gym environment starts with a dedicated conda environment on Python 3.8, with PyTorch and the repo dependencies installed: ```bash conda create -n intermimic-gym python=3.8 conda activate intermimic-gym conda install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia pip install -r requirement.txt ``` The alternative is to start from environment.yml, though the README notes that it includes some optional extras. Then Isaac Gym itself is installed following the NVIDIA instructions. Then comes the fix for the shared library lookup, which is the kind of detail that only appears in documentation written by someone who hit it: ```bash export LD_LIBRARY_PATH="$CONDA_PREFIX/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" ``` You do this after every conda activate intermimic-gym before launching Gym scripts, and the reason is that it ensures libpython3.8.so is discoverable. The Isaac Gym preview binaries link against a specific Python version, and the conda environment does not expose it in the default library path. The Isaac Lab environment is separate again, installed by the official guide, and the README insists on keeping that environment isolated, typically through Isaac Sim's own python or the provided uv or conda environment. The recommended versions are Isaac Sim 5.1.0 with IsaacLab v2.3.1. You export ISAACLAB_PATH once per shell session so the helper scripts, which source $ISAACLAB_PATH/isaaclab.sh, can find the install. There is also an optional step for the record-video flag in the replay script, which needs imageio, and imageio-ffmpeg for MP4 support, installed inside the Isaac Lab python environment. ## The data, and a caveat about reproducing the paper's numbers The Data section is short and contains a caveat worth reading before anything else. You download the dataset, unzip it, and move the extracted folder to InterAct/OMOMO_new/. The README then says this build contains minor fixes to the original release, so your results may deviate slightly from those reported in the paper. That is the kind of note that saves a day of confusion. A dataset that has been patched to fix something is better than the original, but it means a reproduction will not match the paper exactly, and without the note you would assume your setup was wrong. The recommended path is different from the one you would take by default. The README recommends processing the data using the InterAct project, which is a separate repository, to obtain richer HOI skills and higher-quality outputs than the original OMOMO dataset. And there is a news entry confirming that a data conversion pipeline for bringing InterAct into simulation was released, with the processing code living in the InterAct repository rather than here. So the intended workflow is: take raw interaction data, run it through the InterAct conversion pipeline, and then train on the result. The dataset download is the fallback for a quicker start, and the paper's numbers correspond to the original OMOMO release rather than the processed version. ## Replaying the ground truth is the first thing you run The Data Replay section gives two ways to replay the ground-truth data, one per simulator, and running it is the natural first check on a fresh install. Under Isaac Gym: ```bash sh isaacgym/scripts/data_replay.sh ``` Under Isaac Lab: ```bash ./isaaclab/scripts/run_data_replay.sh --num-envs 8 --motion-dir InterAct/OMOMO_new ``` The Isaac Lab invocation shows the two settings you are most likely to change first. The --num-envs flag sets both cfg.num_envs and cfg.scene.num_envs, and the --motion-dir flag points at the data directory, which is where the downloaded and moved folder comes in. The README also lists a --headless flag, and the truncated text at the end of the README continues into it. Two things follow from the structure of that command. The data directory is a path relative to the repository, which means the replay only finds your data if the folder layout matches what the scripts expect, so the move step in the previous section is not optional. And the number of environments is a parallelism knob, so replaying on a machine with a small GPU is a matter of lowering a flag rather than changing code. The two replay paths matter because they are the compatibility check for the whole project. If the ground-truth data replays correctly in both environments, then the training pipeline you are about to run has a working data path, which is the thing that otherwise fails silently. ## A release history that shows the project maturing The News section is a dated log, and reading it in order tells you what the project actually built and when. The codebase and checkpoint for teacher policy inference were released on 2025-03-25, and the training code for teacher policies went live on 2025-04-11 with the invitation to train your own policy. The paper was selected as a CVPR Highlight on 2025-04-04, and a checkpoint with high-fidelity physics and enhanced contact precision followed on 2025-04-18. April 2025 was also when humanoid support arrived, ahead of schedule, starting with a small demo of the Unitree G1 with its original three-finger dexterous hands, and the invitation to explore whole-body loco-manipulation with humanoid robots. The student policy training pipeline shipped on 2025-05-26, and the initial release of the processed data and the teacher policy inference instructions came on 2025-06-03 and 2025-06-10. The InterAct data conversion pipeline was released on 2025-12-07, Isaac Lab support started on 2025-12-15 with data replay ready, and on 2025-12-17 the question was whether Isaac Gym checkpoints are compatible with IsaacLab inference, with a newly released implementation to answer it. Multi-GPU training support came on 2026-02-09. The arc is clear from the order: teacher policies first, then a student policy trained on the data, then a second simulator, then compatibility between the two, then multi-GPU. The last entry is the most recent and the most useful, because multi-GPU training is what turns a research script into something you can run at a useful scale. ## Conclusion Adopt InterMimic if you are doing physics-based human-object interaction research and want a released policy, checkpoints and a replay pipeline you can run rather than building one from scratch, and you have NVIDIA hardware and patience for the Isaac Gym and Isaac Lab setup. Do not adopt it as a lightweight dependency, because the two simulation stacks are heavy and the two environments must be kept isolated. Before you start, read the data section carefully, because the released dataset build contains minor fixes to the original release, so your numbers may differ slightly from the ones in the paper, and the recommended path is to process the data with the InterAct project instead. ## FAQ ### What is InterMimic? It is a research codebase for universal whole-body control in physics-based human-object interactions, with one unified policy spanning diverse full-body interactions with dynamic, heterogeneous objects. It works out of the box for both SMPL-X and Unitree G1 humanoids, and the paper was selected as a CVPR 2025 Highlight. ### How do I set up the Isaac Gym environment for InterMimic? Create a conda environment on Python 3.8 with `conda create -n intermimic-gym python=3.8`, activate it, install pytorch, torchvision, torchaudio and pytorch-cuda 11.6 from the pytorch and nvidia channels, then run `pip install -r requirement.txt`. You also need Isaac Gym itself, and you must export LD_LIBRARY_PATH so libpython3.8.so is discoverable after every activate. ### Will InterMimic reproduce the numbers in the paper exactly? Not exactly. The README notes that the released dataset build contains minor fixes to the original release, so your results may deviate slightly from those reported in the paper. It also recommends processing the data with the separate InterAct project to obtain richer HOI skills and higher-quality outputs than the original OMOMO dataset. ### How do I replay the ground truth data in InterMimic? With Isaac Gym, run `sh isaacgym/scripts/data_replay.sh`. With Isaac Lab, run `./isaaclab/scripts/run_data_replay.sh --num-envs 8 --motion-dir InterAct/OMOMO_new`, where the motion directory points at the dataset you unzipped into place and --num-envs sets both cfg.num_envs and cfg.scene.num_envs. ## Sources - [Issues](https://github.com/Sirui-Xu/InterMimic/issues) - [License: MIT](https://github.com/Sirui-Xu/InterMimic/blob/main/LICENSE) - [Project website](https://sirui-xu.github.io/InterMimic/) - [README](https://github.com/Sirui-Xu/InterMimic/blob/main/README.md) - [Sirui-Xu/InterMimic on GitHub](https://github.com/Sirui-Xu/InterMimic) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/sirui-xu-intermimic