TWIST2 collects humanoid demonstrations in simulation and on the robot

A data collection system, not a controller

TWIST2 is described as a scalable and portable humanoid data collection system, and the arXiv identifier in the README links to the 2025 paper. The author line is a long one, with Angjoo Kanazawa, Rocky Duan, Pieter Abbeel, Guanya Shi and Jiajun Wu among the authors, and a note marking equal advising.

The thing to understand is what the system is for. It is for collecting data, not for training a policy that ships to you. The README's third disclaimer says the high-level policy learning part will be released in a separate repository, modified from an existing diffusion policy project, and that the author is working on releasing it soon. So the deliverable here is the collection apparatus, and the policy is a follow-on.

The pipeline that is here covers a lot of ground. There is a training environment built on NVIDIA Isaac Gym, a separate motion retargeting environment built on MuJoCo, a teleoperation path where a person in a PICO VR headset drives the robot, an online motion server that the two sides talk to over Redis, an ONNX export path for the trained controller, and scripts for evaluation, deployment on real hardware, and the sim to real and sim to sim transitions.

The audience is therefore a robotics researcher or a graduate student with a humanoid platform and a VR headset, and the system is closer to a research platform than to a product. The top level directory listing shows the shape of it, with train.sh, eval.sh, teleop.sh, run_motion_server.sh, sim2real.sh, sim2sim.sh, to_onnx.sh, deploy_real/, data_record.sh, gui.py and gui.sh at the root, and the legged_gym/, rsl_rl/ and pose/ directories holding the training environment, the policy library and the robot control code.

The licence is MIT, the primary language is Python, and the project website is yanjieze.com/TWIST2.

Two conda environments because Isaac Gym and MuJoCo disagree on Python

The installation section opens with the single most important structural fact about the project, and it is a dependency conflict rather than a design decision.

The README says there will be two conda environments for TWIST2. One is called twist2, used for controller training, controller deployment and teleop data collection. The other is called gmr, used for online motion retargeting. The reason given is that Isaac Gym requires Python 3.8, but the newest MuJoCo requires Python 3.10.

So the environment is created like this:

conda env remove -n twist2
conda create -n twist2 python=3.8
conda activate twist2

and the retargeting environment separately:

conda create -n gmr python=3.10 -y
conda activate gmr

This is worth understanding as a category of work rather than an inconvenience. Isaac Gym is a proprietary binary distribution from NVIDIA that pins an older Python, and MuJoCo has moved on. Keeping them in separate environments means a script that runs in one cannot accidentally import the other's packages, and the retargeting side is free to use a current Python and a current MuJoCo.

The Isaac Gym installation is itself manual. The README says to download it from the official NVIDIA link and then install it from the python directory of the download, with the command being a change into the isaacgym/python directory and an editable pip install of the package there. That is a step the reader has to do by hand, and it is the point at which the project stops being self contained.

The Python 3.8 pin also has a knock-on effect on the rest of the stack, because the package list installed into the twist2 environment pins NumPy to a specific version:

The package list shows what the teleoperation path actually needs

The third installation step installs the local packages and the dependency set, and the list is long enough to be a map of the system.

The editable installs cover the three subprojects in the repository, with a change into the rsl_rl directory, a pip install of it in editable mode and a change back, the same for legged_gym, and the same for pose. Those three directories are the reinforcement learning library, the training environment and the robot control code respectively, and installing them in editable mode means your changes to them take effect without a reinstall.

After that comes the dependency set, and it is the list that tells you what the teleoperation pipeline involves:

pip install "numpy==1.23.0" pydelatin wandb tqdm opencv-python ipdb pyfqmr flask dill gdown hydra-core imageio[ffmpeg] mujoco mujoco-python-viewer isaacgym-stubs pytorch-kinematics rich termcolor zmq
pip install redis[hiredis] # for redis communication
pip install pyttsx3 # for voice control
pip install onnx onnxruntime-gpu # for onnx model inference
pip install customtkinter # for gui

There is a pinned NumPy because Isaac Gym needs it, hydra-core for configuration, wandb for experiment tracking, flask and zmq for the network and server side, onnx and onnxruntime-gpu for the exported controller inference, customtkinter for the graphical interface, pyttsx3 for voice control, and redis with the hiredis extra for the communication between the operator side and the robot side. pytorch-kinematics is there for the kinematic model of the robot, and pyfqmr for the motion retargeting problem.

The redis dependency is the interesting one, because it tells you the architecture. The teleoperation pipeline is not a single process. A server runs on one side, the headset and the GUI on another, and the robot on a third, and redis is the bus between them. That is what the run_motion_server.sh script at the root is for.

The redis setup is described in some detail, including enabling the server as a systemd service and editing the configuration file to bind to all interfaces with protected mode off. That is a permissive configuration on purpose, because the components are talking across the network, and it is also a configuration worth understanding before you copy it.

Unitree G1 over cable, with a PICO headset on the operator's head

The first disclaimer states the scope of what the released code can do today: with the current repository, you should be able to control a Unitree G1 via cable connection in both simulation and reality with a PICO VR headset.

The word cable matters. This is not a system that walks a robot loose in a room. The teleoperation path is a person in a VR headset driving a humanoid that is tethered, which is the sensible way to collect demonstration data with a robot that is expensive and that will fall over.

The hardware stack behind that is spelled out in the install steps. For sim to real on a laptop, you need a modified version of the Unitree SDK, and the README links a fork with instructions that clone the SDK2 repository, install the system dependencies with apt, install the Python dependencies including pybind11 and the stub generator, build the Python SDK binding with a build script and an explicit SDK path, and then copy the compiled shared object into the site-packages directory of the current conda environment. The instructions then verify the installation by importing the module and printing the available robot types. If you are doing sim to real with the onboard robot computer rather than a laptop, you do not need the SDK on your laptop at all.

The PICO side is a separate stack again. The README points at the XR-Robotics client releases for installing the PICO SDK on the headset, and at a deb package or a source build for the PC service on your machine, which you install with dpkg and then start before you do teleoperation. There is then a pybind build of the PICO streaming SDK, cloned and built inside the gmr environment, because the streaming SDK needs the newer Python.

So the full chain is a VR headset running a PICO SDK, a PC service on the operator's machine, a Python streaming binding compiled in the newer environment, and the Unitree SDK talking to the robot. The disclaimer about the teleop pipeline being complex and requiring some hardware setup is doing real work in that list.

The dataset, the example motions and a ready made ONNX controller

For anyone who wants to see the system work before assembling all of that, the README offers two shortcuts, and they are the fastest way to evaluate the project.

The first is the dataset. It is hosted on the author's Google Drive, and the README asks that if you use the dataset in your project you also add a proper citation to the work. You unzip it wherever you like and point the root_path setting in the dataset configuration file at the unzipped folder, and the configuration path given in the README is legged_gym/motion_data_configs/twist2_dataset.yaml.

The second is a small set of example motions in the assets/example_motions directory, which the README says you can use to test the system, and which are recorded by the author so there is no licence issue. That is a genuinely useful detail for anyone worried about the data's provenance, and it applies to the example motions rather than to the full dataset.

The third is the controller checkpoint, assets/ckpts/twist2_1017_20k.onnx, provided so you can test the system directly. The name suggests the training run it came from, and the onnx and onnxruntime-gpu packages in the dependency list are what make that checkpoint usable, which also tells you the deployment path is an ONNX inference on the robot rather than a Python policy.

The whole set of assets, the example motions, the controller checkpoint and the dataset, is what a new user should start from, because it is the difference between evaluating the system and rebuilding it.

The scripts at the root are the pipeline in one screen

The top level file listing is the clearest documentation in the repository, because the scripts are named after the stages of the workflow.

There is train.sh for training the controller, eval.sh for evaluation, to_onnx.sh for the export to ONNX, teleop.sh for the teleoperation session, run_motion_server.sh for the server that the teleoperation clients talk to, data_record.sh for recording the demonstrations, sim2sim.sh for running the trained policy in simulation, sim2real.sh for the transition to the real robot, deploy_real/ for the deployment directory, and gui.py with gui.sh for the graphical interface. The doc/ directory holds the rest of the documentation.

Two of these names carry the most information. The presence of both sim2sim.sh and sim2real.sh says the system expects you to validate the policy in simulation before it goes anywhere near the hardware, which is the standard and correct practice for this kind of pipeline. And the fact that data_record.sh is separate from teleop.sh says the recording is a distinct stage of the process rather than a side effect of the teleoperation itself, which is what you would want if you are building a dataset.

The three directories hold the code those scripts drive. legged_gym/ is the training environment and holds the dataset configuration. rsl_rl/ is the policy library, and the editable install in the install steps is what makes local edits to it take effect. pose/ holds the robot control code, which is the layer that talks to the Unitree hardware.

Taken together, the script names describe a research loop: train in simulation, export to ONNX, evaluate, drive the robot by teleoperation, record the data, and use the data to train the next policy.

The author's own caveats, and what they mean for an adopter

The news section of the README contains three disclaimers, and they are the most useful thing in the file for anyone deciding whether to adopt the project.

The first is the scope statement already discussed, control of a Unitree G1 over cable in both simulation and reality with a PICO VR headset.

The second is the maturity statement: the author is still working on better documentation and cleaning some onboard streaming and inference code, because the teleop pipeline is complex and requires some hardware setup. That is an honest admission that the code is in motion, and it is the reason the install instructions contain a duplicated step number, with two separate steps both labelled 4.

The third is the scope of what is not here: the high-level policy learning part will be released in a separate repository, modified from iDP3, and the author is working on releasing it soon. So the data collection system is the released component and the policy work is not.

The news entries are dated 2025-12-02, on which the author announced that TWIST2 is open sourced, said a video tutorial would follow, and linked a first successful reproduction by someone else on bilibili. The earlier entry from 2025-11-05 says TWIST2 is released and that the full code would follow within a month.

So the honest summary is that this is a research system released with a paper, with a hardware-dependent installation path that is still being documented, and with the most interesting downstream component deliberately deferred to another repository. The last push to the repository was on 2026-09-23, there are no GitHub releases, and the licence is MIT, so the code is yours to read and build on, with the caveat that you should expect to do some of the assembly yourself.

Editorial conclusion

Adopt TWIST2 if you are researching humanoid control and need a demonstration collection pipeline that works both in simulation and on a Unitree G1 with a VR headset, and if you are prepared to assemble the hardware and software stack yourself. Do not adopt it expecting a documented, supported product, because the README's own second disclaimer says the documentation is still being written and the onboard streaming and inference code is being cleaned up, and do not adopt it if you need the policy learning component, which the README says will be released in a separate repository. Before you start, read the three disclaimers in the news section, and check the Python version split, since Isaac Gym needs 3.8 and the newest MuJoCo needs 3.10, which is why the project asks for two conda environments.

Frequently asked questions

Official sources

1. amazon-far/TWIST2 on GitHub
2. Issues
3. License: MIT
4. Project website
5. README