hand_teleop_real_sim_mix_adr

A paper and codebase that trains real-world dexterous manipulation from simulated demonstrations, collecting roughly 50 virtual demos plus 15 real ones for fine-tuning.

CyberDemo is described as a data augmentation and training method that uses simulated human demonstration to help real-world dexterous manipulation. The README names the authors, links the paper, and thanks the Technology Innovation Program, project number 20018112, funded by the Korean Ministry of Trade, Industry and Energy for its autonomous manipulation work. The project is written in Python.

The workflow is spelled out in steps. First collect human demonstrations in simulation with any teleoperation method, where about 50 usually suffices, and gather 15 demonstrations in the real world for fine-tuning. Then replay the demonstrations with act.py, verify the augmentation with player augmentation.py, train and augment the simulation data with train act adr.py, and fine-tune on the real demonstrations with train real act.py.

The README is explicit that it only provides example scripts for the method, and points to a baselines directory for the comparison implementations. Teleoperation details live at the linked anyteleop server, and a set of simulation demos is downloadable from Google Drive with a prescribed directory structure.

A long configuration list controls the pipeline: the vision backbone with options like Moco, vit, and clip, task and object names such as pick place, pour, and dclaw with objects like a mustard bottle or sugar box, image augmentation factors of 1 for simulation and 5 for real data, kinematic augmentation for simulation demos, sensitivity checking, and the usual training knobs of learning rate, epochs, and KL weight. A multi-view augmentation path is triggered by task tags such as pick place multi view.

• Source 1 · Project repository
• Source 2 · README