# SPIDER retargets human motion onto robot hands and humanoids > SPIDER is a physics-based retargeting framework that turns human video into robot actions for dexterous hands and humanoids, with Hydra configs per dataset, support for several simulators, and a released dataset of 7,876 successful trajectories. The licence is non-commercial, and the dependency list is heavy. **facebookresearch/spider** — A general physic-based retargeting framework. - Repository: https://github.com/facebookresearch/spider - Stars: 583 · Forks: 66 - Language: Python - License: NOASSERTION - Published: 2026-10-22 · Updated: 2026-10-22 · Language: en - Canonical page: https://hysenlabs.com/projects/facebookresearch-spider ## Retargeting as a pipeline, not a single function SPIDER, Scalable Physics-Informed DExterous Retargeting, is a general framework for physics-based retargeting from human to diverse robot embodiments, covering both dexterous hands and humanoid robots. The description says it is designed to be a minimum, flexible and extendable framework for human2robot retargeting, and the repository ships a pipeline animation from human video to robot actions. That word minimum is the design brief. The code base does not try to be a complete humanoid control stack. It is a set of pieces, with the retargeting pipeline as the spine, that you assemble. The features list names support for 9 or more robots and 6 or more datasets out of the box, native support for multiple simulators and for downstream training pipelines including HDMI and DexMachina, and a sim2real ready claim. The gallery shows the range: hand tasks such as Inspire picking a tea pot, Xhand playing a glass and Schunk picking a board, and humanoid tasks such as G1 picking and running, H1 kicking, and a T1 skip. ## Three simulators, two viewers, and a real robot The portability story is told through three tables in the README. Simulators supported are Genesis, Mujoco Warp and IsaacGym, and the example scripts carry names that match, with `run_dexmachina.py`, `run_hdmi.py` and `run_maniptrans.py` alongside the main `run_mjwp.py` and `run_mjwp_fast.py` entry points. Viewers supported are Mujoco and Rerun, which matters because inspecting a retargeting result is half the work. There is also a deployment section, with real robot gifs for picking a cup, rotating a bulb, unplugging a charger and picking up a duck. The claim to check is sim2real ready, which the README asserts as a feature. The gallery is evidence of runs, not of a measured transfer gap, and the published documentation gives no sim2real success rate, so treat that line as a direction the work is heading rather than a number you can plan against. ## The Hydra override is the configuration surface Configuration is Hydra, and the way the project uses it is worth understanding before you run anything. Each example script is invoked with a `+override=` argument that selects a config, plus `task`, `embodiment_type`, `data_id` and `robot_type` arguments. The README describes the override as picking the right Hydra config and the right task and embodiment type for that dataset. ```bash uv run examples/run_mjwp_fast.py +override=gigahand_fast task=p36-tea embodiment_type=bimanual data_id=0 robot_type=xhand uv run examples/run_mjwp_fast.py +override=arcticv2_fast task=s01-box_use_01 embodiment_type=bimanual data_id=0 robot_type=xhand uv run examples/run_mjwp_fast.py +override=oakinkv2_fast task=pick_spoon_bowl embodiment_type=right data_id=0 robot_type=xhand ``` There is a second variant, `run_mjwp.py`, described as the original, slower config from the paper, and the naming convention is that a `_origin` suffix selects it, with the note that gigahand currently ships `_origin` only. The two script names are the fast path and the paper-faithful path, and the fast one is the one the quickstart leads with. The `spider/` package holds the processing code, and the README points at `spider/process_datasets/arcticv2.py` for the list of supported Arctic objects, which is the kind of file you read when you are trying to work out why your task name is not accepted. ## Dataset processing is where the pipeline diverges The three reference datasets behave differently, and the README is specific about it. OakInk v2 is the recommended OakInk pipeline: it loads directly from the official OakInk-v2 raw data, crops each clip to a short window centred on the grasp moment, with a default of 1.5 seconds pre-grasp and 2.5 seconds post-grasp, and uses the maniptrans-derived right-hand object as the manipulation target. The legacy `oakink` pipeline, which consumes already-baked maniptrans pickles starting after the grasp, is still available through `+override=oakink`, but for new work the recommendation is OakInk v2. Arctic v2 reads Arctic raw_seqs and clips to a 4 second window centred on the object's motion onset, default 2 seconds before and 2 after, and only the object's bottom part is kept because it is rigid. Arctic task names follow the pattern subject-sequence, such as `s01-box_use_01`, `s01-ketchup_use_01` and `s01-laptop_use_01`. So the same script produces different behaviour depending on the source, and the windowing parameters are not cosmetic: they change which part of the motion the retargeting sees. ## The released dataset and how to inspect it On 2026-09-22 the project released `SPIDER retarget_full` on Hugging Face, with 7,876 successful right-hand trajectories from 2,885 source episodes across four datasets and four robot hands. The dataset section names the sources as DexYCB, HOT3D v2, HRDexDB and OakInk, and the robot hands as Allegro, Xhand, Inspire and Sharpa. The news entry phrases it as successful trajectories, which is a meaningful qualifier: these are the retargets that worked, so the dataset is a curated success set rather than a record of every attempt. The repository includes `examples/inspect_dataset.py` and the documentation has a Download and inspect with Viser section, and `viser` is one of the runtime dependencies. The Viser viewer is also where the optional 21-joint MANO skeleton comes in, which requires manotorch plus the gated MANO model files from mano.is.tue.mpg.de. ## A dependency tree that dictates your environment The dependency list is the practical constraint, and the pyproject file is unusually explicit about why. The runtime needs numpy, torch, loguru, tyro, loop-rate-limiters, hydra-core, matplotlib, imageio with ffmpeg, mediapy, opencv-python, mujoco at 3.7.0, mujoco-warp, warp-lang at 1.12.1, rerun-sdk, viser, ipdb, open3d, pymeshlab below 2025, huggingface_hub, gspread, google-auth and mink. Two of those pins carry comments explaining a decision. pymeshlab is held below 2025 because the 2025 wheels require glibc 2.35, and the pin supports older Linux hosts. The OakInk2 toolkit lives outside the resolver and has to be installed manually after `uv sync`. The MANO skeleton has a similar problem, because manotorch pins an unbuildable opencv-python, so it also cannot live in the resolver. The requirements file adds two direct git dependencies, mujoco-warp and newton with its examples extra, both pinned to specific commits. ## Two install paths, and a licence question to settle first There are two quickstarts, and they differ in more than the package manager. The uv path needs git-lfs, because the example datasets are cloned from Hugging Face, and the project warns that `uv` must use Python 3.12. ```bash sudo apt install git-lfs git lfs install git clone https://huggingface.co/datasets/retarget/retarget_example example_datasets ``` ```bash uv sync ``` The conda path builds the environment by hand. ```bash conda create -n spider python=3.12 conda activate spider python -m pip install --upgrade pip python -m pip install -r requirements.txt python -m pip install --no-deps -e . ``` And then the pipeline runs on a processed trial with `python examples/run_mjwp.py`. On the licence, the README badge says CC BY-NC 4.0 while the repository's recorded licence identifier is a custom one that GitHub cannot classify, with the terms in the LICENSE file. The non-commercial marker is the part that matters for adoption. The setup.py header also carries a Meta Platforms copyright notice, and the project sits under the facebookresearch organisation. ## Conclusion Adopt SPIDER if you need human-to-robot retargeting as a pipeline you can extend, you work in simulation first and want sim2real transfer, and you can live with a research-scale dependency tree. Skip it if you need a commercial product, because the README carries a CC BY-NC 4.0 badge, and skip it if you want a light library, since the runtime pulls in PyTorch, MuJoCo, Warp, Open3D and pymeshlab. Before you commit, read the dataset preprocessing notes for your source, because the OakInk and Arctic pipelines clip different windows and the README points at different behaviour for each. ## FAQ ### What is SPIDER and what does it do? SPIDER is a general framework for physics-based retargeting from human to diverse robot embodiments, covering dexterous hands and humanoid robots, providing a pipeline from human video to robot actions. It supports 9 or more robots and 6 or more datasets, and ships a released dataset of 7,876 successful trajectories. ### Which simulators does SPIDER support? Genesis, Mujoco Warp and IsaacGym, with the example scripts `run_dexmachina.py`, `run_hdmi.py` and `run_maniptrans.py` alongside `run_mjwp.py` and `run_mjwp_fast.py`. Viewers are Mujoco and Rerun, and the documentation describes Viser for inspecting the released dataset. ### How do I install SPIDER? With uv, run `uv sync` after cloning the example datasets with git-lfs, and make sure uv uses Python 3.12. With conda, create a Python 3.12 environment, then install from requirements.txt and the package itself with `python -m pip install --no-deps -e .`. ### What licence is SPIDER released under? The README carries a CC BY-NC 4.0 badge, so the terms are non-commercial, while the repository's recorded licence identifier is a custom licence that GitHub cannot classify, with the text in the LICENSE file. The setup.py also carries a Meta Platforms copyright notice. ## Sources - [facebookresearch/spider on GitHub](https://github.com/facebookresearch/spider) - [Issues](https://github.com/facebookresearch/spider/issues) - [README](https://github.com/facebookresearch/spider/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/facebookresearch-spider