WiLoR: a two-stage 3D hand reconstruction model with a fast inference path

Two checkpoints split localization from reconstruction

WiLoR is organised as two separate stages, and the repository makes that explicit in the names of the checkpoints it asks you to fetch. One file is named detector.pt and the second is named wilor_final.ckpt, and both are pulled from the same Hugging Face space that hosts the public demo. The top level of the repository backs this up. A demo.py script sits at the root next to a wilor/ package directory, a demo_img/ folder of sample images, and a pretrained_models/ directory that the download commands write into.

Two stages also mean two files to place, and the instructions treat both downloads as mandatory rather than optional. Anyone expecting a single checkpoint that crops, detects and reconstructs in one pass will find the documented setup points somewhere else. The separation also explains why the demo command carries both an input folder flag and a separate switch for saving the resulting mesh. The detection half inherits from the Ultralytics ecosystem, which the credits list as one of the two sources the code adapts, and the reconstruction half is the research contribution itself, trained on the WHIM dataset that the repository ships as a separate directory with its own download instructions.

Half precision and depth pruning behind the --fast flag

The newest documented feature is a --fast argument on the demo, described in a March 2026 update note. The note says the demo now supports half precision inference and depth pruning, which together produce up to 1.6 times faster inference with minimal performance degradation, quoted as roughly 0.05 millimetres of extra mean per joint position error. The note is unusually candid about the size of the trade, which is the most useful number a reader gets from it.

Half precision is the numeric one: the forward pass runs in reduced precision rather than full float32. Depth pruning is described only by name, and the note does not say what gets pruned or by how much. Neither capability is exposed as its own flag in the documented command line, so you receive the pair as one bundle rather than as two knobs to tune independently. The practical effect matters most in the Gradio interface, where the same model is invoked repeatedly on user uploads and where a 1.6 times speedup shortens the wait on every click. The cost is a small but real increase in joint error, which is a fair price for interactive use and a poor one if your downstream metric is joint accuracy.

The install hinges on a gated MANO download

Installation starts with a recursive clone, and the recursive flag is not cosmetic: submodules are pulled in that a shallow copy would leave empty. The project says the code was tested with PyTorch 2.0.0 and CUDA 11.7, and it recommends an anaconda environment for the dependency set.

Everything then hinges on one manual step that no script performs, the MANO right hand model. You need an account on the MANO website, you download an archive whose files begin with mano_v, and you unzip the right hand file named MANO_RIGHT.pkl into the mano_data/ folder that already exists in the repository. Until that template is in place the model has no hand to deform. The pretrained weights then go into pretrained_models/ with two separate downloads, one per stage.

git clone --recursive https://github.com/rolpotamias/WiLoR.git
cd WiLoR

conda create --name wilor python=3.10
conda activate wilor

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu117
pip install -r requirements.txt

wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt -P ./pretrained_models/
wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/wilor_final.ckpt -P ./pretrained_models/

The two checkpoint names are the ones to watch for. If a stage file is missing or placed elsewhere, the failure surfaces at load time rather than at download time, since the wget calls use the -P flag to place files by directory but not by name.

Running the demo and the local Gradio interface

With the weights and the MANO template in place, the demo is a single command over a folder of images, with the fast mode included in the documented invocation.

python demo.py --img_folder demo_img --out_folder demo_out --save_mesh --fast

The demo_img/ directory that ships with the repository is the intended starting input, and the save_mesh flag asks for mesh output rather than a bare plot. A local browser interface is available for hands-on inspection, which is the fastest way to judge whether the reconstruction quality suits your use case before you wire the model into anything.

python gradio_demo.py

The script sits at the repository root and gradio is listed among the dependencies, so the interface is part of the standard install rather than an optional extra. For a research group the practical value of the Gradio path is iteration speed: you can drag through frames and compare against your own annotations. The repository also includes a download_videos.py script at the top level, which suggests a video-oriented path into the same inference code, although the README does not describe its arguments or output layout. There is also a WHIM dataset directory with a Dataset_instructions.md file for anyone who intends to retrain rather than only run the pretrained stages.

WiLoR-mini trades the dependency list for demo only scope

The requirements file has twenty lines, and it is not a light one. Alongside numpy, opencv-python, timm, einops, pandas, yacs, pyrender and scikit-image, it pulls in pytorch-lightning, xtcocotools, webdataset, three separate hydra packages, rich, pyrootutils and gradio. Two entries are pinned tightly, smplx==0.1.28 and ultralytics==8.1.34, and one is a direct git reference, chumpy installed from mattloper/chumpy. A chumpy install in particular is where Python version pinning tends to bite, which is why the documented environment fixes python=3.10 rather than leaving it open.

A community contributor, warmshao, produced a shortened path that installs with a single pip command.

pip install git+https://github.com/warmshao/WiLoR-mini

The README is careful about what this buys you. The command is presented as a simplified version of WiLoR that can be used for demo only, and it points to the separate WiLoR-mini repository for the details. That distinction matters if you are evaluating WiLoR for research use: the full repository with hydra, webdataset and the lightning trainer is what supports work beyond running the pretrained stages, while the mini install targets the demo path. Anyone building on the model should plan to work from the full clone rather than the one line install, and should read the WiLoR-mini documentation before assuming the two are interchangeable.

Pinned dependencies, no releases, and a dated last push

The repository publishes no GitHub releases, which removes the usual upgrade signal. There is no version tag to track, no changelog and no packaged distribution for the main project, so the only way to know what you have is the commit you cloned. The project is not archived, but the last push was on 2026-04-07, which puts the most recent change roughly six months before today. That is recent enough to show the work is not abandoned and old enough that you should not assume a steady release rhythm.

Upgrades therefore have to be done by hand, and the pinned versions are the anchor. ultralytics==8.1.34 and smplx==0.1.28 both matter, since the detector half rides on the Ultralytics library and the body fitting rides on smplx, and both pins are old enough that a fresh pip install against current releases can pull in code that no longer matches the model code. The README does not document a rollback procedure, nor does it publish a compatibility matrix beyond the single tested combination of PyTorch 2.0.0 with CUDA 11.7.

The maintenance story for WiLoR is therefore a research project with a fixed set of pinned versions rather than a supported product. Expect to pin your environment from a known commit and to check that your GPU driver still matches CUDA 11.7 before blaming the model for odd outputs.

Model weights are non commercial and no derivatives

The licence situation has three layers and all three bind you. The README states that WiLoR models fall under the CC-BY-NC--ND licence stored as license.txt in the repository, and GitHub classifies the overall repository licence as a custom one that it cannot map to a standard identifier. The NC component rules out commercial use of the weights, and the ND component restricts redistribution of modified versions, which is worth noting for anyone planning to fine-tune and publish a checkpoint.

The second layer is the Ultralytics library that the repository depends on, which carries its own licence, and the third is the MANO model, which the README explicitly says falls under the MANO licence. The instruction to comply with the terms of these external licences is direct: using the repository means accepting all of them. That makes the dependency chain more than a packaging detail, because the strictest term in the chain, the non commercial and no derivatives licence on the weights, governs what you can ship.

None of this is legal advice and the licence texts themselves are the authority, but the practical summary is clear. The weights fit research, evaluation and teaching, and they do not fit a commercial product, a customer deliverable or a redistributed derivative model.

Where WiLoR is the wrong tool

Several real constraints push engineers toward something else. The most visible is handedness. The documented setup only asks for MANO_RIGHT.pkl, so a pipeline that must reconstruct left hands has no template to start from and would need extra work that the documentation does not cover. The second is hardware. The tested combination is CUDA 11.7 with PyTorch 2.0.0, and the fast mode is a speedup rather than a way to run on a machine without a GPU, so CPU inference is not a supported path here.

The third is training. The dependency list includes hydra-core, hydra-submitit-launcher, webdataset and pytorch-lightning, which points to a real training stack, and the WHIM dataset directory has its own download instructions, but the README documents no training command, no config file example and no evaluation script. A team that wants to reproduce the paper's training pipeline has to infer it from the code. The fourth is the alternative question. The credits state that parts of the code are taken or adapted from HaMeR and from Ultralytics. WiLoR's difference from HaMeR is the arrangement it adds: an explicit in the wild localization stage with its own detector checkpoint in front of reconstruction, plus the half precision and depth pruning path that HaMeR is not documented here to offer. If you already have HaMeR in a pipeline and only need better detection, the honest comparison is that WiLoR adds a detector and a speed path, not a new kind of hand model.

Editorial conclusion

WiLoR suits research groups that need a working 3D hand reconstruction baseline and can satisfy the MANO download gate and the non commercial weight licence. It does not suit a product team shipping closed source models, or anyone needing left hand output, since the documented setup only places a right hand template. Before committing, check that MANO_RIGHT.pkl sits in mano_data/, that both checkpoints landed in pretrained_models/, and that your CUDA and PyTorch versions match the tested pair of CUDA 11.7 and PyTorch 2.0.0.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. rolpotamias/WiLoR on GitHub