PyMAF: human mesh recovery with a mesh alignment feedback loop

Two papers share one repository, on two branches

The repository holds the code for two related papers, and the default branch tells you which is which. The smpl branch carries PyMAF, 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop, by Hongwen Zhang, Yating Tian, Xinchi Zhou, Wanli Ouyang, Yebin Liu, Limin Wang and Zhenan Sun, presented at ICCV 2021 as an oral paper, with Zhang and Tian marked as equal contribution. The smplx branch carries PyMAF-X, Towards Well-aligned Full-body Model Regression from Monocular Images, by Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun and Yebin Liu, published in TPAMI in 2023. That split is the first thing to check, because the two have different model parameterisations and different expected outputs. The project pages for both are linked from the README, along with the arXiv papers, and the demo results are shown as frame-by-frame reconstruction GIFs from a flash mob and a dance video.

The environment is pinned to an older PyTorch on Python 3.8

The requirements section is specific and dated, which is normal for a research code release. Python 3.8 with a conda environment created with no default packages.

conda create --no-default-packages -n pymafx python=3.8
conda activate pymafx

PyTorch is tested on version 1.9.0 with a matching torchvision and a CUDA toolkit.

conda install pytorch==1.9.0 torchvision==0.10.0 cudatoolkit=11.1 -c pytorch -c conda-forge

pytorch3d comes from a git URL at the stable branch, and the rest of the packages come from requirements.txt.

pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable"

pip install -r requirements.txt

The requirements file itself shows how deep the dependency tree goes. There is chumpy and smplx 0.1.28 for the body model, pyrender and torchgeometry for rendering and geometry, trimesh for mesh handling, opencv-python, scikit-image, scipy pinned to 1.2.2, tensorboardx for logging, ultralytics 8.0.24, and three git dependencies, the cocoapi Python API, a yolov3-pytorch fork and the multi-person tracker. A pinned scipy of 1.2.2 is the kind of detail that will conflict with almost any other environment you already have.

The data directory is a checklist, and SMPL files are gated

The necessary files section is a manual checklist, and it is the part that makes setup slow. A script fetches the mesh downsampling data and the DensePose UV data from other repositories.

bash fetch_data.sh

The SMPL model files have to be collected by hand from the Max Planck site and a third-party repository, renamed and placed into the ./data/smpl directory. That is not a complaint about the project, it is how the SMPL model is licensed. The preprocessed training data comes from SPIN, and the final fits data also comes from SPIN, with an important note that using EFT fits for training is much better, and that compatible npz files are available from a linked share. The pre-trained checkpoint goes into ./data/pretrained_model. The README then shows the expected directory layout, with dataset_extras, the J regressor files for extra and Human3.6M, mesh_downsampling.npz, the pretrained model checkpoint, the three SMPL pkl files for female, male and neutral, smpl_mean_params.npz, final_fits and the UV data directory with the processed and symmetry transform matrices.

Running the demo, from a Colab notebook or from the command line

There is a shortcut for trying the model without any of that. The README points to a Google Colab notebook that needs no environment preparation, and the demo code can also be run locally. For an image input, the demo takes a checkpoint and a file.

python3 demo.py --checkpoint=data/pretrained_model/PyMAF_model_checkpoint.pt --img_file examples/COCO_val2014_000000019667.jpg

For video input the command takes a video file, with a comment noting the difference between a single person video and a video with multiple persons.

# video with single person
python3 demo.py --checkpoint=data/pretrained_model/PyMAF_model_checkpoint.pt --vid_file examples/dancer.mp4
# video with multiple persons
python3 demo.py --checkpoint=data/pretrained_model/PyMAF_model_checkpoint.pt --vid_file examples/flashmob.mp4

The repository has both example files, a COCO validation image and the two videos, and there is also a multi_person_tracker_yolov8.py file at the root.

Evaluation and training commands, and the EFT recommendation

The evaluation section covers two benchmarks. COCO keypoint localization needs the preprocessed data placed into ./data/dataset_extras, and then the eval_coco.py script.

python3 eval_coco.py --checkpoint=data/pretrained_model/PyMAF_model_checkpoint.pt

For 3DPW, the eval.py script takes a dataset argument.

python3 eval.py --checkpoint=data/pretrained_model/PyMAF_model_checkpoint.pt --dataset=3dpw --log_freq=20

Training follows a two-stage schedule. PyMAF is trained on Human3.6M at the first stage and then on a mixture of 2D and 3D datasets at the second, and the README gives the commands for both, with a preset checkpoint for the mixed stage and a batch size set through the misc config.

# training on COCO
CUDA_VISIBLE_DEVICES=0 python3 train.py --regressor pymaf_net --single_dataset --misc TRAIN.BATCH_SIZE 64
# training on mixed datasets
CUDA_VISIBLE_DEVICES=0 python3 train.py --regressor pymaf_net --pretrained_checkpoint path/to/checkpoint_file.pt --misc TRAIN.BATCH_SIZE 64

The one piece of advice the README puts at the top of both the repository and the training section is to use EFT fits, which it says can significantly improve the baseline, with more details in issue 58.

A research repository with configs, core and datasets directories

The repository layout follows the conventions of this kind of release. There are configs, core, datasets, models and utils directories alongside the scripts, plus demo.py, eval.py, eval_coco.py and train.py at the root, and fetch_data.sh for the data step. The configs directory is where the per-experiment settings live, and the core directory holds the model and loss code that the training and evaluation scripts both use, which is the structure that lets you swap the regressor without touching the data pipeline. The licence is one GitHub cannot classify, so the LICENSE file in the root is the authority, and the default branch is smpl rather than main, which catches people out when they clone and expect the PyMAF-X code. The last push was on 2026-09-15, the repository is not archived, and there are no GitHub releases, so the commits on the branch are the version history.

Editorial conclusion

PyMAF fits a research group working on human mesh recovery who wants a PyTorch implementation of the ICCV 2021 method and a way to compare against the SMPL-family baselines it was trained on. It does not fit someone who wants a turnkey pose API, since the setup involves registering for SMPL model files, fetching preprocessed SPIN data and building a specific dependency set. Verify first that you have the SMPL model files, since they are gated and cannot be shipped with the code, and read the note about EFT fits before you train, because the README says it significantly improves the baseline.

Frequently asked questions

Official sources

1. HongwenZhang/PyMAF on GitHub
2. Issues
3. Project website
4. README