PriorMDM turns a text prompt into long human motion in about half a second

A generative prior for motion, layered on the MDM codebase

PriorMDM is the official PyTorch implementation of Human Motion Diffusion as a Generative Prior, published at ICLR 2024, with an arXiv entry at 2303.01418 and a project webpage for the figures and videos.

The technique extends the earlier motion-diffusion-model work, and the README is explicit that everything here is ported from MDM, the original implementation by Guy Tevet and colleagues. The README even points at a shortcut for people who already have MDM installed from the official repo, telling them they can save time and link the dependencies instead of getting them from scratch.

What the project adds is described in a release status table with three rows and three columns. DoubleTake, for long motion, supports training, generation and evaluation. ComMDM, for two-person motion, supports the same three. Fine-tuned motion control also supports all three, and the README states it needs no extra dependencies beyond the base setup.

So the project is three research contributions in one repository, sharing a diffusion backbone and a set of data loaders, rather than a single product with a single use.

The 50-step models are the reason the tool became usable

The most consequential change is a change in the sampling budget, and the README announces it directly: PriorMDM is now about 40X faster, roughly 0.4 seconds per sample.

The arithmetic is given in two parts. The first is that 50 diffusion step models were trained and released instead of 1000 step models, which run 20X faster with comparable results. The second is that calling CLIP once and caching the result makes inference 2X faster for all models, and the README says it is backward compatible with existing models.

The 20X and 2X factors multiply into the roughly 40X headline. The caching change is the smaller one and the more interesting one, because CLIP is only used to embed the text prompt, and a generation script that re-encodes the same prompt for every sample is doing that work hundreds of times for no reason. The README frames the change in the spirit of just pull and enjoy.

The 50-step checkpoints are separate downloads rather than a flag, listed alongside the original 1000-step ones, so a user chooses between speed and the original model by choosing a file.

Setting up conda, the CLIP dependency and the body models

The environment requirements are stated plainly. The code was tested on Ubuntu 18.04.5 LTS and needs Python 3.8, conda3 or miniconda3, and a CUDA capable GPU, with one being enough.

ffmpeg is a system dependency, and the README gives the apt commands for it, with a pointer to a Windows guide for other platforms:

sudo apt update
sudo apt install ffmpeg

The conda environment is created from a file in the repository, and two packages are installed from git rather than from PyPI:

conda env create -f environment.yml
conda activate PriorMDM
python -m spacy download en_core_web_sm
pip install git+https://github.com/openai/CLIP.git
pip install git+https://github.com/GuyTevet/smplx.git

The CLIP install from git is worth noting, because it means the environment is not pinned to a released package version. The same applies to smplx, the SMPL body model library, which the model code depends on for body parameters.

The spaCy model download is a separate step for the same reason. A bare pip install of spaCy does not include the English model, and the text pipeline needs en_core_web_sm.

Downloading SMPL, GloVe and the T2M evaluators with three shell scripts

The model weights that the diffusion process needs are fetched by scripts in a prepare directory, and there are two paths depending on whether MDM is already installed.

If you already have an installed MDM, the README says to first change the path to the full path of your installed MDM and then run:

bash prepare/link_mdm.sh

If this is a first time setup, the dependencies are downloaded with three scripts:

bash prepare/download_smpl_files.sh
bash prepare/download_glove.sh
bash prepare/download_t2m_evaluators.sh

The script names say what they do. SMPL is the body model the motion is expressed in, GloVe provides the word embeddings, and the t2m evaluators are the text-to-motion evaluation models. The April 2026 news entry mentions the evaluator download links being fixed and asks users to re-run the download_t2m_evaluators.sh script, which is the kind of detail worth knowing if an evaluation run misbehaves.

The HumanML3D dataset is needed for all applications. The README says to follow the HumanML3D instructions and then copy the result into the repository:

cp -r ../HumanML3D/HumanML3D ./dataset/HumanML3D

Each variant needs its own dataset and its own checkpoint

After the shared dependencies, the three variants diverge, and the README is explicit about the extra files each one wants.

DoubleTake, the long motion model, needs the BABEL dataset. A processed version goes to ./dataset/babel, and two further downloads for evaluation go to the same directory, along with SMPLH dependencies placed at ./body_models. The base checkpoint my_humanml-encoder-512 is described as a reproduction of the MDM best model without any changes, and the 50-step variant humanml-encoder-512-50steps is the fast one.

ComMDM, the two-person model, needs 3DPW. The README notes that the authors cleaned 3DPW and converted it to HumanML3D format, so a processed version is provided and goes to ./dataset/3dpw. Two checkpoints are listed for it, pw3d_text for text-to-motion and pw3d_prefix for prefix completion.

Fine-tuned motion control needs no extra data. Its two checkpoints are root_horizontal_control, described as the base model finetuned for 80,000 steps on the horizontal part of the root control objective, and its 50-step counterpart, root_horizontal_control_50steps.

All model files are downloaded, unzipped and placed in ./save/. That directory is the one place every checkpoint ends up, so the model argument in a run script always points into the same folder.

WandB logging and the evaluation numbers you should not quote

The April 2026 release also added Weights and Biases support, enabled with the `--train_platform_type WandBPlatform` argument, and fixed several things worth knowing about.

One of those fixes concerns evaluation. The README reports a bug in the evaluation code covering caption length calculation and multi-modality metric normalization, and asks that the fixed results be used when citing PriorMDM. That is a concrete instruction to anyone writing a paper, and it means evaluation numbers published before that date are suspect.

The same news entry lists a visualization double-translation bug, an accuracy problem in loss logging, and the evaluator download links that needed re-running. The last item connects back to the prepare scripts, and the double-translation bug is the kind of thing that makes a rendered video look wrong in a way that is easy to misread as a model problem.

The evaluation release itself dates from April 2023, when the long-motions evaluation scripts were released for both the BABEL and HumanML3D datasets. The fine-tuned motion control scripts were fully released a few days earlier, and the first release of DoubleTake and ComMDM came on 14 April 2023 with training and generation available using pretrained models.

What the repository layout says about the scope

The directory names map onto the pipeline, which is a quick way to understand the project without reading the code. There is a diffusion directory for the denoising machinery, a model directory, data_loaders, train, sample, eval and visualize, plus utils, assets, dataset and body_models, and a prepare directory holding the shell scripts.

The sample directory holds one file per capability, and those four files are the clearest statement of what the project can do. sample/double_take.py covers the long motion generation, sample/two_person_text2motion.py and sample/two_person_prefix_completion.py cover the two ComMDM modes, and sample/finetuned_motion_control.py covers the root control variants. Each of those is where a new user should look to understand the arguments a given model expects.

Training, evaluation and generation are all present, which is unusual for a paper repository and is what the release status table records. The distinction between a research codebase and a usable tool here is that the trained checkpoints and the evaluator models are all provided, so the expensive parts of reproducing the paper do not have to be redone.

The cost of that completeness is the setup. Between ffmpeg, the conda environment, two git installs, three download scripts, the HumanML3D dataset, per-variant datasets and several checkpoints, a first run is a long afternoon of setup before any motion appears.

Editorial conclusion

PriorMDM is worth adopting for research groups working on human motion generation, for anyone extending the MDM codebase who wants long sequence, two-person or root controlled variants with pretrained weights, and for teams that already have MDM installed and can link dependencies instead of re-downloading them. It is the wrong choice for a production animation pipeline, since it needs a CUDA GPU, a conda environment and the SMPL body models before it produces anything. Before running it, check the evaluation results you plan to cite, because the April 2026 release fixed a caption length calculation bug and a multi-modality metric normalization bug, and the README asks that the fixed numbers be used.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. priorMDM/priorMDM on GitHub
4. README