BFM-Zero: a promptable behaviour model for humanoid control

One latent vector, many behaviours

BFM-Zero is a behavioural foundation model for humanoid control, and the word promptable is the whole idea. Instead of training a separate policy for every task, the project learns a latent representation of behaviour and then lets you query it: give it a motion to track, a goal to reach, or a reward to optimise, and it produces the corresponding latent vector z, which the policy then executes.

The training is described as unsupervised reinforcement learning. The training data is motion data rather than task demonstrations with labels, and the evaluation is written in terms of tracking error against expert motions, which is how you tell whether a learned behaviour actually matches the motion it was supposed to reproduce.

The audience is humanoid robotics researchers with the compute to train on. The stated requirements are Python 3.10, a CUDA-capable GPU, and Isaac Sim on Linux or MuJoCo for simulation. The paper is on arXiv as 2511.04131, the project page is at lecar-lab.github.io/BFM-Zero/, and the code is in this repository under the LeCAR Lab organisation, with a custom licence that GitHub's classifier cannot categorise, so the LICENSE file is the reference for terms.

The important framing is that this is research code with a staged release, not a supported package. That shape shows up in everything below.

The code is released in stages, across three branches

The repository README lists the release plan as a checklist, and reading it tells you what you can and cannot do today.

Marked as done are the pretrained checkpoints with sim-to-sim and sim-to-real deployment, which live on a branch called deploy, and the minimal inference code with a tutorial, on a branch called minimal_inference. The full training and evaluation pipelines are also marked done, along with a distributed training setup in MJLab, described as six hours of training, together with teleoperation, implemented through the UFO project.

Still unchecked is the minimal training code with RTX 4090 support. That single open box is the most useful thing on the list for anyone reading it, because it says plainly that the lightest path to training on consumer hardware is not finished. The 4090 mention is a hint about which GPU the team expects people to have, and a sign that the full pipeline assumes more hardware than a workstation card.

So there are three entry points, and they are not equivalent. deploy is for running a trained policy, minimal_inference is for a small inference example, and the default branch carries the training pipeline. Picking the wrong one is the first mistake to avoid.

Setup: Git LFS, uv, and a lot of dependencies

The installation has three steps, and the first one is unusual. The repository uses Git LFS for motion data and models, so a plain clone leaves you with pointer files instead of the actual data. The documented sequence is:

git clone https://github.com/LeCAR-Lab/BFM-Zero.git
cd BFM-Zero
git lfs install
git lfs pull

The README also warns that if the repository has exceeded its LFS budget you can get the data from a Hugging Face dataset path instead, which is worth knowing about before you start a long download.

The second step is the package manager. The project uses uv, installed either with the installer script or with pip:

curl -LsSf https://astral.sh/uv/install.sh | sh

or, with pip, `pip install uv`. The third step resolves the environment from the project directory:

uv sync

That sync pulls a large set of dependencies. The project file pins Python to 3.10 exactly, and the dependency list includes Isaac Lab 2.0.2 and Isaac Sim 4.5.0 on Linux, MuJoCo 3.3.1, torch 2.5.0, Hydra, tyro for the command line, ONNX and onnxruntime, and a package sourced directly from a Git repository, with a separate index configured for NVIDIA packages. The sim2real extras, including a robot SDK and a keyboard helper, are present but commented out, which matches the staged release: the deployment path is in the deploy branch rather than in the default environment.

Training runs through humanoidverse.train and a table of switches

Training is launched with a single module invocation:

uv run python -m humanoidverse.train

The README notes to change buffer_device to "cuda:0" if you have larger video memory, which tells you the replay buffer placement is the first thing to adjust on a bigger card.

The parameters are grouped in the documentation into areas. Scale covers num_env_steps, online_parallel_envs, buffer_size and checkpoint_every_steps. Paths cover work_dir and the environment's lafan_tail_path, which is where the expert motion data comes from. Run control covers seed, use_wandb and the wandb_pname, wandb_gname and wandb_ename naming fields. The policy and optimiser area covers update_agent_every, num_agent_updates, num_seed_steps, and agent settings such as batch_size, lr_actor, lr_critic and discount. Robot and environment settings are overridden through hydra_overrides, with examples like robot=..., robot.control.action_scale=... and env.config.lie_down_init=...

You can override from code by passing a custom TrainConfig, or extend the command line to accept Hydra and tyro overrides, so the configuration is a Python object rather than a locked-in file.

One training target is given: after 50 to 100 million steps, eval/emd should be lower than 0.75. That is a checkpoint for whether a run is on track, not a claim about a result, and it is the one number in the documentation you can use to judge a training run.

The evaluator changed its metrics, so old numbers do not compare

The evaluation section contains a warning that is easy to skim past and expensive to ignore.

The training evaluator now reports joint angle errors as joint_angle_l2_rad, joint_velocity_l2_rad_s and joint_acceleration_l2_rad_s2, which are time-averaged L2 norms. These replace the earlier mpjpe_l, vel_dist and accel_dist metrics, and the documentation states plainly that the old values are not directly comparable. obs_state_emd now includes all configured joints, while the main emd calculation is unchanged.

The practical effect is that any number you remember from an earlier run of this code, or from a paper that used the old metric names, is on a different scale. The emd figure remains the one to watch, since it is the value the 0.75 guidance refers to, but a change of metric names plus a change of what goes into one of them means a discontinuity in your own logged history.

If you are extending this work, the first thing to do is record which metric set your checkpoints were trained under, because the repository does not tag checkpoints with it.

Three inference scripts, one latent z, and an ONNX export

Inference is split by task, and the three scripts share a shape. Tracking inference runs motion tracking and extracts the latent z for a motion, goal inference computes z for different goals, and reward inference computes z for reward-based tasks and evaluates performance. All three use tyro for the command line, and the help output is documented:

uv run python -m humanoidverse.tracking_inference --help
uv run python -m humanoidverse.goal_inference --help
uv run python -m humanoidverse.reward_inference --help

The common arguments are worth reading before the first run. --model_folder points at the trained model directory and must contain checkpoint/ and config.json. --data_path optionally overrides the default LaFan data path. --simulator selects isaacsim, which is the default, or mujoco. --headless defaults to True, and --no-headless shows the viewer. --save_mp4 writes rendered videos.

A concrete tracking run looks like this:

uv run python -m humanoidverse.tracking_inference \
    --model_folder /path/to/model \
    --data_path humanoidverse/data/lafan_29dof.pkl \
    --no-headless \
    --save_mp4

Note the instruction that --model_folder should point at the outer model directory, the one containing checkpoint/, which is an easy mistake to make. The outputs land under model_folder/tracking_inference/ as zs_{MOTION_ID}.pkl, one latent per motion, plus tracking.mp4 comparing expert and policy, with multiple motions producing tracking_<motion_id>.mp4. Goal inference additionally needs goal_frames_lafan29dof.json, which the script searches for in several locations, and writes goal_reaching.pkl as a dictionary mapping goal name to z. Every script exports the policy to ONNX as {model_name}.onnx in an exported/ subdirectory, and that export is the bridge to the deploy branch and to real hardware.

MuJoCo is the escape hatch, and the limits are elsewhere

Two things constrain adoption more than the model itself.

The first is hardware. A CUDA-capable GPU is a stated requirement, and the training path depends on Isaac Lab and Isaac Sim, which the dependency file pins to Linux only. The MuJoCo option is described as a way to run without Isaac Lab, and the documentation qualifies it as usable for sim2sim visualisation. That is a narrower claim than running training in MuJoCo, and it should be read as such.

The second is the incomplete release. The minimal training code for RTX 4090 support is still an open item, and the repository publishes no GitHub releases at all, so there is no versioned artefact to point at. The last push was on 2026-10-19, so the work is moving, but the delivery mechanism is branches and a repository rather than packages.

The licence is the third constraint. It is a custom licence that GitHub's classifier cannot categorise, which means it is not one of the standard permissive licences and the terms have to be read in the LICENSE file before you build on it.

Set against those, the alternative is to train a task-specific policy yourself for each behaviour you need, on the same simulator stack, with a policy per task. The difference in approach is the representation: BFM-Zero amortises behaviour into a latent space that you can query, export to ONNX and ship to a robot, where the per-task approach gives you a separate model, a separate checkpoint and separate deployment for every behaviour.

Editorial conclusion

BFM-Zero fits a research team that wants one humanoid policy prompted by a latent vector instead of a separate policy per task, and it is the wrong tool for a team without a CUDA GPU or without patience for Isaac Sim, since the MuJoCo path is described as a visualisation route rather than a training one. Verify first that the branch you need exists for your task, because the code is split across deploy, minimal_inference and the main branch, that the minimal training code with RTX 4090 support is still an open item in the staged release checklist, and that the licence in the LICENSE file permits your use, because GitHub cannot classify it.

Frequently asked questions

Official sources

1. Issues
2. LeCAR-Lab/BFM-Zero on GitHub
3. Project website
4. README