Tencent-Hunyuan/MixGRPO: reinforcement learning for flow models, run on a 32 GPU cluster

What the project actually is

MixGRPO is titled Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE, and it was accepted to ECCV 2026, announced on 2026-07-01, which is also the date of the last push to the repository. The paper is on arXiv as 2507.21802, there is a project page, and the model checkpoint is on Hugging Face.

The authorship spans industry and academia. The authors are from Hunyuan at Tencent, the School of Computer Science at Peking University and the Computer Center at Peking University, and the code is published under the Tencent-Hunyuan organisation.

The news timeline tells you how the work evolved. The paper and the code were released on 2025-07-30, along with a model checkpoint fine-tuned from FLUX.1 Dev using the MixGRPO algorithm with several rewards at once, HPSv2, ImageReward and Pick Score. In October 2025 the comparison and visualisation results were updated with Flow-GRPO and Flow-DPO, both offline and online, on SD3.5-M LoRA. In February 2026 the project added support for Coefficients-Preserving Sampling as a more principled alternative to standard SDE sampling, and the comparison tables and enhanced visualisations in the paper were updated with it.

So the repository is a living research artefact rather than a release. There are no GitHub releases, the licence file is separate from the packaging metadata, and the way to track it is the paper and the commit history.

The setup is an environment, not an install

The quick start is a sequence of environment steps that assume a research workstation with a GPU and network access to model hubs.

The first step creates a conda environment on Python 3.12:

conda create -n MixGRPO python=3.12
conda activate MixGRPO

The second installs the system level pieces and then runs the repository's own environment script:

sudo yum install -y pdsh pssh mesa-libGL # centos
bash env_setup.sh

The comment on the yum line says centos, and the project notes that the environment dependency is basically the same as DanceGRPO, so the setup is inherited rather than designed from scratch.

The `pdsh` and `pssh` packages are the interesting part of that line, because they are parallel shell tools, and they are needed because the training script is a multi node launcher rather than a single process. The `mesa-libGL` package is there because the video stack underneath needs a GL implementation even headless.

The project structure follows the same logic. There is a `fastvideo/` directory holding the package, `scripts/` with the preprocessing, finetuning and inference scripts, `data/` for the hostfile and the prompt files, `env_setup.sh` at the root, and `assets/` for documentation images.

The reward models are the real dependency

The model preparation section is the longest part of the setup, and it tells you what the algorithm optimises against.

FLUX is downloaded first into `./data/flux` with the Hugging Face CLI, after a login.

Then four reward models, each with its own checkpoint directory. HPS v2.1 needs the HPSv2 repository cloned and the `HPS_v2.1_compressed.pt` and `open_clip_model.safetensors` files downloaded into `./hps_ckpt`. ImageReward needs `med_config.json` and `ImageReward.pt` in `./image_reward_ckpt`. Pick Score and CLIP Score are handled differently, by running a demo script in the package that downloads into the Hugging Face cache automatically, with the proxy options exposed on the command line:

python fastvideo/models/reward_model/pick_score.py \
    --device cuda \
    --http_proxy <Your HTTP_PROXY> \ # Default is None
    --https_proxy <Your HTTPS_PROXY>  # Default is None

The existence of two separate reward model download paths, one by hand and one by script, is a small sign of a codebase that grew rather than was designed in one pass.

The reward set is also the point of the multi reward setup. The released checkpoint was trained with several rewards at once, and choosing which ones to weight is a research decision the scripts leave to you.

Training assumes four nodes of eight GPUs

The training section is explicit about the scale, and that is the main thing to know before you try.

The training dataset is the training prompts in HPDv2, shown in `./data/prompts.txt`, and the pdsh command is used for multi node training with torchrun. The default resource configuration consists of 4 nodes, each with 8 GPUs, totaling 32 GPUs.

The host addresses go into `data/hosts/hostfile`, then a script sets the `INDEX_CUSTOM` environment variable on each node to 0, 1, 2 and 3 respectively, which is the usual pattern for giving each node a rank in a distributed job. The Weights and Biases key is set in the finetuning script, and the training itself is one command:

bash scripts/finetune/finetune_flux_grpo_FastGRPO.sh

Before that, the embeddings of the prompt dataset are precomputed, and the `prompt_path` parameter in `./scripts/preprocess/preprocess_flux_rl_embeddings.sh` is what you adjust to point at your prompts:

bash scripts/preprocess/preprocess_flux_rl_embeddings.sh

The separate preprocessing step is a reward for the model's own design, since computing prompt embeddings once and reusing them across the rollout is the kind of saving that makes a 32 GPU run affordable.

Inference and evaluation reuse the same scripts layout

Running inference is the same shape as training, with the checkpoint downloaded first and one script to adjust and one to run.

The MixGRPO model weight `diffusion_pytorch_model.safetensors` goes into the `./mix_grpo_ckpt` directory, then you adjust the input parameters in `scripts/inference/inference_flux.sh`, which the README says is currently set to default, and execute the single node script:

bash scripts/inference/inference_flux.sh

Evaluation follows the same pattern, with the prompt file set to the path of a JSON file generated during the run, and the test dataset is the test prompts from the same HPDv2 dataset, shown in `./data/prompts_test.txt`.

The layout is worth noticing in itself. Preprocessing, finetuning and inference each have their own directory under `scripts/`, and each step is a shell script with its parameters at the top rather than a Python module you import, which is what you would expect from a project that is primarily a record of one training run.

The packaging tells you how to treat this repository

The `pyproject.toml` is a research project manifest and it is instructive.

The package is named `fastvideo` at version 1.2.0, with a description of FastVideo, and the licence classifier is the Apache Software Licence even though the repository carries a separate `License.txt`. The build backend is setuptools with a floor of 61.0, and the Python requirement is 3.8 or newer, which is noticeably lower than the 3.12 the quick start creates.

The dependency list is pinned exactly, to the patch, for everything that matters: transformers 4.46.1, diffusers 0.32.0, peft 0.13.2, liger_kernel 0.4.1, accelerate 1.0.1, tokenizers 0.20.1, wandb 0.18.5, torchdiffeq 0.2.4, torchmetrics, pytorchvideo, decord, moviepy, bitsandbytes and many more, down to numpy 1.26.3. Those pins are a feature, because a research reproduction needs the exact versions the authors used, and they are also a trap, because this code is tied to one moment in the library ecosystem and will not float.

The packaging also excludes the non library parts, with `assets`, `docker`, `docs` and `scripts` excluded from both the sdist and the wheel, which tells you that the scripts are the product and the Python package is a means. The mypy configuration is strict for a project like this, with `disallow_untyped_calls`, `check_untyped_defs`, `no_implicit_optional` and `warn_return_any` all enabled, and there is a separate `requirements-lint.txt` for the linting toolchain.

There is one naming oddity worth flagging, since it will confuse a reader who arrives from the paper. The repository is MixGRPO and the method is the MixGRPO algorithm, while the Python package is called fastvideo, which is the video generation framework the method is implemented inside.

Editorial conclusion

Adopt this repository if you are doing research on reinforcement learning for flow matching diffusion models and you have the cluster to match, since the scripts encode the exact recipe the paper used, from the FLUX checkpoint through the reward models to the multi node launcher. Do not adopt it as a library to import, because the packaging is a research project named fastvideo with the scripts, data and assets excluded from the wheel, and it will not give you a stable API. Verify first that you can satisfy the hardware and dependency profile, the default configuration is four nodes of eight GPUs, and that you can reproduce the environment, since the exact pins to transformers, diffusers, peft and liger_kernel mean this code is tied to one moment in the library ecosystem.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Tencent-Hunyuan/MixGRPO on GitHub