UniRL runs one RL loop across image, video, and language models, and its two dependency files disagree with each other
UniRL is a Framework for Unified Multimodal Model Reinforcement Learning
At a glance
- What is it?
- UniRL is Tencent-Hunyuan's reinforcement learning framework for unified multimodal models: Hydra configs pick a domain trainer, and pluggable rollout engines, algorithms, model bundles, and reward services plug into one loop. The dependency declaration is where the friction is, because pyproject.toml and requirements.txt name different lower bounds for diffusers and peft.
- Who is it for?
- UniRL is a serious framework for teams already running SGLang or a HuggingFace rollout on multi-GPU nodes, since the model table is backed by one runnable recipe per row and the loop itself is small enough to read. It is a poor fit if you want a lightweight install, because the documented path pulls a full inference stack plus a separately installed flash-attn, and because two dependency files describe the same requirements differently.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- Tencent-Hunyuan/UniRL1,049 ★ · +78
| Repository | 2026-09-19 | 2026-11-15 |
|---|---|---|
| Tencent-Hunyuan/UniRL | 971 | 1k |
78 Stars in the last 57 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
Two dependency files name different lower bounds
pyproject.toml requires `diffusers>=0.38.0` and `peft>=0.20`. requirements.txt, in the same repository root, requires `diffusers>=0.37.0` and `peft>=0.14.0`. Those are not rounding differences, they are four separate lower bounds on two packages, and nothing in either file says which one wins. The install notes in requirements.txt settle the mechanism without settling the conflict: run `pip install -r requirements.txt --no-build-isolation`, then `pip install -e . --no-deps --no-build-isolation`. The second command suppresses dependency resolution entirely, so the environment that results is the one requirements.txt described, while the package metadata advertises the tighter bounds. Files in the tree also include `INSTALL.md`, `lint/`, and `.pre-commit-config.yaml`, so there is a separate install document and a separate lint configuration alongside the two dependency lists.
torch is deliberately absent from the package dependencies
The base dependency list of pyproject.toml contains numpy, ray, diffusers, hydra-core, omegaconf, transformers, huggingface-hub, einops, peft, safetensors, Pillow, requests, psutil, and tensordict, and an inline comment says it is engine-agnostic only, with torch intentionally absent because it enters via exactly one engine extra, which is what allows the per-engine CUDA stacks. So the package itself does not choose a torch. requirements.txt does the choosing instead: `torch>=2.1`, `torchvision>=0.16`, and `sglang[diffusion]==0.5.19`, described in its header comment as the pinned upstream SGLang diffusion dependency. The same comment sequence ends with `pip install flash-attn==2.7.4.post1 --no-build-isolation`, a third step outside both files, so the documented setup is three commands rather than one.
The Python ceiling comes from a wheel, not from the language
`requires-python = ">=3.12,<3.14"`, and the comment above it gives the whole reason in two clauses: the floor follows upstream at 3.12, and the cap exists because st_attn and vsa, which arrive through `sglang[diffusion]`, ship cp310 to cp313 wheels only with no sdist fallback. So the framework's supported Python range is set by the wheel matrix of an attention kernel pulled in by a rollout engine, not by anything in the framework's own code. That has a practical consequence. When a new Python lands, the framework cannot widen its range until the engine publishes a wheel for it, and the version of Python you can use is a function of which rollout backend you intend to use. It is also the clearest sign that the engine, not the framework, is the constraint that sets the other pins in the same file.
transformers is pinned to whatever SGLang ships
`transformers>=5.12,<5.13`, with the comment that it matches SGLang 0.5.19's exact 5.12.1 baseline, and that the patch-range form is kept so the engine-neutral model code can receive compatible 5.12.x fixes. Read together with the missing torch, the dependency strategy is consistent: the engine picks the CUDA stack, and the framework's own model code is written against whatever that engine already installed. Two more comments spell out what breaks when the two drift. peft has a floor of 0.20 because older peft imports transformers cache symbols, HybridCache among them, that were removed in 5.x and fail at import against the pin above. That same comment notes that exclude_modules support, available from 0.14, is what LoRA and NFT adapter injection use, which is why the floor sits above the floor that feature would suggest.
diffusers 0.38 was raised for one model and rechecked for all
The comment on `diffusers>=0.38.0` is the longest in the file and it is a maintenance log. LTX-2 needs 0.38 because `transformer.forward` gained sigma, audio_sigma, isolate_modalities, and use_cross_timestep, the connector gained padding_side, and calculate_shift now pins image_seq_len to max_image_seq_len. A floor of 0.37 was already required for Qwen-Image, whose RoPE derives the text length from the mask. The last clause says 0.38 was verified to keep that behaviour and every other model API unchanged. So a bump made for one row of the model table was then re-validated against the rest of it. The same pattern appears on einops: `einops>=0.7` is a dependency because a vendored Boogu-Image DiT under `unirl/models/boogu_image/vendor/` imports it in transformer_boogu.py, attention_processor.py, and rope.py, and the comment notes it is pure Python and engine-agnostic.
A maintainer note about an unmerged backend sits in the rendered page
Above the model table there is an HTML comment, so it is present in the page source but not in the rendered table, and it is a note to a maintainer rather than to a reader. It says MiniMax-H3 is trainside-only until the vLLM-Omni rollout backend, issues 378 and 420, lands, and it says to update its row here and in `unirl/models/README.md` when it merges. That comment records which side of a dependency the project is waiting on. The Status column of the table below it carries four different values across roughly twenty-one rows: a plain check mark, a check mark with trainside only, a check mark with sglang only, and a puzzle piece for bundle-only. Six rows are marked trainside only, including WAN 2.2 V2V, LTX-2.3, SenseNova-U1.5, and Janus-Pro, and Qwen3-MoE with the VeOmni expert-parallel setup is the only bundle-only row.
Recipes carry a status suffix and launch scripts come in two shapes
Most recipe filenames end in `_trainside`, from `sd3_trainside` and `qwen_image_trainside` through `hunyuan_video10_t2v_trainside` and `ltx2_3_t2av_trainside`. The suffix is a status marker rather than a definition: the table explains the dimension as model and algorithm support being independent and composing within a domain, so any diffusion algorithm runs on a diffusion model and AR algorithms run on AR models, and the shipped recipes are only a sample of the combinations. Alongside training-side rollout, the framework documents async AR and diffusion entrypoints that overlap rollout with training on separate GPU slabs. The example tree carries two launch scripts, `run_experiment_single_node.sh` and `run_experiment_multinode.sh`, alongside `mooncake_master.sh`, and its subdirectories are ar, deep_research, diffusion, pe, sft, and unified_model. Two of those, deep_research and pe, have no row in the visible part of the model table.
The version is dynamic, there are no releases, and the licence field is empty
pyproject.toml sets `dynamic = ["version"]`, so the file that installs the package carries no version string to read. There are no GitHub releases either, which leaves the arXiv identifiers in the news list as the only dated markers: DRPO, Flow-DPPO, and CPPO, all three listed under the same month in 2026, with three separate arXiv numbers. The licence is the other loose end. The metadata carries no licence value at all, pyproject.toml declares `license = { file = "LICENSE" }`, and a `LICENSE` file is present at the root. Those three statements are compatible but they are not the same statement, and which text governs is not stated. The repository also keeps its three team-proposed algorithm folders, CPPO, DRPO, and FlowDPPO, at the top level rather than inside the package, each holding a tutorial and a runnable recipe.
Editorial conclusion
UniRL is a serious framework for teams already running SGLang or a HuggingFace rollout on multi-GPU nodes, since the model table is backed by one runnable recipe per row and the loop itself is small enough to read. It is a poor fit if you want a lightweight install, because the documented path pulls a full inference stack plus a separately installed flash-attn, and because two dependency files describe the same requirements differently. Before adopting it, decide which file is authoritative in your environment, check whether the rows marked trainside only cover the models you actually need, and read the install notes for the OCR and Geneval extras before assuming the base environment is enough.
Frequently asked questions
What is UniRL and what does its single training loop do?
UniRL applies one reinforcement learning post-training loop across multimodal model families: generate samples, score them, compute advantages, update the policy, and sync weights back to rollout workers. A training entrypoint loads a Hydra example config and creates the matching domain trainer, and RL trainers coordinate generation, scoring, and updates across pluggable rollout engines, algorithms, model bundles, reward services, and a distributed runtime built on Ray DevicePool, FSDP, Transfer Queue, and LoRA or full-weight sync.
Which reinforcement learning algorithms does UniRL ship?
Three are proposed by the team and each folder holds a step-by-step tutorial and a runnable recipe: Flow-DPPO for diffusion and flow RL with an exact divergence-based trust-region mask, DRPO for token-level LLM RL with a smooth advantage-weighted quadratic regularizer, and CPPO with a position-weighted, cumulative-prefix-budget Binary-TV mask. The framework also wires in GRPO, DiffusionNFT, DanceGRPO, and MixGRPO as reference algorithms.
What does a trainside recipe mean in UniRL's model table?
It is a status marker rather than a definition. Across the roughly twenty-one visible rows the Status column carries four values: a plain check mark, a check mark with trainside only, a check mark with sglang only, and a puzzle piece for bundle-only. Six rows are marked trainside only, including WAN 2.2 V2V, LTX-2.3, SenseNova-U1.5, Janus-Pro, and MiniMax-H3, and Qwen3-MoE with VeOmni expert parallelism is the only bundle-only row. Async AR and diffusion entrypoints overlap rollout with training on separate GPU slabs.
How do I install UniRL?
requirements.txt documents a three step sequence: `pip install -r requirements.txt --no-build-isolation`, then `pip install -e . --no-deps --no-build-isolation`, then `pip install flash-attn==2.7.4.post1 --no-build-isolation`. The Geneval and OpenMMLab stack, meaning mmcv and mmdet, is intentionally excluded and installed separately when needed, following docs/geneval_mmcv_setup.md. The OCR reward's PaddlePaddle runtime stays in the `eval` extra, because only unirl.reward.local.ocr needs it.
What are the runtime and Python requirements for UniRL?
pyproject.toml requires Python >=3.12,<3.14. The comment above that line says the floor follows upstream and the cap exists because st_attn and vsa through sglang[diffusion] ship cp310 to cp313 wheels only, with no sdist fallback. torch is intentionally absent from the package dependencies and enters through exactly one engine extra, while requirements.txt pins torch>=2.1, torchvision>=0.16, sglang[diffusion]==0.5.19, and transformers>=5.12,<5.13.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/tencent-hunyuan-unirl)Star history chart for your README
[](https://hysenlabs.com/projects/tencent-hunyuan-unirl)Was this useful?