# UnifiedReward: reward models for multimodal generation

> UnifiedReward is the reference implementation of a family of reward models that score generated images and videos, covering the base model, a chain of thought variant, a personalised flex model and an editing model. The repository also carries the training and inference code, the preference data construction pipeline, and two evaluation benchmarks the same team maintains.

**CodeGoat24/UnifiedReward** — Official implementation of UnifiedReward & [NeurIPS 2025] UnifiedReward-Think & UnifiedReward-Flex

- Repository: https://github.com/CodeGoat24/UnifiedReward
- Website: https://codegoat24.github.io/UnifiedReward/
- Stars: 823 · Forks: 39
- Language: Python
- License: MIT
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/codegoat24-unifiedreward

## What the base model is

UnifiedReward is described as the official implementation of the UnifiedReward family, and the base model is what the rest of the family extends.

The README states the claim precisely: UnifiedReward is the first unified reward model for multimodal understanding and generation assessment, enabling both pairwise ranking and pointwise scoring.

The distinction between those two modes is the important part of the design. A pointwise model gives a single score for one output, which is what you need to filter a hundred samples and keep the best. A pairwise model decides which of two outputs is better, which is what you need when you have preferences rather than absolute judgements. Most reward models do one of the two, and a unified model that does both can be used in either pipeline without training two things.

The model family in the repository covers several lines. The model zoo table has columns for the model name, the version, the inference code path and a short description, and the versions are named after the base models they are built on: Qwen35, Qwen3-VL and Qwen25-VL appear repeatedly as the underlying checkpoints.

That naming is the practical detail to note before you plan anything. These are not standalone reward models that accept an image and a prompt; they are reward models with a specific language model backbone, and you need the matching checkpoint from the Hugging Face collections.

## Three lines of checkpoints and what each adds

Beyond the base model, the README documents three extensions, each with its own project page, its own Hugging Face checkpoint collection and its own inference directory in the repository.

UnifiedReward-Flex is described as a unified personalized reward model for vision generation that couples reward modeling with flexible and context-adaptive reasoning. Its checkpoints are published for the Flex collection, and the inference code lives in the UnifiedReward-Flex directory. The word personalized is the one to focus on, since it says the model can follow a user specific definition of what good looks like rather than only a general one.

UnifiedReward-Edit is a unified image generation and editing reward, and the README says it was trained on UnifiedReward-2.0. Its checkpoint collection covers three base model versions and its inference code has a directory of its own.

UnifiedReward-2.0 is the base line that the others build on, with checkpoint collections for the Qwen35, Qwen3-VL and Qwen25-VL families.

UnifiedReward-Think is the fourth line, and the one with a paper behind it. It is presented as the first unified multimodal chain of thought reward model, accepted at NeurIPS 2025, with a project page and its own checkpoint collection.

So the family covers four situations: scoring outputs, scoring outputs with explicit reasoning, scoring with a personalised notion of quality, and scoring the result of an edit rather than a generation.

## Why chain of thought matters for a reward model

The Think variant is worth separating out, because it changes what the model outputs.

The README describes it as a unified multimodal chain-of-thought reward model, obtained through reinforcement fine-tuning, and the project page is titled in the same terms.

A reward model that emits only a score is opaque. You get a number and no way to tell whether the model misunderstood the prompt, misread the image, or applied a preference that happens to be wrong. A chain of thought reward model produces the reasoning alongside the score, which means the rationale can be inspected, and, in a preference setting, it can be compared with the human annotation that produced the preference in the first place.

That is a real difference in how the model is used. A score is a number you filter on. A score with reasoning is something you can put in front of a human who disagrees with it.

The project page for Think also points at a checkpoint collection named for the Qwen3-VL models, which tells you the backbone. The reward models are hosted on the Hugging Face hub rather than in the repository, so the code here and the weights there have to be matched, and the README separates the two by linking a checkpoint collection next to each inference code path.

## The two benchmarks the team maintains

The same team publishes evaluation benchmarks alongside the reward models, and the README lists them in a benchmarks section with project pages, GitHub repositories, dataset spaces and leaderboard spaces.

UniGenBench++ is described as a unified semantic evaluation benchmark for text to image generation. It supports short and long prompts in Chinese and English, with a streamlined evaluation pipeline and a robust offline evaluation model. The offline model is published separately as a checkpoint, and the leaderboards are split into four spaces by language and prompt length, English and Chinese with short and long prompts.

UniREditBench is a unified reasoning-based image editing benchmark, presented at ECCV 2026. Alongside it the team constructs UniREdit-Data-100K, a large-scale synthetic dataset with high-quality chain of thought annotations, and develops UniREdit-Bagel by fine-tuning Bagel on that dataset.

The reason both belong in the same repository is worth noting. A reward model is only as good as the evaluation that measures it, and the benchmarks here cover exactly the two axes the reward family addresses: semantic quality of a generated image, and quality of an edit. The four leaderboard splits for language and prompt length also hint at the point that semantic evaluation is not a single number, since a short English prompt and a long Chinese one fail in different ways.

## A research monorepo rather than a package

The top level of the repository shows how a research codebase is arranged, and the directory names are the map.

The model lines have their own directories: UnifiedReward-Flex, UnifiedReward-Edit and UnifiedReward-Think. The training side has DiffusionDPO and VideoDPO, a preference_data_construction directory for building the preference data the reward models are trained on, and a train.sh script at the root. The inference side has separate implementations for the LLaVA-based models and the Qwen-based models, plus sglang_llava and vllm_qwen directories, which are the serving paths for the two inference stacks.

The supporting directories are modules for shared code, dataset for the data pipeline, benchmark_evaluation for the evaluation harness, scripts for utilities, docs for the documentation site, and trl and llava, which are vendored copies of the two upstream projects the work is built on.

The duplication in the inference directories is the interesting part. Two model families, two serving stacks, two sets of code paths, and the model zoo table exists precisely to tell you which combination to use for a given checkpoint. The table's inference code path column points into those directories, so the table is the routing you need before you touch anything.

That structure is fine for a research release and would be wrong for a library, and it is the main thing to understand before adopting anything from here.

## The packaging metadata is inherited from a fork

There is a detail in the packaging file that will confuse anyone who tries to install this project, and it is worth knowing before you hit it.

The build section declares setuptools as the build requirement with the setuptools build backend, and the project metadata does not describe a reward model at all.

```toml
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "llava"
version = "1.7.0.dev0"
description = "LLaVA OneVision: The Next Generation of LLaVA with Better Image and Video Understanding Capabilities"
readme = "README.md"
requires-python = ">=3.8"
```

The project is named llava, the version is a LLaVA OneVision development version, and the description is the LLaVA OneVision description. The optional dependency groups follow the same pattern, with a standalone group and a train group that pins torch 2.1.2, torchvision 0.16.2, a transformers commit from the GitHub repository, deepspeed 0.14.4, peft 0.4.0, accelerate 0.28.0, bitsandbytes 0.41.0, datasets 2.16.1, pydantic 1.10.8 and urllib3 at 2.0.0 or lower.

So the repository is built on a vendored LLaVA fork and inherits its packaging metadata. The practical consequence is that installing this project does not give you a reward model package, and that the training environment is pinned to a mid 2024 generation of the ecosystem. If you need current torch or transformers behaviour, you will be changing those pins yourself, and the README does not document a supported environment for the reward training code.

## Licence, maintenance and what to check first

The project is MIT licensed, with the licence file at the root of the repository, and the weights are published on the Hugging Face hub under the terms set there.

On maintenance, the last push was on 2026-06-18 and the repository has no GitHub releases, which is normal for a research release where the artefacts are checkpoints and datasets rather than a binary. The project page for the base model is at codegoat24.github.io, with separate sub-pages for the Flex, Think and Pref-GRPO work, and the Pref-GRPO entry describes a pairwise preference reward based GRPO method for stable text to image reinforcement learning, with UniGenBench presented alongside it.

Four checks before you build on this.

Which base model your checkpoint needs, since the collections are published per Qwen variant and the reward model inherits that backbone's context handling. Whether the environment the training code expects still installs, given the pins described above. Whether the licence on the weights permits your use, which is separate from the MIT licence on the code. And what the evaluation will cost you, because the benchmarks are described as having a streamlined pipeline and an offline evaluation model, and a reward model that is itself scored by another large model is a dependency worth pricing before you build a pipeline on it.

## Conclusion

UnifiedReward is the right reference point for a team building a text to image or image to video pipeline that needs a reward signal for filtering or for reinforcement learning, since it covers the base reward model, the chain of thought variant and the flex model for personalised preference in one family, with the training and inference code in the same repository. It is the wrong choice if you want a turnkey reward API, because the project ships research code with pinned training dependencies rather than a supported package, and the wrong choice for a production serving stack. Verify first which checkpoint collection matches your base model, since each model line is published against a specific Qwen variant, and check the pinned dependency versions in the packaging metadata before you build, because they are older than current releases of torch and transformers.

## FAQ

### What is UnifiedReward?

It is a unified reward model for multimodal understanding and generation assessment, enabling both pairwise ranking and pointwise scoring, with checkpoints and code for several model variants in the same family.

### What does UnifiedReward-Think add to the base model?

It is a unified multimodal chain of thought reward model, obtained through reinforcement fine-tuning and presented at NeurIPS 2025, so it produces the reasoning behind a score rather than only the score.

### Which benchmarks does the team publish alongside the reward models?

UniGenBench++, a unified semantic evaluation benchmark for text to image generation supporting short and long prompts in Chinese and English, and UniREditBench, a reasoning-based image editing benchmark presented at ECCV 2026 with the UniREdit-Data-100K dataset.

### Where are the model checkpoints hosted?

On the Hugging Face hub, in separate collections for each model line and each base model variant, with the model zoo table in the README giving the matching inference code path inside the repository.

## Sources

- [CodeGoat24/UnifiedReward on GitHub](https://github.com/CodeGoat24/UnifiedReward)
- [Issues](https://github.com/CodeGoat24/UnifiedReward/issues)
- [License: MIT](https://github.com/CodeGoat24/UnifiedReward/blob/main/LICENSE)
- [Project website](https://codegoat24.github.io/UnifiedReward/)
- [README](https://github.com/CodeGoat24/UnifiedReward/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/codegoat24-unifiedreward