Robo-Dopamine: a process reward model that scores robot progress step by step

Robo-Dopamine scores progress inside a trajectory, not success at the end

Most robot learning stacks read a task as done or not done. Robo-Dopamine is built around a different question: how far along is the robot right now? The reward model at the centre of the project, the General Reward Model (GRM), is a vision-language model that is prompted with a task description and conditioned on multi-view images of the initial state, the goal state, and the "BEFORE" and "AFTER" states of the current step. From that it predicts a relative progress score, or a regress hop, rather than a binary outcome.

That difference matters for precision work. A manipulation task that ends in a near-miss or a partial assembly gives a sparse reward no gradient to climb, while a per-step progress estimate gives the learner something to shape its behaviour around.

The project is the official implementation of "Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulation", accepted to CVPR 2026, with a paper on arXiv (2512.23703) and a project page at robo-dopamine.github.io. It is aimed at robotic manipulation researchers who need a progress signal they can adapt to a new task, not at end users of a finished robot stack.

Multi-Perspective Progress Fusion combines three ways of reading the same step

The GRM does not produce a single number on its own. Its output is fused through Multi-Perspective Progress Fusion, which combines incremental, forward-anchored and backward-anchored predictions into a final fused reward.

The three perspectives are complementary. An incremental reading judges the change between the BEFORE and AFTER images of one step. A forward-anchored reading compares the current state against the goal. A backward-anchored reading looks back toward the initial state to judge how much of the distance has been covered. Fusing them means a step can be recognised as progress even when one frame pair alone is ambiguous, which is the usual situation in manipulation where a gripper either has the part or does not.

The conditioning set explains why the images are multi-view: the model sees the initial state, the goal, and the before and after frames from several cameras at once, along with the task description as text. The two preview models add a separate ingredient, ST Modeling, and the project states they work with single-view or multi-view input, with or without reference target images, which is what lets the same checkpoint be pointed at data that has no goal image captured at all.

One-Shot GRM Adaptation then turns the score into a reward the policy can use

A general reward model is only half of the system, because a pre-trained scorer does not know your robot or your task. The Dopamine-RL framework handles that half in two steps.

The first is One-Shot GRM Adaptation: the pre-trained GRM is adapted to a novel task using a single demonstration. One demonstration is a demanding requirement to state up front, and it is where most of the friction with this project will live. There is no script or robot data format in the documentation that describes what a demonstration has to contain beyond the fact that a single one is enough.

The second step is Policy-Invariant Reward Shaping, which converts the GRM's dense output into a reward signal for the learner. The stated property is that the shaping accelerates learning without altering the optimal policy, so the reward change is supposed to be a reparameterisation of the objective rather than a new objective with different behaviour.

Because the shaping step is separate from the reward model, the framework is described as universally compatible with a wide range of RL algorithms. That is the practical claim to test first: it says the method does not require a specific learner, which is a different and more useful promise than a framework tuned for one algorithm.

Setting up the environment: a pinned dependency list and cuda >=12.8

The setup is a conda environment with Python 3.10 and a requirements file whose entries are pinned to exact versions, including the CUDA 12 wheels for the NVIDIA runtime libraries. The project states that cuda >=12.8 is required. The commands, as given in the setup section, are:

# clone repo.
git clone https://github.com/FlagOpen/Robo-Dopamine.git
cd Robo-Dopamine

# build conda env., and require `cuda >=12.8`
conda create -n robo-dopamine python=3.10
conda activate robo-dopamine
pip install -r requirements.txt

After that, `pip install -r requirements.txt` pulls in the pinned stack: accelerate==1.12.0, datasets==4.1.1, huggingface-hub==0.35.3, numpy==2.2.6, matplotlib==3.10.7 and cupy-cuda12x==13.6.0 among the listed packages. The tree also shows where the work happens: dataset/ for the data pipeline, train/ for training and finetuning, eval/ for evaluation, examples/ for runnable samples, and assets/ for the figures used in the README.

Under examples/ you find inference.py, blank_goal.png, and the demo_table and more_demos directories. That layout implies several demonstration arrangements are covered: a run with no goal image, a table of demos, and a set of additional demos for the few-shot adaptation path.

Four checkpoints, and the two 2.0 previews add ST Modeling

The model zoo lists four GRM checkpoints on Hugging Face under the tanhuajie2001 account.

GRM-3B is a full-trained GRM built from RoboBrain-2.0-3B, and GRM-8B is the same idea from RoboBrain-2.0-8B. Both are described as full-trained rather than preview models.

The GRM-2.0 line is different in kind. GRM-2.0-8B-Preview is the version the project points at for general use because it carries ST Modeling and supports single-view or multi-view cases, with or without reference target images. GRM-2.0-4B-Preview is the lightweight entry point, and the project quantifies its cost as approximately 80 percent accuracy relative to the 8B preview, which makes it a reasonable default when a 4B model fits the hardware you have.

Two release tags exist for the code, v1.0-beta on 2026-03-08 and v2.0-stable on 2026-04-16. There are no GitHub releases beyond those tags, and the weights live on Hugging Face rather than in the repository, so the code version and the checkpoint version can drift apart.

The benchmark, the judge mode and the 35M example dataset

Robo-Dopamine ships three supporting assets beyond the model itself.

The first is Robo-Dopamine-Bench, released with evaluation codes on 2026-01-26. The second is a judge mode, in which Robo-Dopamine-GRM is used to evaluate existing VLAs on the RoboChallenge benchmark across multiple dimensions and metrics, with a leaderboard and a blog for the results. Judge mode is the alternative use to training mode: instead of shaping a reward for a policy you are training, the same GRM ranks policies that already exist, which needs no demonstration and no reward shaping step.

The third is the data side. The data generation pipeline and the finetune codes were released on 2026-02-10 with the note to try finetuning with your own data, and the full Robo-Dopamine-GRM-Dataset followed on 2026-05-13 as a 35M example training dataset for the GRM series.

The order of those releases matters for planning. The benchmark came first, then the training pipeline, then the pretraining dataset, so a team arriving now can evaluate a GRM, generate its own data, and only then decide whether to pretrain at scale.

Where the documentation stops, and what that costs you

Two gaps are visible. The README's usage section is cut off mid-sentence after it introduces the three inference modes, Incremental-Mode, Forward-Mode and Backward-Mode, so the actual invocation lives in examples/inference.py rather than in the documentation. Anyone planning to integrate the GRM into an existing policy loop will be reading that script for the interface.

The second gap is numerical. The documentation gives no latency, throughput or memory figure for any checkpoint, and no accuracy number beyond the relative 80 percent figure between the 4B and 8B previews. Anyone budgeting inference cost for a manipulation loop has to measure it themselves.

There is also a dependency cost to weigh. Pinning exact versions for the whole stack, including CUDA 12 packages, keeps a working environment reproducible but makes it hard to share with a team whose CUDA driver or PyTorch stack differs, and the stated cuda >=12.8 floor rules out older installations outright.

Maintenance is the quieter point. The last push was on 2026-05-13, and the last tag, v2.0-stable, is dated 2026-04-16. The repository is not archived, and the release list shows a steady sequence of model, dataset and benchmark drops through the spring, but the most recent recorded activity is the dataset release rather than a code change. The code is licensed Apache-2.0, which permits commercial use and modification with attribution and notice of changes, and the model weights and dataset are distributed separately on Hugging Face under their own terms.

Editorial conclusion

Robo-Dopamine fits research teams that need a dense progress signal for high-precision manipulation and want to train or evaluate with the released data generation pipeline, finetune codes and Robo-Dopamine-Bench. It is the wrong choice for a small team that needs a dependency set it can upgrade freely, because requirements.txt pins exact versions and the environment demands cuda >=12.8. Verify first that your own demonstration format matches what the One-Shot GRM Adaptation path expects, then check the GRM checkpoint you plan to use against Robo-Dopamine-Bench before wiring it into a policy loop.

Frequently asked questions

Official sources

1. FlagOpen/Robo-Dopamine on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases