Remember-R1: a GRPO training recipe for long-context visual forgetting

The failure mode Remember-R1 targets: visual evidence that decays mid-reasoning

Long-context vision-language models do not usually fail because they never looked at the image. They fail because the image stops mattering. A model answers a question about a chart, then spends the next several hundred tokens reasoning from text it generated itself, and the final answer drifts away from what the pixels actually showed. The README frames this as "long-context visual forgetting" and the paper title uses the same phrase, so the target is specific: degradation of visual grounding as the reasoning trajectory lengthens.

The intended user is a researcher who already has a reasoning VLM and wants to change how it is trained, not an application developer who wants an API. Everything in the repository assumes you can supply a base model path, a JSONL training file and an image folder, and that you have the hardware to run distributed RL. There is no inference server, no CLI for asking questions and no hosted endpoint. The Hugging Face links point at checkpoints, and the repository itself is the training recipe that produced them.

Three process rewards layered on top of answer correctness

The mechanism is reward shaping inside a GRPO loop. The README states that during training Remember-R1 augments the answer correctness reward with three process-level rewards defined on the target reasoning trajectory. Each one targets a different way visual dependence can drop out:

Visual Vocabulary Reward pushes the model to state visual evidence explicitly rather than gesturing at it. Visual Memory Reward pushes visual dependence to persist into later reasoning steps, which is the direct countermeasure to forgetting. Visual Key-Region Reward pushes attention toward regions of the image that the question actually concerns.

The consequence of this design is that the reward is not a single scalar you can swap out casually. It is computed against a reasoning trajectory, so the training loop has to retain and inspect intermediate steps, and the balance among the four reward terms becomes a hyperparameter you will be tuning. The README does not publish the reward weights, so treat the released script as a skeleton and expect to search for coefficients that suit your data. The repository layout supports this reading: src/ holds the training code, run_scripts/ holds launchers, and dataset/ is an empty mount point where you drop your own JSONL and images.

Installing Remember-R1 and running a first training job

The README gives a two-stage install: clone, then create a Python 3.10 environment and install the pinned requirements plus the vendored open-r1-multimodal package in editable mode.

git clone https://github.com/Ch921-cell/Remember-R1.git
cd Remember-R1

conda create -n remember-r1 python=3.10 -y
conda activate remember-r1
pip install -r requirements.txt
pip install -e src/open-r1-multimodal

After the second block the environment should contain torch>=2.5.1, transformers==4.49.0, deepspeed==0.15.4 and the trl fork pinned to the main branch, all listed in requirements.txt. The README warns that the default launcher uses DeepSpeed ZeRO-3 and FlashAttention 2 and that you should install versions compatible with your CUDA and PyTorch setup. That is the first real obstacle: flash-attn is listed unpinned, so it compiles against whatever toolchain you have, and a mismatch surfaces as a build failure rather than a clear error message.

Data preparation is a download step, not a conversion step. The README says to download the training datasets from the dataset release page and place them in the dataset/ directory. Training then runs through environment variables and one script:

export MODEL_PATH="/path/to/model"
export DATA_PATHS="/path/to/train.jsonl"
export IMAGE_FOLDERS="/path/to/dataset"

bash run_scripts/run_grpo_remember.sh

The three variables are the entire documented configuration surface: base model, training file, image roots. Anything else you want to change lives inside the shell script, which the README does not annotate. Expect to open it and read the underlying Python before your first run produces anything meaningful.

Evaluation means standing up VLMEvalKit yourself

The repository does not ship an evaluation script. The README points to VLMEvalKit from open-compass and gives the install and invocation pattern:

git clone https://github.com/open-compass/VLMEvalKit.git
cd VLMEvalKit
pip install -e .

python run.py --model <model_name_or_path> --data <dataset_name>

You prepare the benchmark datasets according to the VLMEvalKit documentation, then configure the model checkpoint and evaluation settings inside VLMEvalKit and run it. Two things follow from this. First, the numbers in the paper are not reproducible from this repository alone, because the benchmark preparation lives in a different project with its own data requirements. Second, the README never names which benchmarks were used, so you cannot tell from the repository whether your own long-context visual test set is comparable to the paper's. If your goal is to check the claim on your data, that is fine. If your goal is to match a reported score, you are reconstructing the setup from the paper, not from the code.

Where Remember-R1 is the wrong tool

The clearest limitation is scope. This is a training repository for a method, released alongside a paper, with no releases retrieved and no versioned artifacts. There is no changelog, no compatibility matrix and no documented upgrade path. The dependency set makes that concrete: trl is pinned to a git URL at main, which means a fresh install today can pull different code than a fresh install did last month, and transformers==4.49.0 is pinned exactly while torch is a floor (>=2.5.1). Combining a floating git dependency with a hard version pin is a recipe for environments that work once and then stop working after an upstream commit.

Hardware is the second boundary. DeepSpeed ZeRO-3 plus FlashAttention 2 plus GRPO on a 3B or 7B vision-language model is multi-GPU work. A single consumer card will not run the default launcher in any useful configuration, and the README offers no single-GPU fallback. If you want to evaluate a long-context VLM rather than retrain one, downloading the 7B checkpoint and running it through VLMEvalKit is the sensible path, and Remember-R1's training code is irrelevant to you.

The third case is reward engineering you cannot inspect. Because the README does not document the weights or the exact computation of the three process rewards, you cannot reason about them from the README alone. You will be reading src/open-r1-multimodal to find out what the Visual Memory Reward actually measures. That is normal for research code, but it is worth stating plainly rather than treating the README as a specification.

VLM-R1 and VAPO: what changes if you use them instead

The acknowledgements name VLM-R1 and VAPO as upstream influences, and the difference in approach matters when you are choosing what to build on. VLM-R1 is the general reinforcement-learning framework for vision-language models that this repository is built around, and Remember-R1 vendors a copy of its multimodal training package under src/open-r1-multimodal. If you want a documented, reusable RL harness for VLMs, VLM-R1 is the broader base and Remember-R1 is a specialized configuration on top of it. Choosing Remember-R1 means accepting a narrower, paper-shaped codebase in exchange for the specific reward design.

VAPO is a different kind of alternative: it addresses value-based reinforcement learning for reasoning, and the acknowledgement suggests it informed the training approach here. The practical distinction is where the signal comes from. Remember-R1's contribution sits in process rewards computed over the reasoning trajectory, so its value depends entirely on whether trajectory-level supervision is the bottleneck in your model. If your problem is reward estimation or credit assignment rather than visual grounding decay, the three visual rewards will not address it, and you are better served by the underlying RL framework.

Licence and the cost of keeping this running

Remember-R1 is Apache-2.0, which permits commercial use, modification and redistribution provided you keep the licence and notices, and it includes an explicit patent grant. That is the permissive end of research code licences and it is a point in the project's favour. Note that the licence covers this repository; the model checkpoints and datasets on Hugging Face are separate artifacts with their own terms, and the README does not state what those are. Check them before you ship anything derived from the weights.

Maintenance is the harder question. The last push to the default branch was on 2026-08-28, and the news entries in the README date from 2026.08.29 back to 2026.07.10, so the repository is recent. There are no releases, which means there is no version to pin and no upgrade notes to read. Your upgrade cost is therefore the cost of re-resolving requirements.txt plus the trl git dependency, and the realistic mitigation is to freeze your own environment after the first successful build rather than reinstalling later. Budget for that, and for reading the launcher script before you trust it, because the README documents three environment variables and nothing else.

Editorial conclusion

Adopt Remember-R1 if you already run GRPO or PPO style RL on vision-language models and want the three process rewards as a starting point for your own reward shaping; the run_scripts/run_grpo_remember.sh launcher and the released 3B and 7B checkpoints are the parts worth reading first. Do not adopt it if you need a supported library, a documented evaluation pipeline, or a way to reproduce the paper's benchmark numbers without assembling VLMEvalKit yourself, because the repository ships training code and data pointers, not a turnkey harness. Before committing compute, verify three things: that your CUDA and PyTorch build can compile flash-attn and deepspeed==0.15.4, that the dataset release page contains the JSONL files your run script expects, and that you have a long-context visual benchmark of your own, since the README does not name the evaluation datasets used in the paper.

Frequently asked questions

Official sources

1. Ch921-cell/Remember-R1 on GitHub
2. Issues
3. License: Apache-2.0
4. README