Reasoning Gym for procedural reasoning and verifiable rewards

Reasoning Gym targets verifiable training data

Reasoning Gym addresses a specific gap in reinforcement learning for reasoning models: training systems need a repeatable supply of tasks whose answers can be checked without asking another model to judge them. The README describes a community-created Python library of procedural dataset generators and algorithmically verifiable reasoning environments. Its stated goal is virtually infinite training data with adjustable complexity. That makes it relevant to researchers and engineers building reinforcement learning pipelines for large language models, especially where a reward function must be reproducible across generated examples.

The library covers more than 100 task types across areas including algebra, arithmetic, computation, cognition, geometry, graph theory, logic, and games. It is not limited to tasks with one canonical answer. The README specifically names Rubik's Cube and Countdown as examples with multiple correct solutions. For those environments, procedural verification is necessary because a model may produce a valid sequence or solution that differs from one stored answer. The standard dataset interface is therefore more important than a simple answer field alone.

The intended users are engineers who already have, or are assembling, a reinforcement learning training stack. Reasoning Gym supplies data generation and scoring mechanisms. The repository lists integrations and examples for Verifiers, TRL, Unsloth, veRL, and related training work, but the library does not present itself as a complete optimizer or distributed training system. The README says its data can be used with any major RL training framework, while also pointing to the `verifiers` library as the simplest documented starting point.

Procedural datasets use a common creation and scoring interface

The core flow is visible in the README's quickstart. `reasoning_gym.create_dataset()` receives a task name, a size, and an optional seed. It returns an iterable dataset whose entries contain a `question`, an `answer`, and task-specific `metadata`. The metadata carries information used by the procedural verifier. A caller can generate a batch, inspect its records, and invoke `data.score_answer()` to verify an answer against an entry.

The seed is important for reproducibility. In the documented example, `seed=42` makes the same generation request repeatable in principle, which helps with experiment tracking and comparisons between training configurations. Task-specific options are passed through the same call. The example configures `leg_counting` with `max_animals=20`, showing that complexity and content are controlled through keyword arguments rather than by editing generated examples.

For mixed training distributions, the `composite` dataset accepts `DatasetSpec` objects. Each specification names a task and supplies a weight and configuration dictionary. The documented example gives `leg_counting` a weight of 2 and `figlet_font` a weight of 1, with the latter configured by `min_word_len` and `max_word_len`. This supports curriculum or mixture design at the data layer. It does not, by itself, define how batches are scheduled during model training.

Verification is not limited to exact string equality. The README documents `cascade_score`, which applies progressively lenient fallback matchers for strings, numbers, and symbolic math. It is intended to reduce false negatives caused by formatting differences such as LaTeX wrappers, casing, and numeric representation. Symbolic math verification requires the optional `scoring` extra, which declares `math-verify>=0.7.0` in `pyproject.toml`.

Installation and first generated verification task

Reasoning Gym requires Python 3.10 or later. The documented installation from PyPI is:

pip install reasoning-gym

The package uses Hatchling as its build backend and declares the project name as `reasoning_gym` in `pyproject.toml`. The README warns that the PyPI version can lag behind `main`. If you need a specific release, pinning `reasoning-gym` in your environment is therefore more predictable than assuming that the package index always represents the current default branch. The repository version in `pyproject.toml` is `0.1.26.dev0`, while the listed releases include `v0.1.25`, so the source tree and published package can differ.

A first useful step is to generate ten `leg_counting` examples and verify each stored answer. This exercises creation, iteration, metadata inspection, and scoring rather than only printing a prompt:

import reasoning_gym

data = reasoning_gym.create_dataset('leg_counting', size=10, seed=42)
for i, x in enumerate(data):
    print(f'{i}: q="{x['question']}", a="{x['answer']}"')
    assert data.score_answer(answer=x['answer'], entry=x) == 1.0

The README shows entries such as a question about a sea slug and a deer with an answer of `4`, and another about two sheep and two dogs with an answer of `16`. The metadata should contain the generated animal counts and the computed total. The assertion demonstrates the intended reward path, although it checks the generator's own answer rather than testing an external model response.

To test formatting-tolerant scoring, install the optional extra and compare a symbolic-looking response with the expected value:

pip install reasoning-gym[scoring]

from reasoning_gym import cascade_score

assert cascade_score(answer=r"\text{42}", expected="42") == 1.0

This example verifies a documented use case. It does not establish how every task will interpret malformed, ambiguous, or adversarial answers.

Training integration separates generation from model updates

The repository documents several paths for connecting Reasoning Gym to reinforcement learning. For dynamic generation, the paper experiments use custom dataset code that creates samples at runtime and calls the Reasoning Gym scoring function to produce a training reward. This approach avoids materializing the entire dataset before training and fits pipelines that can create samples inside the rollout loop.

For a build-first workflow, the `scripts/hf_dataset/` directory contains a script that generates Reasoning Gym data and converts it to a Hugging Face dataset. The documented output columns are `question`, `answer`, and `metadata`. Task configurations are prepared in YAML, then passed to `save_hf_dataset.py` with the desired arguments. This creates a persistent dataset artifact, but the scoring function still expects an entry object and a model response. The generated row is not the same thing as a finished reward value.

The README provides a framework-neutral scoring pattern:

from reasoning_gym import get_score_answer_fn

for entry in dataset:
    model_response = generate_response(entry["question"])
    rg_score_fn = get_score_answer_fn(entry["metadata"]["source_dataset"])
    score = rg_score_fn(model_response, entry)

This code assumes that `dataset` and `generate_response()` come from the surrounding application. It shows the interface boundary clearly: Reasoning Gym receives the task entry and candidate response, while the application owns response generation and decides what to do with the returned score. The repository also lists `examples/verifiers` as a documented integration starting point. Other examples cover `trl`, `unsloth`, and `veRL`, so adoption can range from a small local experiment to a larger training stack.

This separation is a design benefit for reuse, but it creates integration work. The caller must preserve the metadata needed by the selected scorer, choose how scores enter the RL algorithm, and handle dataset generation throughput.

The API supports many tasks but not every scoring strategy

Reasoning Gym's breadth is also its main adoption constraint. More than 100 task types are available, but the repository does not claim that all tasks use the same difficulty model, answer format, or verification mechanism. Single-answer tasks can compare a computed result, while games and search problems may need to evaluate a sequence or solution state. The presence of procedural verifiers solves part of this issue, but it does not make every reward semantically identical.

The optional cascade scorer helps with representation differences. A model might return a number in a LaTeX wrapper, use different casing, or express the same numeric value in another notation. The documented scorer handles selected string, numeric, and symbolic cases. That is useful for reducing formatting false negatives, but it also creates a dependency on task-specific parsing and matcher behavior. The `scoring` extra is not installed by the default dependency set, so symbolic comparison requires an explicit environment choice.

The API also exposes task configuration through arbitrary-looking keyword arguments. That makes it possible to tune generators, but the caller must know the valid parameters for the selected task. `GALLERY.md` is the documented place to find available datasets and configurable parameters. Without checking that file and the relevant task implementation, a configuration value may produce a distribution unlike the intended one or may not be accepted by the task.

The library does not claim to solve reward hacking, data contamination, or the gap between procedural task success and general reasoning ability. Those concerns remain properties of the training setup and evaluation design.

Licensing, maintenance, and operational choices

The repository declares `Apache-2.0` in `pyproject.toml` and includes `LICENSE` and `NOTICE.txt` at the top level. The package metadata also classifies the project under the Apache Software License. That gives adopters a defined open-source licensing basis, but application owners still need to review the license text, preserve required notices where applicable, and account for dependencies and model terms in their own distribution. This review is an engineering and compliance step, not legal advice.

The repository is not archived. Its last push was on 2026-04-17. The release list includes `v0.1.25` on 2026-03-28, `v0.1.24` on 2025-09-29, and `v0.1.23` on 2025-07-05. The README says the project is under active development and that PyPI may lag behind `main`, but the last-push date is the relevant maintenance fact for this review. The version in the source tree is `0.1.26.dev0`, so developers should decide whether they need a published release or the default branch before depending on a behavior.

Optional dependency groups are clearly separated in `pyproject.toml`: `test`, `server`, `cli`, `scoring`, and `scripts`. A training project can avoid installing the server or script dependencies unless it needs them. The `rgc` console script is also declared, but the README does not provide a complete CLI tutorial, so its behavior should be confirmed in `tools.cli.rgc` before relying on it.

A real alternative is a fixed, pre-generated dataset. That approach is simpler when tasks do not need procedural variation and avoids runtime generation cost. Reasoning Gym is the better fit when changing complexity, mixing task types, or verifying many valid solutions is part of the experiment design. The trade-off is dependency on generator semantics and on the stability of the selected scorer across package revisions.

Editorial conclusion

Adopt Reasoning Gym when your RL pipeline needs configurable procedural tasks, deterministic seeds, and answer verification across both single-answer and solution-search environments. Do not treat it as a turnkey trainer or as a replacement for a framework that handles model rollout, optimization, checkpointing, and distributed execution. Before adoption, verify the task-specific configuration, scoring behavior, Python 3.10 compatibility, and the Apache-2.0 implications for your distribution.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. open-thought/reasoning-gym on GitHub
4. README
5. Releases