Model or dataset
sapientinc/HRM avatar
sapientinc/HRM

sapientinc/HRM: a 27M parameter model that trains Sudoku and ARC from scratch

Hierarchical Reasoning Model Official Release

12,667 stars1,807 forksPythonApache-2.0

At a glance

What is it?
The Hierarchical Reasoning Model is a research release, not a general model: 27 million parameters, two recurrent modules, one training run per puzzle domain, and 8-GPU launch commands that expect a specific CUDA toolchain.
Who is it for?
Use this repository if you want to reproduce the small-sample grid puzzle numbers on hardware you control, and do not treat it as a general reasoning system or a text model. The ARC-AGI path is the only one documented end to end, from dataset builder to evaluate.py to a notebook.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Activity is slowing. The repository last received commits 7 months ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 13, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • sapientinc/HRM12,667 ★ · +30
Star history
Repository2026-09-132026-11-13
sapientinc/HRM12.6k12.7k

30 Stars in the last 61 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

Two interdependent recurrent loops inside 27 million parameters

Hierarchical Reasoning Model is a single recurrent network split into two interdependent modules: a high-level module that runs slow abstract planning and a low-level module that handles rapid detailed computation, the split the project borrows from hierarchical, multi-timescale processing it attributes to the human brain. Sequential reasoning happens inside one forward pass, and the intermediate steps carry no supervision of their own, so there is nothing to label except the final answer. The whole model is 27 million parameters, trains on 1000 samples, and starts from random weights: no pre-training stage and no chain of thought traces to collect.

That budget only makes sense for grid shaped problems, and the file layout states it plainly. pretrain.py, evaluate.py, puzzle_dataset.py, a dataset/ directory of builder scripts, plus models/, config/ and utils/ describe a pipeline that turns a puzzle directory into tensors and back into an exact match rate. There is no text checkpoint, no tokenizer and no generation entry point, so a reader expecting a language model will find a puzzle trainer. The paper sits at https://arxiv.org/abs/2506.21734 and the authors point readers to a Discord at https://discord.gg/sapient.

torchrun is pinned to eight ranks in every full scale command

Full scale runs are written for an 8-GPU node, and each one is the same torchrun invocation with per task values bolted on. ARC-1 is OMP_NUM_THREADS=8 torchrun --nproc-per-node 8 pretrain.py at a stated runtime of about 24 hours. ARC-2 adds data_path=data/arc-2-aug-1000, and the note that a checkpoint after 8 hours is often sufficient. The 1000-sample Sudoku run finishes in about 10 minutes and the 30x30 maze run in about 1 hour.

Sudoku-Hard is where the schedule spreads across a long line of overrides, and those keys reach into the model rather than the trainer:

bash
OMP_NUM_THREADS=8 torchrun --nproc-per-node 8 pretrain.py data_path=data/sudoku-hard-full epochs=100 eval_interval=10 lr_min_ratio=0.1 global_batch_size=2304 lr=3e-4 puzzle_emb_lr=3e-4 weight_decay=0.1 puzzle_emb_weight_decay=0.1 arch.loss.loss_type=softmax_cross_entropy arch.L_cycles=8 arch.halt_max_steps=8 arch.pos_encodings=learned

L_cycles, halt_max_steps, pos_encodings and loss_type point at a cycle count, a step cap, a choice of position encoding and an objective, and only the last is spelled out in its value. The repository explains none of them, so learning their effect means reading models/ and config/ and running the comparison. On a single GPU you also have to drop --nproc-per-node 8 and rescale global_batch_size yourself, with no documented formula for doing it.

data_path has to match a directory a builder script already wrote

Data is a separate build step, and the trainer only accepts a directory that already exists. Sudoku comes from python dataset/build_sudoku_dataset.py --output-dir data/sudoku-extreme-1k-aug-1000 --subsample-size 1000 --num-aug 1000, and the same script with no arguments produces the full version. ARC goes through build_arc_dataset.py, where the bare call covers ARC official plus ConceptARC for 960 examples, and the ARC-2 call passes --dataset-dirs dataset/raw-data/ARC-AGI-2/data --output-dir data/arc-2-aug-1000 for 1120. Mazes come from build_maze_dataset.py, 1000 examples.

The raw ARC-2 data arrives as a submodule, so full scale preparation opens with git submodule update --init --recursive, a step the single GPU Sudoku demo never mentions. Opening puzzle_visualizer.html in a browser and uploading the data/... folder is the fastest way to catch a builder that wrote the wrong augmentation count. The loose end is ARC-1: its launch line carries no data_path argument at all, so it depends on a default path the project never states.

requirements.txt names ten packages and not one version

requirements.txt lists torch, adam-atan2, einops, tqdm, coolname, pydantic, argdantic, wandb, omegaconf, hydra-core and huggingface_hub with no constraints on any of them. The optimizer choice, the config stack and the dataset class all resolve to whatever is newest in the environment, and no environment is recorded as the one this code was written against. Build tooling is a separate concern from runtime dependencies: packaging, ninja, wheel, setuptools and setuptools-scm are installed on their own because the repository compiles CUDA extensions rather than running pure Python.

FlashAttention is a third path, chosen by GPU generation. On Hopper you clone [email protected]:Dao-AILab/flash-attention.git, cd into flash-attention/hopper and run python setup.py install; on Ampere or earlier it is one pip3 install flash-attn. The Hopper route is a source build against your local toolchain, and nothing pins a commit or a CUDA version for it. Two small inconsistencies make a wrong build easy: the prerequisite block uses pip3 throughout while the dependency step uses pip, and the FlashAttention 2 route is a bare wheel while the Hopper route is a clone. When torch, CUDA and flash-attn disagree, the symptom is a compile error inside the extension, not a message naming the mismatch.

Sudoku at 1000 samples destabilizes near the end of training

The notes section admits two limits on the headline results. Small sample runs carry accuracy variance of around plus or minus 2 points, so a single run measures little, and any comparison between two configurations needs more than one seed. On the 1000-example Sudoku-Extreme dataset, late stage overfitting can cause numerical instability during training and during Q-learning, and the recommended response is to stop early once training accuracy approaches 100 percent. A curve that keeps climbing can therefore be a worse artifact than one that stops.

Two things follow. First, eval/exact_accuracy in W&B is the number that matters, and it is read off a run you chose to truncate. Second, the instability note refers to Q-learning while the architecture description names two recurrent modules, and nothing in the repository connects the two, so you cannot tell from the documentation which training path that warning applies to. The ARC-2 line that a checkpoint after 8 hours is often sufficient out of roughly 24 hours says early stopping is the expected mode here. No documented command includes a best-checkpoint flag.

Three checkpoints, no releases, and uneven evaluation coverage

Three trained checkpoints are published under the sapientinc account: ARC-AGI-2, Sudoku 9x9 Extreme on 1000 examples, and Maze 30x30 Hard on 1000 examples. The repository has no GitHub releases, so there is no tag, no version string and no changelog to pin against, and no revision is attached to the checkpoint links. A run made today and a run made six months from now have nothing recorded that would let you line them up.

Evaluation coverage is uneven too. ARC-AGI has a documented path, OMP_NUM_THREADS=8 torchrun --nproc-per-node 8 evaluate.py checkpoint=<CHECKPOINT_PATH> followed by the arc_eval.ipynb notebook. Sudoku and maze have no equivalent: the only documented output is the eval/exact_accuracy metric in W&B. For two of the three published models, loading and scoring locally means assembling the path yourself from pretrain.py and models/. Because wandb login comes before any training and the metric lives in W&B, every run depends on an account at wandb.ai before you can see a score, and a reader who needs an offline reproduction has to handle that dependency first. The Apache-2.0 license covers reuse, not the missing scaffolding.

The install recipe is frozen at CUDA 12.6 and the last push is 2026-03-31

Environment setup is a three step manual path pinned to one CUDA release. The installer is fetched for cuda_12.6.3_560.35.05_linux.run, installed with sudo sh cuda_installer.run --silent --toolkit --override, and CUDA_HOME is then set to /usr/local/cuda-12.6 before torch, torchvision and torchaudio come from the cu126 wheel index with pip3 install torch torchvision torchaudio --index-url $PYTORCH_INDEX_URL. The repository compiles its own CUDA extensions, so the toolchain has to be right before the first import.

None of it is version tolerant. There is no second CUDA path, no container definition, and no CI configuration among the top level entries, which are .gitignore, .gitmodules, .vscode/, LICENSE, README.md, arc_eval.ipynb, assets/, config/, dataset/, evaluate.py, models/, pretrain.py, puzzle_dataset.py, puzzle_visualizer.html, requirements.txt and utils/. The project is not archived, but the last commit landed on 2026-03-31 and nothing has been committed since, so read the recipe as a snapshot of one machine rather than a maintained matrix. On a newer driver, no document here says what breaks or whether FlashAttention 2 still applies, and that is where your own testing starts. The paper is at arxiv.org/abs/2506.21734 and the project homepage is sapient.inc.

Editorial conclusion

Use this repository if you want to reproduce the small-sample grid puzzle numbers on hardware you control, and do not treat it as a general reasoning system or a text model. The ARC-AGI path is the only one documented end to end, from dataset builder to evaluate.py to a notebook. Before you start, confirm your CUDA version matches the 12.6 recipe, decide whether a wandb.ai account is acceptable, and budget a checkpoint selection step the code does not do for you.

Frequently asked questions

What is the AI model HRM?

Hierarchical Reasoning Model is a 27 million parameter recurrent architecture with a high-level module for slow abstract planning and a low-level module for rapid detailed computation, trained from random weights on 1000 samples with no pre-training and no chain of thought data.

What is HRM in AI?

It reports nearly perfect performance on complex Sudoku puzzles and optimal path finding in large mazes, and on the Abstraction and Reasoning Corpus it outperforms much larger models with significantly longer context windows.

What is an HRM model?

In this repository an HRM model is a per task training run: separate commands exist for ARC-1, ARC-2, Sudoku Extreme and Maze 30x30, with published checkpoints for ARC-2, Sudoku Extreme and Maze 30x30, and the full scale launches assume an 8-GPU node.

Official sources

  1. Issues
  2. License: Apache-2.0
  3. Project website
  4. README
  5. sapientinc/HRM on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/sapientinc-hrm.svg)](https://hysenlabs.com/projects/sapientinc-hrm)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/sapientinc-hrm.svg)](https://hysenlabs.com/projects/sapientinc-hrm)

Was this useful?