Search and Learn is a set of recipes for spending more compute on inference

Recipes, not a framework

Search and Learn is described as recipes to scale inference-time compute of open models, and the readme is explicit about the shape of the project. It is simple by design and mostly consists of two directories.

The scripts directory holds the code that scales test-time compute for open models. The recipes directory holds the configurations for applying different search algorithms at test time, and it also contains the accelerate configs and the slurm scripts.

The key design decision is that each recipe takes the form of a YAML file which contains all the parameters associated with a single inference run. So an experiment is a file, not a script, and a set of experiments is a directory of files that can be reviewed, diffed and version controlled.

Three search algorithms are currently supported, Best-of-N, beam search, and Diverse Verifier Tree Search. The repository is from the Hugging Face team, the licence is Apache-2.0, and the news section has one entry, an initial release on December 16, 2024 with the code to replicate the test-time compute results of the linked blog post. The last push was on 2026-09-23 and there are no GitHub releases, so the repository is a snapshot of the research code rather than a versioned tool.

The name comes from Rich Sutton's Bitter Lesson, and the readme quotes the part about general purpose methods that continue to scale with increased computation, and about search and learning being two such methods.

The problem it takes on: test-time compute

The What is this section gives the motivation, and it is the clearest statement of why a repository like this exists.

Over the last few years, the scaling of train-time compute has dominated the progress of LLMs. That paradigm has proven remarkably effective, but the resources needed to pretrain ever larger models are becoming prohibitively expensive, with billion-dollar clusters already on the horizon.

That has sparked interest in a complementary approach, test-time compute scaling. Rather than relying on ever larger pretraining budgets, test-time methods use dynamic inference strategies that allow models to think longer on harder problems. The prominent example given is OpenAI's o1 model, which shows consistent improvement on difficult math and coding problems as the amount of test-time compute increases.

The gap the project names is the important part. Although we don't know how o1 was trained, Search and Learn aims to fill that gap by providing the community with a series of recipes that enable open models to solve complex problems if you give them enough time to think.

So the deliverable is a recipe book with code, aimed at a specific open research question, and the blog post and the collection of models and datasets on the Hugging Face Hub are the results it points at.

Search against verifiers, scored step by step

The Contents section lists what the initial release focuses on, and there are two techniques.

The first is search against verifiers. The idea is to guide LLMs to search for solutions to verifiable problems, which the readme names as math and code, by using a stepwise or process reward model to score each step. The techniques included are Best-of-N sampling and tree search.

The second is training process reward models, which are reward models that provide a sequence of scores, one for each step of the reasoning process. The readme explains why that matters: this ability to provide fine-grained feedback makes PRMs a natural fit for search methods with LLMs.

Put together, the two are one system. A verifier tells you whether a finished answer is right, which is enough for Best-of-N, since you can sample N answers and keep the best. A process reward model tells you whether a step in the middle is on track, which is what makes tree search possible, because a search that can only score leaves cannot prune a branch early.

The three supported algorithms correspond to that distinction. Best-of-N is sampling and scoring, beam search keeps the best partial paths, and Diverse Verifier Tree Search is the tree variant, and the recipe YAML is where you choose between them and set the budget for the run.

Setting up an environment, and logging in to the Hub

The installation instructions are four steps, and they assume you are going to push models back to the Hub as well as read from it.

First, create a Python virtual environment. The example uses Conda with Python 3.11.

conda create -n sal python=3.11 && conda activate sal

Second, install the project in editable mode with the dev extra.

pip install -e '.[dev]'

Third, log into your Hugging Face account, because the models and datasets used in the recipes are hosted on the Hub.

huggingface-cli login

Fourth, install Git LFS so that you can push models to the Hugging Face Hub.

sudo apt-get install git-lfs

After that, the readme points you at the scripts and recipes directories for the instructions specific to each of them, and the recipes README is where the launch commands and config files live if your goal is to replicate the published results.

The order matters more than it looks. The dev extra pulls in a specific vLLM version, and vLLM and the GPUs it wants are the reason the environment setup is the first thing the documentation does rather than an afterthought.

The dependency list is the architecture

The setup file is short and it explains what the project actually runs on.

install_requires = [
    "accelerate",
    "pebble",  # for parallel processing
    "latex2sympy2==1.9.1",  # for MATH answer parsing
    "word2number",  # for MATH answer parsing
    "transformers>=4.47.0",
    "fastapi",
    "hf_transfer",
]

The comments are the interesting part, because they name the two hardest parts of the system. pebble is for parallel processing, which is how a tree search fans out. latex2sympy2 and word2number are for MATH answer parsing, which is the verifier for the math half of the work, since a model answers a maths problem with a number and comparing two answers means normalising both into something comparable.

That is the piece most demo code gets wrong. If the answer parser disagrees with the model about what counts as the same answer, the verifier is wrong, and a wrong verifier turns a search into a random walk.

transformers is the model layer, accelerate is the device and distribution layer, fastapi suggests a served inference path, and hf_transfer is the fast downloader for the model weights.

The extras show the same layering. The dev extra pins vllm at 0.6.3 and adds the quality and tests extras, which are ruff and isort for linting and pytest for tests. There is a separate trl extra, installed from the GitHub repository of the Hugging Face TRL project rather than from PyPI, which is the training side of the process reward model work. The package itself is version 0.1.0 and is classified as Development Status 3 - Alpha.

The Makefile exists to make sure you test the local code

The Makefile is two targets and a short comment, and the comment is the important part.

It says to make sure to test the local checkout in scripts and not the pre-installed one, and it does not use quotes. The way it does that is to export PYTHONPATH as src, so the package in the working tree shadows whatever version is installed in the environment.

That is a small thing that catches a genuinely annoying class of bug. The project is installed with pip install -e, so an editable install normally points at the source, but the scripts directory imports the package as a module and can pick up a different copy depending on how the interpreter is invoked, and the symptom is a change you made having no effect on the run. Exporting the path makes the behaviour the same in every case.

The check directories are src, scripts and tests, and the style target runs ruff with the import sorting rule selected and the fix flag, then ruff format over the same directories and setup.py. The quality target runs the same ruff check without the fix, which is what a CI job would run.

The project structure in the readme matches the tree. There is a recipes directory for the recipe configs, accelerate configs and slurm scripts, a scripts directory, a src directory for the source code, a tests directory for the unit tests, a Makefile with commands like make style, a pyproject.toml used mostly for configuring code quality and tests, and a setup.py that makes the project pip installable so that sal can be imported.

How to cite it, and what it is based on

The Citation section asks for two references, and the second one is the more important.

If you find the content of the repository useful, cite the work as Scaling test-time compute with open models, by Edward Beeching, Lewis Tunstall and Sasha Rush, with the URL of the blog post.

@misc{beeching2024scalingtesttimecompute,
      title={Scaling test-time compute with open models},
      author={Edward Beeching and Lewis Tunstall and Sasha Rush},
      url={https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling-test-time-compute},
}

The readme also asks you to cite the original work by DeepMind upon which the repository is based, Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters, by Charlie Snell, Jaehoon Lee, Kelvin Xu and Aviral Kumar, from 2024, on arXiv.

That distinction is worth keeping straight if you write about the results. The Hugging Face work applies the idea to open models and makes it reproducible; the Snell and Lee paper is the original result. A recipe that reproduces the scaling behaviour is evidence for the method, not a new claim about it.

The repository is Apache-2.0, the setup file carries the standard Hugging Face copyright header, and the setup file names the package search-and-learn, importable as sal.

Editorial conclusion

Adopt search-and-learn if you are reproducing the test-time compute results or want a working reference implementation of search against verifiers with a process reward model, and you have the GPU time to run it. Do not adopt it as a production inference service, since the package is marked Development Status 3 - Alpha and the recipes are tied to the models and datasets of a specific study. Verify first which recipe and model pairing you are using, that the verifier you point at is a process reward model rather than an outcome model, and that your environment has the pinned vLLM version the dev extra installs.

Frequently asked questions

Official sources

1. huggingface/search-and-learn on GitHub
2. Issues
3. License: Apache-2.0
4. README