Model or dataset
Human-Agent-Society/reef avatar
Human-Agent-Society/reef

Reef: continual learning infrastructure that versions agent updates like git commits

Continual learning infra for self-improving agents

7,827 stars737 forksPythonApache-2.0

At a glance

What is it?
Reef connects serving, feedback, training and artifact delivery into one loop, with a Git LFS-backed version history. It targets teams who already have a model endpoint or a GPU training stack and want the loop around it, not another inference server.
Who is it for?
Adopt Reef if you already run an OpenAI-compatible endpoint or a Slime/SGLang GPU stack and the missing piece is the loop that turns scored interactions into versioned artifacts you can roll back. Do not adopt it if you need a mature release: v0.1.3 is the newest tag and the project is pre-1.0.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository received new commits within the last day.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • Human-Agent-Society/reef7,827 ★ · +6,747
Star history
Repository2026-09-122026-11-15
Human-Agent-Society/reef1.1k7.8k

6,747 Stars in the last 64 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

The gap Reef claims: serving and training frameworks both stop short of versioning

Inference engines such as vLLM and SGLang serve live traffic but do not train weights. RL training frameworks such as Slime, veRL and AReaL train weights but do not serve live traffic. Neither group, per the README's comparison table, manages versions or stays live through updates. Reef positions itself in the fourth column of that table, claiming all five rows: serving, training, version management, staying live through updates, and evolving things that are not weights at all, meaning prompts, rules and skills.

That last row is the interesting one. Most agent stacks treat the harness (the prompt template, the tool rules, the skill definitions) as source code edited by a human and deployed like any other change. Reef treats it as a learning surface alongside model weights. The README states plainly that a deployment's recipe determines which surface its scenarios update, so the same serving endpoint can be pointed at weight training or at harness optimization depending on configuration. The intended user is a team that already has one half of the loop working and wants the other half without hand-rolling a feedback store, an eligibility filter and an artifact registry.

The four-step loop and which directory implements each step

Reef describes each learning cycle as four steps, and the README maps each to a module path. Step one, Serve: agent requests arrive and interactions are recorded, handled by reef/service for requests and records and reef/inference for inference and artifact updates. Step two, Observe: feedback is matched back to recorded interactions, handled by reef/storage/records.py for storage and reef/record2sample for matching, eligibility and samples. Step three, Grow: an update is produced from the eligible records, via reef/recipe for recipe integration and reef/train for the training pipeline, job data and training jobs. Step four, Commit: a selection policy is applied, candidates are evaluated in reef/evaluation, and the version history and release components live in reef/artifact.

The mechanism that makes this more than a pipeline diagram is the receipt. When a request hits the inference endpoint, Reef adds an x-reef-agent-record-id response header. That value is what a later report cites to identify which interaction it is judging. The README calls it the receipt. Feedback therefore arrives out of band, possibly seconds or minutes later, and the Observe step's job is to join it back to the right record and decide whether the pair is eligible for training. Eligibility is not automatic: it is computed in record2sample, which means a scored interaction can be recorded, matched, and still never influence an update.

Scenarios are created by header, not by configuration file

A request carries the x-reef-scenario header. According to the README, a new scenario name creates a scenario using the deployment's configured recipe. Requests do not select recipes. This is a deliberate constraint worth noting: you cannot route one request to weight training and the next to harness optimization by changing a header. The recipe is fixed at deployment level, and the header only partitions traffic into scenarios within it.

That design keeps the training path predictable, but it means experimenting with two learning surfaces requires two deployments. For teams evaluating whether harness optimization or weight training pays off more on their traffic, that is a real cost in GPU time and operational surface.

Getting it running: install, serve, and report

Reef's artifact and checkpoint functionality requires the git-lfs system package, and the README notes that Reef initializes Git LFS locally for its artifact repositories. That is a system-level dependency, not a pip extra, and it belongs in your container image before anything else.

From PyPI, the documented sequence is uv venv && source .venv/bin/activate, then uv pip install reef-infra, then a version check with python3 -c "import reef; print(reef.__version__)". From source, the README adds git lfs install before cloning, then uv pip install -e. in the checkout. The source path is the one the training examples assume.

For the weight-training path, the README gives the SAO example (arXiv:2607.07508). You install the extra and the runtime group: uv pip install -e ".[slime]" followed by uv pip install --no-deps --group runtime. Then you export MODEL_PATH and REEF_TOKEN, and start the server with reef serve -c recipes/sao/examples/imo_answerbench/serve.yaml --inference.model-path "$MODEL_PATH" --reef.port "8900". Readiness is checked with curl -f http://127.0.0.1:8900/healthz.

The inference endpoint is OpenAI- and Anthropic-compatible: /v1/chat/completions, /v1/responses and /v1/messages accept the provider's own request body. In the README's Python example, a client posts to /v1/chat/completions with a model and messages array, reads the receipt from the x-reef-agent-record-id response header, then sends a report containing a score, feedback and the receipts being evaluated. The example reports both a numeric score and a short textual explanation.

The harness path exists to avoid GPUs, and that is the real selling point

The README's goal table separates three paths by what you need. Weight training needs a trainable model, a supported GPU stack, and feedback your recipe can use. Harness optimization needs a model endpoint, representative tasks, and an evaluator, with no local training GPUs. Test-time training needs an execution environment, a correctness checker, and a measurable objective.

For a large share of teams, the harness path is the practical one. Prompts, rules and skills can be revised and evaluated without renting accelerators, and the feedback signal is an evaluator you write rather than a reward model you train. The trade-off is that harness optimization is bounded by the quality of that evaluator. If your evaluator is a simple string match, the loop will optimize toward exact string matching, which is a narrow objective. Check what the reef-eval package that the core install depends on provides before writing your own.

Where Reef is the wrong tool

Reef assumes a live service. If you are doing offline fine-tuning on a static dataset, the serve, observe and commit steps are overhead: you already have your records, you already have your labels, and you do not need receipts or a selection policy. A standard training run plus your own checkpoint directory is simpler.

Reef also assumes you can evaluate candidates before committing them. The Commit step applies a configured selection policy and publishes accepted updates, which implies a way to compare a candidate against the incumbent. If your objective is subjective, expensive to measure, or only observable long after deployment, the selection policy has nothing solid to stand on.

The git-lfs dependency is a second constraint. Artifact and checkpoint functionality requires it, and Reef initializes Git LFS locally for artifact repositories. In environments where Git LFS is unavailable, blocked, or where large binary objects in git are prohibited by policy, the artifact and versioning half of Reef does not function. Since version management is one of the five rows Reef claims in its own comparison table, losing it removes a large part of the reason to choose Reef over wiring a serving engine to a training framework yourself.

Compared with an RL framework plus a hand-rolled feedback store

The obvious alternative for the weight-training path is to use Slime, veRL or AReaL directly and write your own glue: a table of interactions, a feedback endpoint, a join query, and a script that exports eligible rows into the training job's expected format. That is not exotic. Many teams already have it.

The difference in approach is where the state lives. In the hand-rolled version, the training framework owns the run and your glue produces inputs for it. In Reef, the deployment owns the loop: the server that answers /v1/chat/completions is the same system that records the interaction, matches the later report, produces the update, evaluates it, and writes it into a version history under reef/artifact. The training framework becomes a component behind reef/recipe rather than the center of the system. Whether that inversion is worth it depends on how much you care about the version history and about staying live through updates, which are the two rows the README says neither inference engines nor training frameworks provide.

Maintenance, maturity and licence

Reef is at v0.1.3, tagged 2026-10-14, after v0.1.1 on 2026-09-25 and v0.1.2 on 2026-10-02, with the last push to main on 2026-11-02. That is a pre-1.0 project. The README links a roadmap issue for direction, and the documentation lives at reefinfra.ai rather than in the repository, so the repository alone does not tell you the full intended surface. Expect the recipe layout and configuration keys to move.

The practical maintenance cost is concentrated in two places. First, the recipe: the SAO example's serve.yaml lives under recipes/sao/examples/imo_answerbench/, and any change to how record2sample computes eligibility or how evaluation compares candidates will likely require recipe changes. Second, the artifact store: Git LFS repositories grow, so check the documentation for retention or pruning of old versions. Upgrading means re-checking the GPU requirements page before bumping, since the slime extra and the runtime dependency group are installed separately and could drift.

Licence is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant. That is permissive and standard for infrastructure. This is not legal advice; if you redistribute Reef or bundle it into a product, read the LICENSE file with your own counsel.

Editorial conclusion

Adopt Reef if you already run an OpenAI-compatible endpoint or a Slime/SGLang GPU stack and the missing piece is the loop that turns scored interactions into versioned artifacts you can roll back. Do not adopt it if you need a mature release: v0.1.3 is the newest tag and the project is pre-1.0. Before committing, verify that git-lfs is installed on every host that will hold artifact repositories, confirm your GPU environment matches the Evolve your model requirements page, and check whether your recipe's feedback shape is one the record2sample eligibility logic actually accepts.

Frequently asked questions

Does Reef need a GPU to run?

The core install is built for CPU. The dependency list is deliberately kept free of training and GPU packages so that pip install reef-infra supports the base recipe configuration on CPU, and the heavier stacks are separate extras.

Why does Reef install as reef-infra but import as reef?

They are the same code under two names. The project metadata sets the distribution name to reef-infra and notes that the import package remains reef, so the install command and the import statement deliberately differ.

Can one Reef server update both model weights and prompts?

Not in a single deployment. Reef supports two learning surfaces, weights and harnesses, and the deployment's recipe decides which one its scenarios update. Since requests do not select recipes, one server runs the recipe named in its configuration.

Why does the Reef source install start with git lfs install?

Reef's artifact and checkpoint functionality requires the git-lfs system package, and Reef initializes Git LFS locally for its artifact repositories. The command comes before the clone for that reason, not as a general Git habit.

How does feedback reach the right response in Reef?

Each response carries an x-reef-agent-record-id header whose value is the receipt. A report includes that receipt along with a numeric score and, when the recipe needs more, textual or structured feedback, and the endpoint validates the report schema in reef/core/reports/.

Official sources

  1. Human-Agent-Society/reef on GitHub
  2. Issues
  3. License: Apache-2.0
  4. README
  5. Releases
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/human-agent-society-reef.svg)](https://hysenlabs.com/projects/human-agent-society-reef)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/human-agent-society-reef.svg)](https://hysenlabs.com/projects/human-agent-society-reef)

Was this useful?