Autoresearch skill: a git-backed modify, verify, keep-or-revert loop for coding agents

Every iteration is one change, a commit, a measurement and a verdict

The loop has eight steps. The agent reviews the current state, the git history and the results log, picks the next change from what worked, failed or remains untried, and makes one focused change. It commits before verification, runs mechanical verification such as tests, benchmarks or scores, and then decides: improved means keep, worse means git revert, crashed means fix or skip. It logs the result and repeats until the iteration count is complete or the goal is met.

Committing before verifying is what makes the revert cheap, because every experiment is a commit with an experiment: prefix and the agent reads git log and git diff before each iteration. Progress is logged in TSV format. The design is borrowed from Karpathy's autoresearch, a 630-line Python script described as improving ML models overnight at about 100 experiments per night.

The tagline says forever, but the rules say bounded by default

The repository description ends with the words Repeat forever, and the README tagline promises that you wake up to results. The rule table is stricter: every command has a default iteration count, and unlimited is opt-in through an Iterations: unlimited setting.

That difference changes how you run it. An unattended session left on the default stops by itself, while an unlimited one keeps spending tokens and creating commits until you stop it. Choose a bound that matches your budget, and treat unlimited as a deliberate decision for a task with a cheap, reliable metric. The same table asks for atomic changes, so a failure points to one change, and for simplicity: equal results with less code count as a keep.

Setup refuses vague goals because the metric must be mechanical

Before looping, the agent performs a one-time setup. It reads all in-scope files, extracts or asks for a mechanical metric, defines which files may be modified and which are read-only, and establishes a baseline by running verification on the current state as iteration zero. Then it shows you the setup and begins.

A fourth rule explains why this is strict: there is a mechanical search signal and independent acceptance, with no subjective looks-good, and the optimized metric never certifies that behaviour was preserved. In practice that means a metric such as a test count can be gamed by an agent that weakens the tests, so the scope should mark test files as read-only. The classic invocation passes Metric and Verify lines explicitly. A goal like make the writing better has no number, and the loop has nothing to judge it with.

Version 2.2.0 added an orchestrator that derives its own success predicate

In the autonomous orchestrator mode, you type a plain-language goal to the autoresearch command. It classifies the goal, derives a Success predicate, asks you to confirm it once, and then loops across subcommands until done, without you chaining them. The flow diagram lists plan, loop, debug, fix, secure and ship stages, with STRIDE, OWASP and red team checks under the secure stage.

Invocations with Metric and Verify lines still run the classic loop unchanged. The risk of the orchestrator is that the success predicate is derived by the model, so the one confirmation is the only place a human checks it. Read that predicate carefully, because everything afterwards optimises for it, including whatever shortcuts it permits.

Fork-and-flush forks fresh branches when the loop plateaus

Version 2.3.0 adds fork-and-flush, which the README ties to a paper by Cai and colleagues. The loop forks fresh-context branches from the best commit so far at 33% and 66% of the budget, or after 5 iterations with no keep. It runs each branch briefly and keeps the winner.

The point is escaping a plateau: when small changes stop helping, branches with a clean context try different directions instead of repeating the same ideas. The cost is extra runs and extra branches in your repository, and the rule of 5 iterations without a keep means a noisy metric can trigger forks that were not needed. The same release names pi, Muse and fork-and-flush in its title, which makes it a larger change than the patch release before it.

Hook guardrails exist only for Claude Code and pi

Five hosts are supported for the core skill, bundled runtime, installation and verification: Claude Code, OpenCode, OpenAI Codex, pi and Muse. Hook guardrails ship only for Claude Code, through native hooks, and for pi, through extension events. The README calls hooks defense in depth and not a security sandbox.

The protections include scout-block, which keeps node_modules, .git and caches out of context, privacy-block, which blocks reading .env files, SSH keys and credentials, and dangerous-cmd-block, which blocks force-push, rm -rf and git reset --hard. A simplify gate warns at 400 lines of code and blocks at 800 before shipping. The consequence is plain: on OpenCode, Codex or Muse nothing in this repository blocks a destructive command, so run those hosts in a disposable checkout. All hooks are on by default and can be disabled individually.

A thin routing file replaced a hundred-thousand-token skill

Version 2.1.0 rebuilt the architecture. The earlier single SKILL.md of 813 lines, about 100K tokens per invocation, became a 41-line routing file plus 12 self-contained command files of 94 to 120 lines, about 5K to 8K tokens per invocation. The project calls this a 95% token reduction with the same capability surface, which is its own measurement.

For pi, the root package.json exists so that pi can install the repository directly:

pi install git:github.com/uditgoenka/autoresearch

Its pi field loads the extension from pi-extension/src/index.ts. Recent releases are v2.3.0 and v2.2.3, both on 2026-10-31, and v2.2.2 on 2026-08-12. The last push was on 2026-10-31, and the license is MIT.

Editorial conclusion

Autoresearch fits work where a number can judge each change, such as test pass counts, benchmark times or lint scores, and where git history is an acceptable record of experiments. It is a poor fit for goals that need human taste, and on OpenCode, Codex or Muse it runs without the hook guardrails that Claude Code and pi receive. Define the metric and the file scope first, keep iterations bounded, and review the experiment commits before merging.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. uditgoenka/autoresearch on GitHub