AutoHarness: a self-learning skill layer for Claude Code

The problem AutoHarness solves, and who it is for

Claude Code users accumulate skills, and the accumulation does not manage itself. Write a skill for one debugging session, a second for a similar one next week, and you get near-duplicates that compete for recall. Nothing tells you which of them is still being followed. Nothing removes the ones that stopped mattering. The README frames the wider bet this way: the harness does much of the work, yet it is still rebuilt by hand every model generation, and AutoHarness bets one slice of it, the skill layer, can maintain itself.

The intended user is a Claude Code user who works in long tool-heavy stretches on a single machine and is willing to let a plugin write into .claude/skills/. Not a team looking for a shared skill registry, and not someone who wants a benchmark score. The README is explicit that validation happens in use rather than on a benchmark: a skill survives by being adhered to in later turns, measured as loads over the requests it was available for. There is no oracle on the active path and no tokens spent on a dedicated eval.

How the reflection loop actually fires

The mechanism is a counter, not a timer. Every main-session tool call advances a counter, and the turn that pushes it past AUTOHARNESS_REFLECT_EVERY_N ends with a background reflection. The default is 50. That choice has a visible consequence: a conversation that only talks never triggers a reflection, because no tool calls accumulate. A working stretch does. The same quantum governs the curator, the periodic pass that merges the library as a whole, at AUTOHARNESS_CONSOLIDATE_EVERY_N, default 250. Consolidation is deliberately rarer than distillation.

What carries the reflection is a separate knob. AUTOHARNESS_CARRIER defaults to bundle, which hands a redacted window plus a prior-context digest to a fresh subagent. The alternative, fork, resumes and forks the session that just ended so the reflector reads the real conversation on the parent's warm cache. The README states the default stays bundle until the cache-hit measurement is in, which is an honest admission that the cheaper-looking option has not been measured yet. AUTOHARNESS_DIGEST_EXCHANGES, default 20, controls how many exchanges before the episode window are compressed into that digest, text and tool names only. The README notes the fork carrier does not use it, since fork replays the real conversation instead.

Merging, pruning and the ledger that keeps them apart

A new episode does not always produce a new skill. The reflector compares it against what already exists and folds same-scenario skills into one, so the layer consolidates by category instead of accreting near-duplicates. The detail worth noticing is what a fold leaves behind: the README says a fold records which skill absorbed which, so a merge is never mistaken for a death. That matters if you ever audit the library, because a skill disappearing from the directory is ambiguous without that record.

Every create and update logs its scenario and decision to a per-skill ledger. The README describes this as the raw material to build a benchmark from real usage if you ever want one, which is the right framing: the evidence is kept, but no benchmark is shipped. The scope boundary is equally explicit. AutoHarness touches only the skills it generated through the plugin. Skills you wrote or installed are left alone. When you uninstall, the skills it landed and its own state stay on disk, and each self-authored skill carries a self-authored ledger marker so you can tell them from yours.

Installing AutoHarness and landing a first skill

The prerequisite is python3 on your PATH. AutoHarness runs entirely as Python with zero third-party dependencies, and the README states its hooks and MCP server will not fire without it. The platform badge lists Linux and macOS.

Installation happens inside the Claude Code input box, not a shell. Two commands add the marketplace and install the plugin under its full plugin@marketplace id.

/plugin marketplace add tigerless-labs/autoharness
/plugin install autoharness@autoharness

After that, run /reload-plugins or restart Claude Code. Configuration is described as zero config: it starts watching sessions and lands learned skills into .claude/skills/ in the background.

There is nothing you must invoke, but one entry point exists. After you work something out, /learn distills the session you are in right now, and the README says the lesson goes through the same proposal-and-validation chain the background pass uses.

/learn

Updating is a terminal operation, and the order matters: refresh the catalog first, then update, then restart.

claude plugin marketplace update autoharness
claude plugin update autoharness@autoharness

The README explains why the refresh comes first: without it, update checks a stale local catalog and may report already at the latest version when a newer release actually shipped. A version bump is a fresh cached copy, not a hot reload, so restarting is required.

The session-start index is the part you may not want

Every session opens with a grouped index of the skills AutoHarness wrote, so recall does not depend on the host happening to surface them. The host's native recall is left as it was. That index costs context on every single session, and the README treats the cost as a real one rather than a rounding error. AUTOHARNESS_INDEX_DESC_MAX_CHARS defaults to 60 and is described as a per-line description budget, with the index framed as a scan surface rather than the full trigger text. Raise it and every session pays more context.

If you decide the index is not worth that, AUTOHARNESS_INDEX_SUSPENDED=1 stops injecting it entirely. The README is clear that everything else keeps running: the lifecycle pass, the use and view counters, the last-run summary. It presents this as the switch for measuring what the index is actually worth, and for anyone unwilling to spend the context on it. That is a more useful escape hatch than a global off switch, because it isolates one component instead of disabling the plugin.

A separate cap applies to a skill's own description. AUTOHARNESS_SKILL_DESC_MAX_CHARS defaults to 1024 and is described as a hard cap matching the host's limit. The README text is truncated at that point, so the exact host-side wording is not available here.

Where AutoHarness is the wrong tool

The clearest failure mode is a workflow with few tool calls and a lot of discussion. Reflections are counted in tool calls, so a session spent reasoning in prose advances the counter barely at all and may never trigger a background pass. You can still call /learn explicitly, but the automatic layer that the project is built around will look inert.

A second limitation is the carrier. The default bundle path hands a redacted window and a text-only digest to a fresh subagent, which means the reflector sees a compressed view rather than the conversation. The README says fork exists and that the default stays bundle until the cache-hit measurement is in. Until that measurement is published, choosing fork means taking on behaviour the project has not finished evaluating.

Third, this is not a shared or team-level skill system. State lives in ~/.claude/autoharness/ globally or <repo>/.claude/autoharness/ per project, and the skills land under .claude/skills/. There is no described mechanism for reviewing, approving or distributing what it learns across people. If your requirement is a curated skill library with sign-off before anything reaches other engineers, the automatic write path is the wrong shape.

Finally, the README does not document rollback of a single learned skill, nor a way to undo a consolidation that folded two skills you wanted kept separate. The ledger records which skill absorbed which, which gives you the information to reconstruct, but reconstruction is manual.

How this differs from writing skills by hand or running an eval harness

The obvious alternative is the plain Claude Code workflow: you write skills yourself under .claude/skills/ and maintain them by hand. The difference is not convenience, it is where the signal comes from. A hand-written skill encodes what you believed was worth keeping at the moment you wrote it, and it never expires. AutoHarness derives the skill from the session you were already having, and then lets adherence in later turns decide whether it survives. The cost of that trade is control: a hand-written library changes only when you change it, while an AutoHarness library changes on its own at a cadence you set with AUTOHARNESS_REFLECT_EVERY_N and AUTOHARNESS_CONSOLIDATE_EVERY_N.

A different kind of alternative is an evaluation harness that scores skills on a held-out benchmark before promoting them. AutoHarness deliberately does not do this. The README states there is no oracle on the active path and no tokens spent on a dedicated eval, and that a skill survives by being adhered to in later turns. That design avoids the cost of a benchmark run, but it also means the promotion signal is usage rather than correctness. A skill that gets loaded often is not necessarily a skill that is right; it is a skill that keeps being surfaced. The per-skill ledger is where you would go to check that distinction, since it records the scenario and decision behind each create and update.

Maintenance, upgrades and what the MIT licence leaves you

The last push to the repository was on 2026-09-04, and the repository is not archived. No releases were retrieved, so there is no tagged version to pin against. The README describes the installed copy as cached by the version field in plugin.json, and states a release reaches users only when that field is bumped. That is the upgrade mechanism: there is no separate changelog described, and the practical way to know you are current is the plugin update command returning something other than already at the latest version after a catalog refresh.

Third-party marketplaces have auto-update off by default. The README gives the path to turn it on once: /plugin, then Marketplaces, then autoharness, then Enable auto-update. Leaving it off means every upgrade is a manual three-step sequence of refresh, update and restart, and forgetting the restart leaves you on the old cached copy.

The licence is MIT, which is permissive and places few obligations on how you use or redistribute the code. That says nothing about the skills AutoHarness generates from your sessions, which are your own content and live outside the plugin in .claude/skills/ and the state directories. If those sessions contain material you cannot redistribute, the licence on the plugin does not change that; the generated files inherit whatever constraints apply to their source. This is a factual boundary, not legal advice.

Uninstalling is two commands, and the README is explicit that it only stops the plugin from running. The skills it landed and its own state stay on disk. Clearing them means deleting the state directory (~/.claude/autoharness/ globally, <repo>/.claude/autoharness/ per project) and the self-authored skills under .claude/skills/, each of which carries a self-authored ledger marker.

Editorial conclusion

Adopt AutoHarness if you already run Claude Code on Linux or macOS, have python3 on PATH, and want skills distilled from your own sessions rather than written by hand. Skip it if you work on Windows (the badge lists Linux and macOS only), if you need a stable tagged release to pin (the README describes version bumps in plugin.json, and no releases were retrieved), or if you are unwilling to spend context on the session-start index. Before you trust it, verify three things: that the reflection cadence of 50 tool calls matches how long your working stretches actually are, that the index description budget of 60 characters is enough for your skills to be recognizable at session start, and that a merge recorded in the per-skill ledger matches what you expected, since a fold is not a deletion.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. tigerless-labs/autoharness on GitHub