AutoHarness Wraps an LLM Client and Inserts a Governance Pipeline Between the Model and Its Tools
AutoHarness: Automated Harness Engineering for AI Agents
At a glance
- What is it?
- AutoHarness is a Python framework that intercepts tool calls made by an LLM client and runs them through a configurable pipeline of risk classification, permission checks, output sanitization and audit logging. The design is genuinely useful for agents that touch a shell or a filesystem, but the default mode is the heaviest one, and the framework is only three weeks old at the time of writing.
- Who is it for?
- AutoHarness fits teams running agents that can execute shell commands or touch files, and who need an audit trail or a permission boundary rather than another orchestration graph. It does not fit teams whose agents only call read-only APIs, or anyone who needs a stable API surface today: v0.1.0 shipped on 2026-04-02 and the repository was last pushed the same day.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 166 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap AutoHarness targets: agents that reason well and execute carelessly
The README frames the problem with a slogan: "Agent = Model + Harness. The model reasons. The harness does everything else." That is marketing, but it points at a real split. A model that has learned to plan a refactor still has no opinion about whether the shell command it just emitted should run. The harness is the layer that decides. AutoHarness is an attempt to make that layer a library you install rather than code you write per project.
The target user is someone building an agent that calls tools, where at least some of those tools can change state. The README's own comparison table lists the failure it cares about: "Agent runs rm -rf /, nothing stops it." Alongside that it lists context blowing past a token limit, missing per-call cost attribution, prompt injection, absent audit trails, and every agent sharing one permission set. Those six items are the product's scope. If your agent only retrieves documents and summarizes them, most of this is overhead you will pay for and never use.
What actually sits between the model and the tool call
The core mechanism is a fixed pipeline. The README shows it as six numbered stages: Parse and Validate, Risk Classify, Permission Check, Execute, Output Sanitize, Audit Log. Every tool call flows through that sequence. The Standard mode extends it to eight steps, and Enhanced to fourteen, adding a turn governor, alias resolution and failure hooks on top of the base six.
The integration point is the interesting part. AutoHarness.wrap() takes an existing client object and returns something with the same call shape. The README example passes OpenAI() into AutoHarness.wrap and then calls client.chat.completions.create with a model, messages and a tools array containing a Bash function. So the wrapper is not a new SDK you migrate to; it is a proxy that observes and can veto. Separately, AgentLoop is a full loop you configure with a model name and a constitution file path, for cases where you want the framework to own the iteration rather than just police it.
Governance rules live in a YAML constitution. The README shows a single key, mode, set to core, standard or enhanced. The CLI can validate a constitution file and export one in a different harness format, which suggests the constitution is intended to be portable across agent runtimes rather than tied to this one.
Getting it installed and picking a pipeline mode
Installation is a clone and an editable install:
git clone https://github.com/aiming-lab/AutoHarness.git cd AutoHarness && pip install -e .
The README states Python 3.10 or later. There is no published package install command in the material, so the editable install from a clone is the documented path.
The first decision after install is the mode. The README is explicit that Enhanced is the default: "Users get the strongest governance out of the box. Switch to Core for minimal overhead." You change it either in the constitution file with mode: core, or from the shell with autoharness mode enhanced. Running autoharness mode with no argument prints the current mode.
Other CLI surface documented in the README: autoharness init runs an interactive wizard covering agent type, LLM provider, security level and pipeline mode. autoharness check --stdin --format json checks a tool call against your rules, taking tool_name and tool_input. autoharness audit summary reads the audit log. autoharness install --target claude-code registers the framework as a Claude Code hook, which is the one integration path that does not require writing Python. autoharness export --format cursor writes a constitution out for a different harness.
That default is worth pausing on. Shipping the fourteen-step pipeline as the out-of-box behaviour means a new user's first experience is the most expensive configuration, and the README's own table labels it with a warning symbol. It is a defensible choice for a safety-oriented project, but it inverts the usual convention of starting minimal.
Where the pipeline is thin, and where it is the wrong tool
The risk classification stage is pattern matching. The README says built-in risk patterns detect dangerous operations, secret exposure, path traversal "and more," without enumerating the patterns or describing how they are matched. That is the weakest documented link in the chain. A pattern list is a denylist, and denylists fail on the operations nobody thought to write down. An agent that reaches a destructive outcome through two innocuous-looking commands in sequence may pass every individual check.
The output sanitizer has the same shape of problem in reverse. It runs after execution, which means the tool has already acted. Sanitizing output protects the model's context and anything downstream that reads it; it does not undo a side effect. Anyone reading "Output Sanitize" as a safety net should read the stage order again.
The cost and context claims are similarly unquantified. The README lists token budget management and per-call cost attribution with model-aware pricing, and mentions truncation keeping context under control. It gives no numbers, no pricing table, and no description of what happens when the budget is exhausted. Whether truncation drops the oldest turns, summarizes them, or fails the call is not stated in the material.
Finally, the project is young. v0.1.0 was released 2026-04-02 and the repository's last push is the same timestamp. The README advertises 958 passing tests, which tells you the author runs a suite, not that the API will hold still. Treat every import path as provisional.
How it differs from LangGraph and Guardrails AI
The README's comparison table puts AutoHarness against LangGraph, Guardrails AI and the OpenAI SDK. The distinctions it draws are worth taking at face value because they describe architecture, not quality.
LangGraph is an orchestration framework. Its job is to express an agent as a graph of nodes and edges, and the README credits it with multi-agent support and marks it as having no tool governance pipeline, no context management and no input or output validation. If your problem is "I cannot express this control flow," LangGraph is the tool. If your problem is "my control flow works but I cannot stop it from running rm -rf," that is AutoHarness's stated territory.
Guardrails AI is validation-first. The README marks it as output-only for tool governance and as having rails for validation, but no pipeline and no context management. The difference is where the check sits. A rail validates a value. AutoHarness's pipeline wraps the act of calling a tool, with permission checks before execution and an audit record after. Those are different insertion points and they catch different failures.
The OpenAI SDK is the baseline: trimming for context, handoff for multi-agent, nothing for governance or validation. AutoHarness.wrap() is designed to sit on top of exactly this kind of client, so the two are not really competitors. The README also claims "0 vendor lock-in," which is consistent with a wrapper that accepts a client you already own, though the constitution export command suggests the author is thinking about portability more broadly.
Maintenance surface, licence, and what to check before committing
The licence is MIT, stated in the README badge and the repository metadata. That permits commercial use and modification with the copyright notice retained. It is the permissive default and carries no copyleft obligation. This is a description of the licence text, not legal advice; if the distinction matters to your organisation, read the LICENSE file.
The maintenance cost is the constitution. Every rule you add is a file you own, and the framework's behaviour depends on that file being correct. The CLI gives you autoharness validate constitution.yaml to check it, which is the right primitive, but the rules themselves are your maintenance burden, not the project's. Expect to revisit them whenever your agent gains a new tool.
The upgrade cost is unknown from the material. There is one release, v0.1.0, and no changelog or migration notes beyond the release announcement. The README does link translated documentation in eight languages, which is a real ongoing translation surface for a project this young, and one that will drift from the English source quickly.
What to verify first, concretely: run autoharness check --stdin --format json with your own tool_name and tool_input values and see which of your real operations the risk patterns flag. Then run an agent turn and read the JSONL audit output to confirm the decisions recorded match what you expected. If the patterns miss the operations you actually care about, the pipeline is not doing the job you bought it for, and no amount of mode switching will fix that.
Editorial conclusion
AutoHarness fits teams running agents that can execute shell commands or touch files, and who need an audit trail or a permission boundary rather than another orchestration graph. It does not fit teams whose agents only call read-only APIs, or anyone who needs a stable API surface today: v0.1.0 shipped on 2026-04-02 and the repository was last pushed the same day. Before adopting it, run autoharness check --stdin --format json against your own tool calls and read the generated JSONL audit output to confirm the risk patterns actually match the operations your agent performs.
Community notes