best-of-Agent-Harnesses: a ranked index with an MCP server for harness selection
🏆 Curated, ranked list of AI agent harnesses (100+) — plus an MCP server, llms.txt & JSON so agents can recommend them too. Rescored weekly.
At a glance
- What is it?
- RyanAlberts/best-of-Agent-Harnesses is a curated, weekly-rescored list of more than 100 agent harnesses, shipped as a static site, an llms.txt, a JSON file and an MCP server so that both humans and agents can query it. Its real value is the pick-by-use-case routing and the harness-versus-model argument, not the star ranking embedded in the tables.
- Who is it for?
- Adopt this list if you are choosing a harness for a specific model and task and want the decision pages and MCP endpoints rather than a README dump. Do not adopt it as a dependency, a vendored dataset, or a substitute for running your own trial.
- Can I use it commercially?
- Yes, with credit. CC-BY-SA-4.0 allows commercial use as long as you credit the authors and indicate what you changed. It is written for creative content, so check how it applies to any code.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem: harness choice is a benchmark variable nobody controls
The repository opens with a claim that is easy to skim past and hard to argue with. According to the README, on SWE-bench Pro, swapping the agent harness changed pass@1 more than many model upgrades do, citing an AINews post from August 2026 and analysis by @joelniklaus. The same figures are given per model: 23% to 52% pass@1 on GLM-5.2, and 15% to 36% on Gemma 4 26B. The README also states that harness rankings barely transfer across models, with a rank correlation of -0.05.
If those numbers hold, then most harness comparisons circulating in blog posts are measuring the model, not the harness, and the reader is left with a ranking that does not apply to their stack. That is the gap this repository targets. It is aimed at engineers who have already picked a model and now need to pick the runtime around it: which tools exist, what needs approval, what the model sees each turn, what survives a crash. The README frames that runtime as the harness and quotes Simon Willison's definition of the agent itself as "an LLM agent runs tools in a loop to achieve a goal", with the harness being everything around that loop. Andrej Karpathy's framing, that the model is the kernel process of a new operating system and the harness is the rest of the OS, is used to set the scope.
The audience is therefore narrow and technical: people shipping agentic systems who need to compare environment, orchestration, lifecycle and guardrails across projects, not people looking for a first agent tutorial.
What is actually in the repository: 164 entries, two charts, and a decision layer
The badge at the top of the README says 164 projects, while the repository description says 100+, so the count has grown past the description. Entries are grouped by category, and the README states that projects are marked with a star for headless-ready and a different mark for durable, both of which appear in the tables.
The more interesting artifact is the decision layer layered on top. There is a guide titled How to pick a harness, described as six questions that turn the list into a decision. There is a trial protocol, How to test-drive a harness, described as a two-week protocol with a fair setup, tasks from your own repos, seven measurements and a walk-away test. Then there are head-to-head comparison pages, each naming the projects being compared: OpenClaw vs Hermes for always-on personal agents, a terminal coding agent page covering opencode, Codex, Gemini CLI, crush and goose, a multi-agent orchestration page covering the OpenAI Agents SDK, CrewAI, AutoGen and LangGraph, a memory layer page covering Mem0, Letta and claude-mem, sandboxing covering E2B, Daytona and Modal, evals covering SWE-bench, inspect_ai and Terminal-Bench, eval and observability platforms covering Langfuse, LangSmith, Braintrust and Phoenix, browser agents covering browser-use, Stagehand, Playwright MCP and chrome-devtools-mcp, browser infrastructure covering Browserbase, Steel and Hyperbrowser, Claude Code skill packs covering superpowers, GStack, get-shit-done and Anthropic Skills, and context files covering AGENTS.md, CLAUDE.md, skills and MCP tool search.
Two generated charts are committed as SVG assets. One plots every project by adoption surface area against GitHub stars, where adoption surface area is described as a simplicity to capability axis. The other places loop-owning projects by designed autonomy regime and failure-recovery tier. The README states both regenerate from the list data on every refresh, which is the detail that makes them worth looking at: they are derived, not hand-drawn.
The MCP server and the JSON export are the part that changes how you use it
Most curated lists are read once and then bookmarked and forgotten. This one exposes itself as data. The README lists three machine-readable surfaces: an MCP server exposing tools including recommend and pick_harness, an llms.txt file, and a harnesses.json file. The stated purpose is that your agent can recommend harnesses too, matched to your model and task, rather than inheriting whichever harness someone else benchmarked.
That is a coherent design. If harness rankings do not transfer across models, then a static ranked list is the wrong output format and a query interface is the right one, because the answer depends on inputs the list author cannot know in advance. The recommend and pick_harness tools are the only two named in the material, and the README does not document their parameters or return shapes in the excerpt available here, so the exact matching logic cannot be confirmed from what is supplied. What can be confirmed is the intent: the query is parameterised by model and task.
There is also a searchable site at ryanalberts.github.io/best-of-Agent-Harnesses, described as one page per harness with filters for capability, autonomy and recovery. The same three axes appear in the charts and the marks in the tables, so the taxonomy is consistent across the HTML site, the SVGs and the JSON. Consistency across three renderings of one dataset is harder to maintain than it looks, and the weekly rescore is what keeps them from drifting apart.
How the ranking is computed, and why you should read the axes instead
The README is explicit about the scoring inputs: projects are ranked by relevance to harness concerns (environment, orchestration, lifecycle, guardrails) and by stars and activity. It also names the axis used in the landscape chart, adoption surface area, defined as a simplicity to capability scale.
That combination deserves scrutiny. Relevance to harness concerns is a judgement call made by the maintainer, and the README does not publish the weighting between that judgement and the star count. Stars are an adoption proxy with a well-known bias toward older projects and toward whatever was fashionable when a category formed, and a harness released six months ago will lose that comparison by default. Activity is a better signal, but the README does not state the window used to compute it.
The axes are the more defensible part of the design. Autonomy asks how much unsupervised rope a project is designed to give, and recovery asks what happens when a run dies. Those are questions with answers you can check against your own requirements without needing to trust the maintainer's weighting. A team running unattended overnight jobs cares about the durable mark; a team that wants a human in the loop before every file write cares about the autonomy placement. The ranking order is the least useful thing on the page, and the marks and axis positions are the most useful.
Getting it running: the surfaces you can actually call
The repository is a list, not a library, so installation is mostly about wiring up the machine-readable surfaces. The README gives file paths rather than commands: llms.txt and harnesses.json at the repository root, and the MCP server described under a For agents section. Because the project is Python and the MCP server is one of its deliverables, the server is the only component with a runtime, and the README excerpt does not include the launch command or the package name, so the exact invocation cannot be confirmed from the material supplied here. That is a real gap for anyone who wants to point an agent at it today.
The static site is the zero-setup path: the homepage URL serves one page per harness with filters for capability, autonomy and recovery, and needs nothing installed. The JSON file is the path for anyone building their own index or diffing two weeks of rankings. The llms.txt file is the path for pasting context into a model that does not speak MCP.
The contribution workflow is described in a Contribution section of the README, and the badge row states contributions are welcome. The list ships as dated releases, with the most recent being list-2026-09 on 2026-09-09, which gives you a stable identifier to pin if you consume the JSON programmatically. Pinning to a dated release is the only versioning guarantee visible in the material.
Where the list is the wrong tool
A weekly rescore is a maintenance promise, and the failure mode is quiet. If the rescore job stops or a category goes stale, the site, the SVGs and the JSON all keep rendering confidently, and nothing in the material describes a freshness indicator on individual entries. You cannot tell from a row whether it was re-checked this week or last year.
The second limitation is the one the project admits in its own framing. If harness rankings barely transfer across models, then any ranked order is an average over an unknown distribution of models and tasks, and the specific model you are running may sit at the edge of that distribution. The MCP server is the proposed fix, but a recommendation tool is only as good as the metadata behind it, and the README excerpt does not show the schema of harnesses.json or how capability tags are assigned and verified.
The third limitation is scope. The list is about harnesses and harness techniques, and the comparison pages are all harness-adjacent: orchestration, memory, sandboxing, evals, browser agents, skill packs, context files. If your problem is model selection, prompt design, or fine-tuning, this repository has nothing for you, and the harness-versus-model numbers in the README could mislead you into thinking the harness is always the higher-leverage variable. The cited figures are pass@1 on SWE-bench Pro, which is a coding benchmark; the README does not claim the same magnitude of effect for other task families, and neither should you.
Alternatives, and the difference in approach
The obvious alternative is an awesome-list in the same space, which collects links under headings and leaves evaluation to the reader. The difference here is that this repository commits to a ranking, publishes the axes behind it, regenerates charts from the data, and exposes the whole thing over MCP and JSON. An awesome-list does not rescore, so it never goes stale in a way you can measure, and it never claims a position either. If you want a neutral index and you are doing your own evaluation anyway, a plain awesome-list is honest about what it is and this repository is not.
A second alternative is to skip the list and benchmark directly, which is what the repository's own two-week trial protocol describes. That protocol, tasks from your own repos and seven measurements, is the thing that would actually settle the question for your stack, and it is more work than reading a ranking. The repository is best understood as a way to cut the candidate set before you run that trial, not as a replacement for it.
A third alternative sits inside the same repository: the head-to-head comparison pages. If you already know you are choosing between, say, LangGraph and CrewAI for orchestration, the multi-agent orchestration page is more directly useful than the full ranked list, because it is scoped to the decision you are actually making. Reading the whole list when you have already narrowed to two projects is wasted effort.
Licence, maintenance cost, and what to check before you rely on it
The repository is licensed CC-BY-SA-4.0. That is a content licence, not a software licence, which fits a list whose primary output is prose, JSON and SVG. The share-alike term matters if you intend to redistribute the data or build on it: a derivative list or an adapted dataset carries the same licence forward, and attribution is required. If you are ingesting harnesses.json into an internal tool, the licence question is whether that counts as a derivative work, and that is a question for your own counsel rather than something this review can settle.
The maintenance cost on your side is low but not zero. There is nothing to install if you only read the site. If you consume the JSON, you inherit a schema that the README excerpt does not document, and you inherit a weekly update cadence, which means either pinning to a dated release such as list-2026-09 or accepting that your index shifts under you. The MCP server is the surface most likely to change shape, since it is the newest and the least documented in the material available here.
Before relying on a recommendation, check three things in the repository itself: whether harnesses.json documents its fields, whether the MCP server's launch instructions are present outside the README excerpt, and whether the entry for your candidate harness has been touched in the current release. The first two are documentation gaps that a reader can verify in a minute. The third is the one that matters, because a ranked list is only as good as the freshness of the row you are reading, and this project's freshness signal lives in its release tags rather than in the table.
Editorial conclusion
Adopt this list if you are choosing a harness for a specific model and task and want the decision pages and MCP endpoints rather than a README dump. Do not adopt it as a dependency, a vendored dataset, or a substitute for running your own trial. Before trusting a recommendation, open harnesses.json and check when the entry was last touched, because the list is rescored weekly and stale rows look identical to fresh ones.
Community notes