Harness-1: a 20B search agent trained inside a stateful retrieval harness

The division of labour: state in the harness, judgement in the policy

Harness-1's core idea is an architectural split. The harness maintains recoverable search state, everything mechanical about a research session: candidate documents, curated evidence, evidence links, verification records, and budget-aware context. The policy, the 20-billion-parameter model, keeps the semantic decisions: what to search, which documents to inspect or curate, what claims to verify, and when the evidence is sufficient.

That split is the interesting claim, because it inverts the usual design where the model's context window is the workspace and everything must be squeezed into it. Here the workspace is an explicit, inspectable structure, and the model commands it. Long-horizon search fails when state is implicit and context overflows; externalizing the state is the fix the name advertises, and the accompanying paper at arXiv:2606.02373 carries the details.

The framing is aimed squarely at a cost question: matching frontier AI's search capability with a 20B model. A model two orders of magnitude smaller than frontier scale, compensating with structure rather than parameters, is a claim you can evaluate cheaply, which is the best property a research release can have.

Quickstart: serving the checkpoint locally

The minimal path is documented for Linux with Python 3.11 or newer, uv installed, a CUDA-compatible NVIDIA GPU environment, vLLM with GPT-OSS support, and access to the released checkpoint. Three commands cover the smoke test:

uv sync --extra vllm

export HARNESS1_HF_MODEL=pat-jj/harness-1

less docs/run_vllm_browsecompplus.md

vLLM downloads the weights from Hugging Face on first use and reuses the local cache, so the checkpoint identifier is the only model configuration you touch. The detailed guide the third command opens covers the vLLM and BrowseComp+ evaluation path end to end.

What you can do from there is enumerated honestly: serve the checkpoint locally, run raw completions smoke tests with token-id outputs, evaluate search behaviour on BrowseComp+ when a compatible retrieval backend is available, run Tinker-hosted inference against the published checkpoint, inspect and extend the harness, tool environment, training scripts and evaluation runners, and run ablations and baselines. It is a research repository's task list, written like one.

Training data, published with its corpora

The training data is released as a dataset repository with a stage column, and the numbers are refreshingly small: 899 raw GPT-5.4-generated SFT trajectories for supervised fine-tuning, and 3,453 SEC training-split query records for reinforcement learning. Both the generation and training scripts are named in the README, so the provenance of each stage is traceable to code.

from datasets import load_dataset

ds = load_dataset("pat-jj/harness-1-train-data", split="train")
sft = ds.filter(lambda row: row["stage"] == "sft")
rl = ds.filter(lambda row: row["stage"] == "rl")

The same dataset repository ships the retrieval corpora under a corpora directory, with chunk text and cleaned metadata for BrowseComp+, web, patents and SEC, loadable directly as parquet. Publishing corpora alongside trajectories closes the usual reproduction gap where the model is open but the world it searched is not.

The README then adds the reproducibility caveat that matters: the released chunks provide text and metadata, but the code still expects a compatible retrieval backend for full search evaluation, and the recommended path for rebuilding indexes is a named external data-generation pipeline. Open data plus a documented index-rebuild route is the honest version of reproducibility when you cannot ship a live search index.

Setup levels, scaled to intent

The documentation is tiered, which respects that different readers want different depths. Minimum model serving is the two commands above plus the guide. Full BrowseComp+ evaluation adds real prerequisites: the BrowseComp+ query, qrel and answer files on disk, a Chroma collection with corpus chunks whose document identifiers match the qrels, OpenAI credentials for the retrieval the harness uses, and optional Baseten reranker credentials if reranking is enabled.

Development and training is the third tier, with a plain environment for lightweight tests and two smoke scripts covering imports and the command line before anything heavier runs.

Credentials are handled with a template you copy only when needed:

cp .env.example .env.local

The scope of each key is documented one line at a time: a Hugging Face token only if checkpoint access needs auth, an OpenAI key for retrieval and evaluation, Chroma keys for backed evaluation, Baseten only for the optional reranker, and a Tinker key for Tinker-hosted paths. Real credentials stay out of the ignored environment files, and the repository's own smoke tests will tell you if the harness imports before you spend an evening on setup.

Repository layout, mapped to its claims

The layout mirrors the architecture claim, which is how you can tell the repository was organized by someone expecting readers. A harness directory holds the shared search harness with tools, trajectory, task, reranking and configuration modules; an inference directory holds evaluation, component ablations and vLLM utilities, with baseline runners for in-domain and transfer comparisons in a subdirectory; training carries SFT data generation, SFT, RL and launch scripts; and a model-export directory contains helpers for merging a private Tinker adapter into a Hugging Face model, which quietly documents how the checkpoint was produced.

Two more entries deserve note. A local Tinker cookbook is vendored as a dependency of the training scripts, pinning the training framework. And the tests are described exactly as lightweight import and CLI smoke tests, an honest description rather than a quality claim.

The licence is Apache-2.0, there are no tagged releases, and the last push was on 2026-06-15, so building on it means pinning a commit of a research artifact, with everything that implies about stability and about how much the published harness will move.

Who this is for, and what to weigh it against

The audience is precise: researchers studying long-horizon search agents and agentic reinforcement learning, and engineers evaluating whether structured search state plus a mid-scale model can substitute for a frontier model in their retrieval pipelines. The evaluation surface, ablations, in-domain and transfer baselines, published corpora, is provided for exactly that comparison.

The alternatives are the two poles the project positions between. Frontier hosted models with built-in search give stronger raw capability with no infrastructure, at per-query cost and with no visibility into the search state. Open search-agent frameworks around small models give structure without the trained policy, assembling prompting and tools by hand where Harness-1 learned the decisions. The bet here is that the trained 20B policy plus the externalized harness beats both on cost-control and inspectability respectively.

What to weigh: the GPU requirement is real, the retrieval backend is yours to provide, the numbers that matter are on BrowseComp+ in the paper rather than in the README, and a single-author research repository at version zero is a starting point, not a product. As a starting point, it is an unusually complete one: weights, data, corpora, scripts and export path, all published at once.

Editorial conclusion

Harness-1 fits researchers and engineers testing whether a 20B policy inside a stateful retrieval harness can do long-horizon search work that would otherwise go to a frontier model, with checkpoint, training data, corpora and ablation runners all released under Apache-2.0. It does not fit anyone without a CUDA GPU and a compatible retrieval backend, or anyone wanting a maintained product rather than a research artifact with no tagged releases. Verify first: that vLLM serves the checkpoint on your hardware, that your retrieval backend's document identifiers match the published corpora's, and the paper's BrowseComp+ numbers against your own queries before trusting the capability claim. The last push was on 2026-06-15.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. pat-jj/harness-1 on GitHub
4. README