Research Harness: file-first research pipelines for agent-assisted work

The problem Research Harness addresses: research that leaves no trail

A long agent-assisted research task can end with a polished PDF and still leave basic questions unanswered. Which sources support this paragraph? What changed after the last failure? Can the work resume tomorrow without reconstructing a chat? What did PASS actually verify? The README states the project's premise directly: research should leave a trail, not just an answer.

The intended user is someone running research through a coding agent, in Codex or Claude Code, where the default output is a final answer plus a long conversation. Research Harness reorganizes that into a file-first, recoverable Run: a Goal, a Workflow, pinned Pipeline contracts, recoverable Units, research Artifacts, completion checks, Run Evidence, and bounded diagnosis. The README is explicit that this is not an autonomous-scientist claim. It is infrastructure for making agent-assisted research inspectable, resumable, and honest about what has and has not been proven.

That framing matters for adoption. If you want a tool that produces research, this is the wrong shape. If you want a tool that records how research was produced and refuses to continue when the rules change underneath it, the design is aimed at you.

How a Run is structured: Goal, Workflow, pinned Pipeline, Units, Artifacts

The data flow in the README is a single line: Goal -> Run -> Evidence -> Artifact. The expanded flow adds the ownership at each transition: Goal to Workflow, Workflow to a pinned Pipeline contract, Pipeline to recoverable Units, Units to research Artifacts, Artifacts to completion checks, checks to Run Evidence, and Evidence to bounded diagnosis that loops back to repair and rerun.

Three mechanisms carry that structure. First, the contract is pinned. harness-lock.v2 snapshots the selected Pipeline and hashes its inheritance bundle, Skill implementations, and Harness Kernel. An active Run fails closed if the Pipeline or Kernel drifts, so it cannot silently continue under different rules. Second, completion is evidence-backed: a DONE cell alone is not success. The Attempt, required outputs, Artifact hashes, Workflow checks, Manifest, and Completion Event must agree. Third, failure has an address: Doctor, Audit, scorecards and the Failure ledger distinguish an observable defect from its owning repair surface, and improvement diagnoses rather than rewriting the Harness in place.

Human checkpoints follow the same discipline. Approval is bound to the reviewed Artifact hashes, so changing an approved outline, scope, or protocol revokes the stale authorization. That is a real constraint, not a formality: you cannot approve an outline, edit it, and keep the approval.

The workspace layout reflects the same priorities. GOAL.md holds the requested outcome and constraints, UNITS.csv the explicit plan and current Unit state, DECISIONS.md the human checkpoints and choices, papers/ and outline/ the research evidence and intermediate structure, output/ the deliverable plus scorecards, audits and repair reports, and .harness/ the Run identity, Attempts, Events, hashes and provenance.

Installing Research Harness from a source checkout and running research-brief

The README states that Research Harness currently runs from a source checkout with Python 3.10+ and uv. There is no published package install documented; the entry point is the rh script declared in pyproject.toml as tooling.product_cli:main. Start by cloning and syncing the locked dependency set:

git clone https://github.com/WILLOSCAR/research-units-pipeline-skills.git
cd research-units-pipeline-skills
uv sync --locked

uv sync --locked installs the dependencies pinned in uv.lock, which include pypdf>=4.0 and PyYAML>=6.0. Next, create a goal and bind it to a workflow. The README's example uses research-brief, which produces output/SNAPSHOT.md and is meant for understanding a topic and deciding what to read:

uv run rh goal create \
  --goal "Understand test-time adaptation for robotics and decide what to read" \
  --workflow research-brief \
  --workspace workspaces/robot-adaptation

uv run rh run start --workspace workspaces/robot-adaptation

The Run advances until it finishes or reaches an unmet prerequisite. For research-brief, the README says to inspect the paper set, taxonomy, outline and C2 review block, then continue. Status, approval and resume are separate commands, and the evidence trail is inspected with its own subcommand:

uv run rh run status --workspace workspaces/robot-adaptation
uv run rh run approve --workspace workspaces/robot-adaptation --checkpoint C2
uv run rh run resume --workspace workspaces/robot-adaptation
uv run rh evidence inspect --workspace workspaces/robot-adaptation --excerpt

After that, the workspace contains the readable deliverable and its evidence trail. If a contract fails, the README points to a diagnosis command rather than a manual edit:

uv run rh improve diagnose --workspace workspaces/robot-adaptation

One practical note on the activation surface: in Codex or Claude Code the README shows a single sentence instead of a command, for example "Use research-brief to map test-time adaptation for robotics and tell me what to read first." The seven executable Pipeline contracts are research-brief, paper-review, evidence-review, arxiv-survey, arxiv-survey-latex, idea-brainstorm and source-tutorial. graduate-paper remains a research-stage Chinese thesis path, not one of the seven.

What a PASS actually proves, and what it does not

The README separates three claims that are easy to blur. Execution integrity means Attempts, state, Manifests, hashes and provenance agree. It does not establish that the answer is good. Contract acceptance means required Artifacts satisfy observable Workflow checks. It does not establish scientific truth or exhaustive retrieval. Research quality means usefulness and correctness on realistic inputs, and it does not establish validity beyond the evaluated cases.

The repository implements the first two layers. The third, by the project's own statement, needs repeated Runs, held-out evaluation and expert judgment. Reports use qualified evidence rather than turning every green check into a research-quality claim.

This is the most important thing to understand before adopting it, because the naming invites the wrong reading. A green check in this system is a statement about the process, not about the finding. If your team treats a passing Run as a validated result, you have misread the tool, and the README says so in advance.

The survey residue failure and why the gate exists

The README documents a concrete failure that shaped the current gate. The Survey writer can bootstrap provisional prose from structured evidence packs and versioned templates. Early versions completed the delivery path but left too much of that scaffold in the paper: the historical course-paper sample matches template fragments in 96/140 sentences, or 68.6%.

That number is worth pausing on. The pipeline finished, the delivery path was satisfied, and the output was still largely template residue. This is exactly the hollow-writing failure that evidence-first methodology is supposed to prevent, and it survived a working pipeline. The README states that this failure is now a cont, with the sentence truncated in the available text, so the precise gate mechanism is not documented here.

The example directories hint at the response. examples/course-paper-residue-pass/ sits alongside examples/course-paper-pilot/, and there are typed-engine proof examples for research-brief and paper-review. The project appears to keep regression cases for the failure it fixed, which is the right instinct. I would still want to read the full gate description in the repository before trusting the fix, because the available README truncates it.

Input boundaries: where Research Harness refuses to work

The README describes the input boundaries as intentional. paper-review will not invent a manuscript. source-tutorial will not invent a source pack. evidence-review writes a protocol and pauses for approval before retrieval. Each workflow has a required starting point: research-brief needs a topic, paper-review needs a manuscript, evidence-review needs a review question, arxiv-survey and arxiv-survey-latex need a topic plus delivery constraints, idea-brainstorm needs a topic and scope, and source-tutorial needs a source pack and an audience.

These are failure modes if you arrive with the wrong expectation. If you have no source pack, source-tutorial is unavailable, and the README points to the usage guides in readme/README.en.md for those setup paths rather than offering a fallback. If you want a review of a paper you have not supplied, paper-review will not proceed.

There is also a maintenance consideration. The optional test extra declares pytest>=8.0 and ruff>=0.9, and the repository carries a verification workflow at .github/workflows/verify.yml. The last push was on 2026-09-15. The README does not document a rollback procedure for a Run, and it does not document what happens to in-flight Runs when the Harness Kernel is upgraded, beyond the fail-closed behaviour on drift.

How it differs from a plain agent workflow or a notebook

The closest everyday alternative is running the same research inside a coding agent with no harness: you describe the goal, the agent works, and the result is a final answer plus a conversation. The difference is not quality of prose, it is where the state lives. In the plain-agent case the state is the chat history, and resuming tomorrow means reconstructing context. In Research Harness the state is on disk: GOAL.md, UNITS.csv, DECISIONS.md, papers/, outline/, output/ and .harness/. Resumption is a command, not a reconstruction.

A second alternative is a notebook or script pipeline you write yourself. That gives you file-based state and reproducibility, and for a fixed analysis it is often the better tool. What it does not give you is the pinned contract. harness-lock.v2 hashes the Pipeline, its inheritance bundle, Skill implementations and the Kernel, and an active Run fails closed on drift. A hand-written script will happily run to completion after you edit a helper function, and nothing in the output will tell you the rules changed. That is the specific gap this project targets.

The trade-off is ceremony. You get UNITS.csv, DECISIONS.md, checkpoints like C2, and a diagnosis step before repair. For a two-hour question, that is overhead with no return. For a multi-day review where someone will later ask which sources support a paragraph, the overhead is the product.

Editorial conclusion

Adopt Research Harness if you are running long, multi-step literature work with an agent and you need the trail: who approved which outline, which sources back which paragraph, and where a failure belongs. Skip it if you want a one-shot summary, if your work has no file-based intermediate artifacts, or if you cannot accept that the repository implements execution integrity and contract acceptance but not research quality. Before committing, clone the repository, run uv sync --locked, execute the research-brief example against your own topic, then open .harness/ and confirm that the Attempts, Events and hashes it writes are the ones you would want to audit six months later.

Frequently asked questions

Official sources

1. Issues
2. README
3. WILLOSCAR/research-units-pipeline-skills on GitHub