Shepherd: reversible execution traces for agent runs

The problem Shepherd takes on: agent work you cannot inspect or undo

Most agent frameworks hand you a final answer. The intermediate work, the files the agent touched, and the decisions it made along the way are gone by the time the process exits. If the result is wrong, your only recovery is to run the whole thing again and hope for a different outcome.

Shepherd is built for the case where something else is watching the agent. Its topics list names meta-agents, runtime supervision, meta-optimization, MCTS-RL and tree-RL, and the README describes a runtime substrate for agent work that needs inspection, reversibility and supervision. The audience is not someone writing a first chatbot. It is someone building a second agent whose job is to supervise, optimize or train a first one, and who therefore needs the first agent's execution to be addressable after the fact: a run you can fork, replay, revert or hand to a reviewer.

The README frames the unit of work as a task that returns a reviewable proposal. Nothing touches your files until you accept it. That single design choice is what makes the rest of the system coherent: if the agent cannot write into your working tree, then the trace is the only record of what it did, and the trace has to be good.

How the trace, the retained output and the permission surface fit together

A Shepherd task is a plain Python function with no body. The signature and the docstring are the contract the agent fulfils at runtime. That is an unusual inversion: the type annotations are not decoration, they are the permission surface. The README states that repo: sp.GitRepo is an explicit read-write workspace-handle grant, that May[GitRepo, ReadOnly] marks a task that must inspect without mutating, and that an unannotated repo parameter is just an ordinary value parameter rather than a handle. Grants are declared per bound repository in the signature.

Enforcement is at the OS level, not in Python. The README says grant enforcement runs on macOS through Seatbelt and on Linux through Landlock inside a privileged container. That distinction matters: a read-only grant is refused by the kernel, so it holds even if the agent's own code tries to write. It also explains the platform story. Windows is unsupported, and the README is direct about why, saying enforcement there would be advisory-only at best, and pointing users to WSL.

When a run finishes, its output does not land in your directory. It becomes a retained output, described in the README as a proposal held safely to one side. From there you can run it without applying it, or select, apply or discard it. Apply is the interesting verb, because the README describes merging the retained output onto a workspace that moved on while the agent was working. The trace remembers the decision either way, according to the README, which is what makes the record useful to a meta-agent later.

Installing Shepherd and running the offline quickstart

Shepherd publishes to PyPI as shepherd-ai and requires Python 3.11 or newer. The README gives a single install line for users, and a separate editable closure for people working on Shepherd itself.

pip install shepherd-ai

The offline quickstart needs no API key. It exercises the same retained-output machinery through Shepherd's deterministic provider, which the README describes as the agent lane minus the agent. Create a scratch directory, turn it into a workspace, then fetch and run the demo.

mkdir /tmp/shepherd-quickstart && cd /tmp/shepherd-quickstart
shepherd init
shepherd demo write quickstart > quickstart_demo.py
python quickstart_demo.py

shepherd init turns the directory into a Shepherd workspace, and the demo script registers and runs a task, retaining its result. To see what happened, list the runs and then read the changeset.

shepherd run list
shepherd run changeset --latest
shepherd run show --latest

The first command shows the run and its status, the second shows what the run wrote as a retained output, and the third prints the full record. The README notes that adding --json to any read command gives the durable machine payload, which is the form a meta-agent would consume. Backend selection and the rest of the run surface are in the docs rather than the README.

The agent-backed lane needs the claude CLI, either signed in or with an ANTHROPIC_API_KEY set. On a subscription, the README recommends exporting a long-lived token, because a short-lived signed-in session cannot be refreshed from inside the sandbox and may work interactively yet fail under Shepherd.

export CLAUDE_CODE_OAUTH_TOKEN=$(claude setup-token)
shepherd doctor claude --probe

The doctor command reports which credential you have, and --probe performs a real auth round-trip under Shepherd's config in the parent process, not a jailed run. The README also warns that an HTTP 403 from Claude is an account or organization policy limit rather than a login problem, and that a hung claude CLI surfaces as a budget timeout rather than an auth error.

Where Shepherd is the wrong tool

The alpha warning is the first real constraint. The README states that Shepherd is in early alpha, under active development, and that APIs may still change between releases. The last push to the repository was on 2026-09-09, so the project is moving, which is exactly why the API surface is not something to build a long-lived integration against without pinning.

Windows is a hard boundary, not a rough edge. The README says enforcement would be advisory-only there and directs users to WSL. If your build fleet is Windows containers, Shepherd's permission model does not apply to you in the way the README describes.

Linux enforcement carries its own cost. Landlock enforcement runs in a privileged container, per the README. A privileged container is a meaningful thing to ask of a CI system or a shared host, and some environments will not grant it. On macOS the mechanism is Seatbelt, so the two platforms are not equivalent in how the grant is enforced even though the Python-level spelling is the same.

The agent lane also inherits the failure modes of the CLI it wraps. The README is unusually candid here: org-policy 403s, stale CLI hangs that look like budget timeouts, and sandbox refreshes that fail on short-lived sessions. None of these are Shepherd bugs, but they land in Shepherd's error surface, and a meta-agent that supervises runs has to distinguish them. If you want a plain sandboxed code runner with no trace and no review step, the retained-output model is overhead you would be paying for and not using.

Shepherd against Docker commit as a fork mechanism

The obvious alternative for capturing an agent's environment is a container snapshot: run the agent in a container and commit it when you want to branch. Docker commit is well understood and works on any platform Docker runs on, including Windows, which Shepherd does not support. It also gives you a real image you can push, share and run elsewhere.

The difference is in what gets captured and how fast. Shepherd couples the agent and its environment in a copy-on-write fork, which the project describes as roughly five times faster than docker commit, with around 95% KV-cache reuse on replay. Those are the project's own figures and I have not reproduced them. The structural point stands regardless of the exact numbers: a container commit serializes a filesystem layer and produces an image, while a copy-on-write fork keeps the parent's memory and cache pages shared and only diverges on write. For a meta-agent that wants to explore many branches of the same run, that is the difference between branching being a background operation and branching being the bottleneck.

The second difference is the review step. A docker commit gives you an image; you still have to decide what to do with it. Shepherd's retained output is a proposal with select, apply and discard as first-class commands, and the trace records which one you chose. If your workflow already has a review process, a committed image slots into it. If you are building the review process itself, Shepherd has opinions about it already.

Licence, maintenance and the cost of upgrading

Shepherd is MIT licensed, which is permissive: you can use it commercially, modify it and redistribute it, provided the copyright notice and permission notice are preserved. That is the standard reading of MIT and not legal advice; if the licence terms matter to your organization, read the LICENSE file in the repository rather than a summary.

The repository is not archived and the last push was on 2026-09-09. The version in pyproject.toml is 0.3.1, matching the most recent release, and the release notes describe v0.3.1 as adding citation checking as a packaged workflow. The two prior releases, v0.3.0 and v0.2.1, landed in July 2026, and v0.3.0 is described as completing the settlement vocabulary. That sequence is worth reading as a signal about the API: a vocabulary still being completed at 0.3.0 means names in the public surface were still being settled two months before the latest release.

Upgrade cost therefore concentrates in the Python API rather than in the runtime. The workspace is a uv workspace with many member packages (shepherd-core, shepherd-runtime, shepherd-providers, shepherd-sandboxes, shepherd-authoring, vcs-core, commons-vcs and others), and the top-level pyproject.toml pins the workspace together. If you depend on Shepherd, pin the version and read the release notes before moving, because the alpha warning in the README explicitly covers API change between releases. The integration-tests directory and the Makefile's per-package test targets are where a contributor would go to see whether an upgrade broke something, but that is a contributor's workflow, not a consumer's.

Editorial conclusion

Adopt Shepherd if you are building a meta-agent that has to supervise, fork or replay other agents and you can live with an alpha whose APIs may change between releases. Do not adopt it if you need Windows, if you want a stable API today, or if you are only after a sandboxed code runner: the trace, the retained-output review step and the permission surface are the product, and you would be carrying all of it for one feature. Before you commit, run shepherd doctor claude --probe to see which credential your machine actually has, and read the permission section of the README to confirm that May[GitRepo, ReadOnly] and May[GitRepo, ReadWrite] map onto the enforcement you expect on your platform.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. shepherd-agents/shepherd on GitHub