Foreman: a decision model supervising a coding agent from above

The architecture: two loops that never have to stop each other

Foreman is a native Python asyncio runtime with two concurrent loops, and the diagram in the README draws the separation precisely.

The coding agent loop is reason, then tool, then observe, then edit, then test, and then continue. The Foreman loop is watch, then assess, then decision model, then decide as a Python policy, then intervene. The bridge between them is that worker output and lifecycle events flow into an independent, debounced observation loop while the subprocess remains active.

That last clause is the design constraint the whole project rests on. The worker does not have to stop working for the factory to think. That is what makes supervision cheap enough to be frequent, and it is why the observation loop is described as debounced: the supervisor needs to batch a burst of file edits into one assessment rather than asking a decision model after every write.

The diagram also shows the two directions. Factory evidence flows upward from the workers, and interventions flow back down into the test step, which then continues.

What Foreman explicitly does not do is worth reading closely, because it defines the scope. It does not replace a worker's reason, tool and observe loop, and it does not choose its individual tools or files. Missions stay broad. The intervention is steering, stopping or retrying at the job level, not micro-managing edits.

That is a coherent position, and it is a narrower one than a coding agent framework. Foreman is a supervisor, not a replacement.

What the decision model is actually asked to judge

The responsibilities tree in the README shows the scorecard the decision model produces, with example values for a particular run.

There are five groups. Completion has implementation_complete, requirements_satisfied and ready_to_finish. Verification has tests_sufficient and needs_verification. Worker health has worker_stuck, work_off_track and meaningful_progress. Repository instructions has agents_md_drift. Human escalation has needs_human.

The values in the diagram are the interesting part, because they are not all near the same end. In that run, implementation_complete reads .91 and requirements_satisfied .79, but ready_to_finish is only .21 while needs_verification is .82 and tests_sufficient is .34. So the model was fairly confident the code existed and roughly met the brief, and simultaneously unconvinced that the evidence was good enough to call it finished.

That combination is exactly the failure mode a human reviewer worries about, and a probability per responsibility is a better interface for it than a single pass or fail verdict. The routing decision then maps those probabilities onto continue, steer, stop or retry, and then verify or finish.

agents_md_drift is the one that catches the eye, because it is about the project instructions themselves rather than about the work. A worker that is quietly ignoring the repository's agent instructions is a problem no amount of code review would necessarily surface.

The output of the whole thing is a decision, and the README is clear that the decision model is only half of it. The other half is the Python policy, and the responsibility definitions live in TOML files under the package data for the foreman.responsibilities package.

Why a decision model at all, rather than another language model call

The rationale section is the part of the README that explains the bet rather than the feature list, and it is worth reading because the project is a research question as much as a tool.

The argument starts with a classification. Coding agents are relatively slow, stateful generative systems. Supervisory questions such as is this worker stuck, or does this now need independent verification, are narrower. Decision models accept evidence and typed questions and return probabilities.

So the claim is that the supervisory question is a different shape of problem from the coding question, and deserves a different interface. A decision model returns calibrated probabilities against typed responsibilities rather than free text, which means the routing logic in Python can threshold on a number instead of parsing a sentence.

The consequence is that the provider adapter is replaceable independently of the worker backends and the Python policy. Three layers, three independent seams.

The project supports TypeSafe AI's Jev and OpenAI's Decisions API, and there is a separate document on using Jev through OpenRouter, with the base URL override documented in the environment example. The tagline the project uses is generative models work, Foreman watches the work, which is the whole thesis in six words.

There is also a document titled what Foreman is proving, and the README states the effectiveness of the whole thing depends on the quality of the configured checks, evidence, thresholds and underlying models. That is an unusually honest framing for a project at version 0.5.

Worker backends are pluggable, and the App Server transport is what makes steering possible

Foreman currently runs one coding worker at a time, and the worker implementation is replaceable because the runtime depends on a small worker protocol rather than Codex-specific types.

The backend is selected with FOREMAN_WORKER_BACKEND, with codex as the default, plus opencode, hermes and deepagents. Running a job with a different backend is a one-line change:

FOREMAN_WORKER_BACKEND=opencode foreman run --repo ./my-project --job "Add request retries"

Each backend adapts differently. The OpenCode backend runs opencode run non-interactively and streams its output like the Codex exec backend. The Hermes backend runs hermes chat in headless single-query mode and decodes its newline-delimited JSON events into tool-activity summaries. The Deep Agents backend expects a separately installed dcode CLI.

The default Codex backend is the interesting one, because by default a real worker is a Codex App Server thread and turn launched over its JSONL protocol:

codex app-server --listen stdio://
thread/start → turn/start → turn/steer or turn/interrupt

That transport is what makes live steering possible. The App Server transport keeps the active Codex thread addressable, allowing Foreman to send a supervisory update into an in-flight turn. The previous stable codex exec transport is still available through FOREMAN_CODEX_BACKEND=exec, but it cannot accept live steering.

So if you want to interrupt a worker mid-task with a correction, you need the App Server path. That is a real dependency on a specific Codex transport, and it is the one place where the pluggability has a practical edge.

Live steering, verifiers, and the environment variables that control them

The intervention side of the system has its own configuration, and the environment example shows how much of it is tunable.

Live steering uses Codex App Server by default, and the environment file records that exec is the non-steerable fallback. The steering switches are FOREMAN_STEERING_ENABLED, FOREMAN_MAX_STEERS_PER_WORKER and FOREMAN_STEERING_GRACE_SECONDS, with a default of one steer per worker and a grace period of 30 seconds. A grace period is the sensible design: a supervisor that reacts instantly to every intermediate state would steer a worker that was about to finish correctly.

A verifier is another worker, using the selected backend with an independent, deterministic verification mission. That is a separate agent doing the checking, not the same agent grading its own homework, and it explains the diagram line where tests feed factory evidence upward.

The decision provider side has its own block. FOREMAN_DECISION_PROVIDER is jev by default or openai, with FOREMAN_DECISION_MODEL, FOREMAN_DECISION_TIMEOUT_SECONDS and a TYPESAFE_API_KEY that you create at the typesafe console. The comment notes that OPENAI_API_KEY is also accepted but may be shared with worker models, which is a small but real warning about mixing credentials.

The Deep Agents block sets FOREMAN_DEEPAGENTS_EXECUTABLE, FOREMAN_DEEPAGENTS_MODEL, FOREMAN_DEEPAGENTS_MAX_TURNS and FOREMAN_DEEPAGENTS_SHELL_ALLOW_LIST, with a recommended value for the allow list, which is a sandbox control for a backend that can run shell commands.

The codex worker settings are separate from the decision model settings, with FOREMAN_CODEX_EXECUTABLE and FOREMAN_CODEX_MODEL. That separation is deliberate and named in the comment: independent of FOREMAN_DECISION_MODEL.

Packaging, dependencies and the shape of the documentation set

The Python package is foreman-core at version 0.5.1, requires Python 3.11 or newer, and is MIT licensed with the classifier list covering 3.11, 3.12 and 3.13. The console entry point is foreman = foreman.cli:app, and the project homepage is thruwire.ai.

The runtime dependencies are small and tell you the domain. typesafe-sdk for the decision provider, typer for the command line, pydantic for the typed evidence and configuration, rich for terminal output, keyring for credential storage, and python-dotenv for the environment file. The OpenAI provider is an optional extra, along with a dev group carrying pytest, pytest-asyncio and ruff.

The keyring dependency is a small decision with real consequences. It means API keys can come from the system credential store rather than sitting in a dotenv file, which is the right default for a tool that handles several providers' credentials.

The test and lint setup is minimal but strict. pytest runs with asyncio_mode set to auto, so async tests need no decorator, and ruff is configured with a line length of 100, target version py311, and a rule selection of E, F, I, UP, B and ASYNC. The ASYNC group is the interesting one, since it flags blocking calls inside async code, which in a runtime built around two concurrent loops is the failure mode most likely to cause a stall.

The documentation set is extensive and is itself a signal of how much of this project is a proposal. There is a theory document on semantic supervision, one on decision models and provider adapters, one on why Jev fits, one on what Foreman is proving, one on the runtime and event flow, one on evidence providers, one on responsibility configuration and routing, one on extensions, one on coding-assistant hooks and attached workers, one on live steering, and one on worker backends. There are also integrations/ directories for the Deep Agents plugin and for Pi and Pi Durable, and a marketplace plugin for Codex.

Editorial conclusion

Adopt Foreman if you are already running coding agents unattended and want an independent check on whether the work is actually done, since the separation between the worker loop and the supervision loop is the whole idea. Do not adopt it expecting a general autonomous coding system, because Foreman explicitly does not choose the agent's tools or files and currently runs one worker at a time. Before you configure it, decide which decision provider you will use, Jev by default or the OpenAI Decisions API, and treat the thresholds and evidence providers as the real configuration work, because the documentation states its effectiveness depends on the quality of those.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. thruwire/foreman on GitHub