coleam00/ai-software-factory: GitHub issues in, merged pull requests out

What an issue-to-merge factory actually replaces

The README frames the project as a dark factory, borrowing the lights-out manufacturing term: work goes in as an issue, validated code comes out, and no human sits at the keyboard in between. It is aimed at people who already have a repository and a product document, not at people looking for a greenfield generator. The README says the repo holds either an application or only a PRD, and the interview is the same either way because a PRD answers what the code would have.

The claim worth examining is the second sentence of the README: the automation is the easy half, and being able to trust a merge nobody read is the hard half. Most of what ships in this repository exists for that second half. That is a different product from an agent that opens a pull request for you to review. Here the merge assessor is supposed to be something that did not write the code, and the repository's job is to give that assessor enough context to say no.

The three files that decide whether the gate holds

The installer writes factory and project templates and installs a pinned Archon source, but it cannot write the files that describe your product. The README names three: MISSION.md, harness/END-TO-END.md and .factory/holdout/HOLDOUT.md. The agent is told to read the repo first, then ask at most four questions with proposed answers already filled in, so the cheapest reply is yes.

The out-of-scope list in MISSION.md is the load-bearing part. The README says it is how the factory recognises that a plausible, well-argued, easy request is drift rather than a good idea, and it tells the agent to propose seven entries and let the user strike the wrong ones. That ordering matters: asking a user to invent an out-of-scope list from nothing produces a list of things nobody wanted anyway, which catches nothing.

harness/END-TO-END.md holds two to five journeys in plain English, and the constraint on them is strict. Journeys describe what the product does today, never what it should do. A journey written for behaviour that does not exist leaves the gate red before the first lap, and the README states the consequence plainly: nothing can merge, including the change that would make it pass. The PRD case is carved out as the exception, because a PRD is the one document that describes behaviour before the code exists.

The holdout file gets the most honest line in the README. A directory called holdout does not make it private, and the agent is told to arrange actual isolation from the builder if the scenarios are meant to be hidden. Anyone treating the folder name as a security boundary has misread it.

How Archon workflows carry every agent step

The architecture is deliberately thin on this side. All AI work runs through Archon's SDLC pack, and this repository supplies setup, scheduling, project context and ordinary app resources. The installer brings its own pinned Archon source, so you do not check out Archon or merge its pull requests yourself. The default pins exact revision a02b9ab6bc59f4aff6cae8bfca4723b3eadadb28, whose SDLC pack the README lists as supporting grounded intake, reviewed delivery, scoped validation reuse, independent runtime and holdout verification, bounded repair, discovery handling, merge queues and deterministic deployment.

The instruction to the agent is explicit that factory scripts must not add direct coding-agent subprocesses and must not create factory-local workflow copies. That is a governance choice as much as a technical one: one workflow definition, one place to change it, and no drift between what the factory claims to run and what it actually runs. It also means the factory inherits Archon's revision cadence rather than setting its own.

One file bridges the two systems. The installer adds a pointer to factory/WORKFLOW_POLICY.md in the project's native AGENTS.md without replacing existing guidance. The README describes that policy as supplying the factory-specific bootstrap, runtime, review and state-label requirements that the general-purpose shared workflows intentionally do not assume. If you already maintain AGENTS.md, expect an addition rather than a rewrite, and check the diff anyway.

Installing it and getting one issue through the factory

The documented install is not a package install. The README gives a block of text to paste to your coding agent inside the repository where you want the factory, and that prompt tells the agent to read the README, run the installer, help write the mission and validation journeys, configure Archon, and leave scheduling off until one issue has gone through. Prerequisites are git, Python 3.10+, an authenticated gh CLI with the workflow scope, a GitHub remote, a coding agent, bun and uv. The README notes the agent can help install missing tools.

Authentication first, because git itself needs to push, including CI workflow files:

gh auth login -s workflow
gh auth setup-git

Then clone the factory outside the application repo, or update an existing clean clone with a fast-forward-only pull:

git clone https://github.com/coleam00/ai-software-factory ~/ai-software-factory

Run the installer from the root of the application repo, not the factory clone. Use python3 if that is the Python command on the machine. If the repository lives on a server, the README says to run installation and configuration there over SSH:

python ~/ai-software-factory/bin/factory.py init

The installer writes the templates and installs the pinned Archon source. What it does not do is stated just as clearly: it does not configure provider authentication, change model tiers or start a schedule. It preserves project configuration on upgrades. After it runs, configure your provider through native Archon settings. For Claude Code the README offers Sonnet for small and medium work and Opus for large as a starting point, with the caveat to use model identifiers supported by the installed provider and your account.

Before the first ticket, the README describes a calibration you are meant to run: baseline verified, a relevant deliberate fault failed, and a wrong-identity check inconclusive. That third outcome is the interesting one. A wrong-identity check that passes is a sign the harness is not discriminating, not a sign of quality.

Where this breaks, and who should not adopt it

The failure mode the README spends the most words on is a red gate that cannot clear itself. If you write journeys for behaviour the product does not have yet, nothing merges, and that includes the change that would add the behaviour. The escape hatch is the PRD case, where the repo holds only a product document and the agent writes all three files plus harness commands from it. If you are somewhere in between, with a half-built app and a roadmap, you are in the worst position for this tool: too much code for the PRD path, too little verified behaviour for the journey path.

The second limitation is that the installer does not finish the job. It leaves provider authentication, model tiers and scheduling to you. The README's own instruction is to leave scheduling off until one issue has been watched through the factory. Anyone expecting a single command to produce a running autonomous pipeline will stop at step two.

The third is the holdout isolation problem, which the README raises rather than solves. Scenarios in .factory/holdout/HOLDOUT.md are only independent if you arrange isolation from the builder yourself. Nothing in the installer enforces that separation. If the same agent can read the holdout and write the code, the holdout is a second set of tests with a more impressive name.

Finally, the repository states that native live acceptance remains separate from installation readiness. A successful init is not evidence that the factory can ship anything. It is evidence that the files landed.

How it differs from a CI pipeline with an agent bolted on

The closest thing most teams already run is a CI pipeline plus an agent that opens pull requests, with a human reviewer as the gate. The difference in approach is where the judgement sits. In the CI-plus-agent setup, the reviewer is the assessor and the pipeline checks mechanical properties. Here the assessor is a separate AI step that did not write the code, and it is reading against MISSION.md, the journeys and the holdout rather than against a style guide.

That changes what you have to author. A CI pipeline needs a test command and a branch protection rule. This factory needs a written statement of what the product must never become, because the assessor has no other way to classify a well-argued request as drift. The README treats the out-of-scope list as the deciding artifact, and that is consistent with the design: an assessor with no negative definition will approve anything plausible.

The other structural difference is the scheduled regression. The README says a scheduled regression can re-test what already merged and publish bugs it finds. In a normal pipeline, merged code is done. Here it stays under test, which is a materially different operating model and the reason the README tells you to keep scheduling off until you have watched one issue complete the loop.

Editorial conclusion

Adopt it if you already run a repo with a real test command and you are willing to write MISSION.md, harness/END-TO-END.md and .factory/holdout/HOLDOUT.md before the first ticket. Do not adopt it if you want the factory to decide what the product should be, or if your journeys describe behaviour that does not exist yet, because the gate stays red and nothing merges. Before you file anything, run bin/factory.py init, confirm the pinned Archon revision a02b9ab6bc59f4aff6cae8bfca4723b3eadadb28 landed, and check that the deliberate-fault calibration actually fails.

Frequently asked questions

Official sources

1. coleam00/ai-software-factory on GitHub
2. Issues
3. README