FactCircuit: a verification loop that keeps its receipts

A verdict is not the same thing as a path

FactCircuit's pitch is that most verification systems return an answer, while this one preserves the path. The README lists what is preserved: immutable source versions, exact evidence spans, staged semantic checks, unresolved gaps, bounded retrieval tasks, stop reasons, hashes and resource usage.

That framing responds to a specific set of failure modes, and the README names them. URLs get revised. Syndicated stories look independent. A nearby number, date, negation or actor can attach to the wrong event. A retrieval loop can forget why it searched, reuse stale feedback, or leak documents published after the evaluation cutoff.

Each of those is a way for a confident answer to be wrong in a way a label cannot reveal. A system that says `true` gives you nothing to inspect. A system that says `true` and shows which revision of which document supported it, with the exact span, plus the tasks it could not complete, gives you something to argue with.

The implementation is described as an MIT licensed, adapter driven Python harness for building and evaluating auditable verification loops over changing news evidence. The standard library core runs offline. Model backed tracing runs through a local Codex login by default, with an explicit opt in OpenAI API route. That default matters: the interesting behaviour of the system, the decomposition and the evidence loop, can be exercised with no network at all.

The package metadata is honest about where the project sits. The development status classifier says Alpha, the README badge says research preview, and the intended audience is science and research.

The failure surfaces the design makes explicit

Seven properties carry most of the design, and each maps onto one of those failure modes.

Evidence is versioned rather than overwritten. Same URL revisions retain separate identities, with exact source spans, timestamps and content hashes, so a page that changed after you read it is a different object rather than a silently different answer.

Every returned item is decomposed. A document cannot influence the evidence graph merely because a retriever happened to find it. That is a direct attack on the failure where a search result gets treated as support because it was retrieved, and it is why decomposition appears in the pipeline before the critics.

Evidence and reality are judged separately. The question of whether this snapshot supports the claim is isolated from the question of whether the event happened in the world. These are different questions, and most conflations of them come from asking them at the same time.

Source lineage is distinct from semantic support. Ten copies of one wire story are not ten independent sources, and a contradiction does not identify the original publisher. So the graph tracks where a document came from separately from what it appears to say.

Uncertainty becomes work. Missing information is emitted as an exact fetch, search or reanalysis task and re-enters the same bounded loop, rather than being resolved by the model guessing or by the run ending quietly.

Bad state fails closed. Invalid schemas, ungrounded spans, duplicate probes, stale task receipts, cross event bindings and exhausted budgets are recorded instead of being partially committed.

Evaluation is a first class feature rather than an afterthought: fixed targets, cutoff checks, gold isolation, paired comparisons, bootstrap intervals and machine readable traces are part of the harness.

Two critics looking at the same document

The pipeline is easier to read as a sequence of narrowing stages than as a diagram.

It starts from a frozen claim and an evidence cutoff, and the first thing produced is a deterministic TargetPlan. From there retrieval runs, returning documents that are frozen into an immutable snapshot and then admitted against the cutoff. The cutoff admission step is what prevents the leak of material published after the evaluation date, which the README lists as one of the retrieval loop's failure modes.

Admitted documents become atoms, atoms become lineage, and a decomposition critic decides how the document actually decomposes. The output is assembled atomically in Python, which is the phrase to take seriously: the assembly is code, not a model call, so the structure of the evidence graph is deterministic once the atoms exist.

At that point the pipeline forks into two branches. One critic judges the evidence, asking whether this snapshot supports the claim. The other judges the world, asking whether the event happened. Both branches converge on a decision and an audit trace, and from there exact unresolved tasks feed back into the loop. The whole cycle is bounded by rounds, calls, output and time.

The fork is the architectural centre of the project. The separation between the two critics is what lets the system say that a document supports a claim while the underlying event is contradicted, or the reverse, and it is why the quickstart's expected result is a contradiction rather than a verdict.

The repository structure mirrors this decomposition. The `factcircuit/` package holds the harness, a `newsverify/` package sits alongside it with its own CLI entry point and bundled data, `examples/` holds the fixture files, `experiments/` and `integrations/` hold dated work, and `reports/` and `scripts/` cover output and tooling.

Installing the pinned release and running the offline quickstart

The quick start instructs you to begin with the pinned v0.3.2 release, and you need Git and Python 3.11 or newer. Installation downloads build tools, and the first example then runs entirely offline with no API key, model download or runtime dependencies. The package manifest confirms that last point: the runtime dependencies list is empty.

On macOS or Linux the sequence is:

git clone --branch v0.3.2 --depth 1 https://github.com/superwesleyhys-ux/factcircuit.git
cd factcircuit
python3 -m venv .venv
.venv/bin/python -m pip install .
.venv/bin/python -m factcircuit --version
.venv/bin/python -m factcircuit quickstart --output runs/first-run

The README gives the Windows PowerShell equivalent with `py -3 -m venv` and the `.venv\Scripts\python.exe` path form, so the same two commands work on both platforms.

What you should see is short:

factcircuit 0.3.2
Offline annotated example: contradicted; 3 rounds; 0 model calls.
Read runs/first-run/SUMMARY.md

The summary file lives in the output directory alongside `inputs.json`, `config.json` and `trace.json`, and four fictional evidence versions pass through three retrieval rounds, including source tracing and a publisher correction. The README is explicit about what this proves and what it does not: the semantic annotations are hand authored, so the run checks the working evidence loop, not model accuracy. Existing output directories are never overwritten.

That last detail is a small sign of care that matters for a tool whose whole point is reproducibility. A verification harness that silently overwrites a previous run destroys the record it was built to keep.

Tracing, scoring and comparing without a model

Everything in the offline command set runs in the local Python process with no API key, HTTP service, model SDK or remote inference. The source URLs are audit metadata and are never fetched.

The tracing commands work on documents you already have:

# Trace supplied snapshots; text is preserved, fact status stays not_checked
python3 -m factcircuit trace examples/local_trace.json --output reports/local-trace.json

# Every subcommand documents itself
python3 -m factcircuit --help

The note that the text is preserved while the fact status stays `not_checked` is worth understanding. Tracing a supplied snapshot does not verify anything; it exercises the decomposition and the evidence loop over evidence you already have, and leaves the judgement to a later stage. That separation is what makes the loop testable offline.

The evaluation commands are the part that turns traces into evidence about a system:

# Score fixed predictions against separate gold labels
python3 -m factcircuit score examples/evaluation_gold.json \
  examples/evaluation_predictions.json --output reports/all-metrics-local.json

# Paired comparison with event-cluster bootstrap intervals
python3 -m factcircuit compare examples/evaluation_gold.json \
  examples/comparison_baseline.json examples/comparison_candidate.json \
  --bootstrap-samples 100 --seed 0 --output reports/comparison-local.json

# Regression suite
python3 -m unittest discover -s tests -v

The compare command pairs a baseline against a candidate and produces bootstrap intervals with a fixed seed and a fixed sample count, which is what makes the comparison repeatable rather than a number that changes when you rerun it. The regression suite runs under unittest.

One caveat the README states plainly: the bundled predictions are deliberately imperfect handwritten fixtures for checking arithmetic, and they are not model performance results. The fixtures include a dev pool, head to head cases, early risk and double loop cases, and evaluation gold and prediction files, so a good deal of the harness's shape is visible in the examples directory.

Model backed tracing lives in the `trace-model`, `trace-news`, `early-risk` and `double-loop` subcommands, which run the same decomposition and verification adapters through one of three tunnels, with local the default and the OpenAI route available as an opt in extra.

Alpha status, an MIT licence and a fast moving line

The release history shows how quickly the project is moving. Version 0.3.2 was published on 2026-09-08, 0.3.1 on 2026-09-06 and 0.3.0 on 2026-09-06. The 0.3.0 release is labelled as a pre-rename release, and the README notes that the earlier Accuracy Tracing and NewsVerify Harness milestone is preserved in a separate repository, factcircuit-development-archive. The `newsverify/` package and its console entry point remain in this repository, so both names are present in the tree.

The last push was on 2026-10-09 and the repository is not archived. There is a CHANGELOG.md, a RELEASE_MANIFEST.json, a SECURITY.md, a CONTRIBUTING.md and a CLAUDE.md alongside the documentation directory.

The licence is MIT, declared both in the README badge and in the package metadata, and the package metadata lists the operating system as independent. The optional model extra pins `openai==3.8.0` and `socksio==1.0.0` exactly, which is the right choice for a package that may run with credentials in the environment.

The tooling configuration shows how the project wants to be worked on. Ruff is configured with a line length of 120 and target version py311, and the exclude list is instructive: dated experiment directories are treated as archived records of published runs and are kept as they ran, with only the reusable experiment modules linted. Preserving a published run unchanged is the same principle the harness applies to evidence, applied to the code that produced it.

What the project does not claim is as important as what it does. There is no accuracy number to quote, no dataset of news claims with a public scoreboard in the visible documentation, and the offline fixtures are arithmetic checks rather than evaluations. The value of FactCircuit is the shape of the audit trail, and a team that needs that shape will find it useful, while a team that needs a fact checking product will not.

Editorial conclusion

Adopt FactCircuit when your work needs to defend a verification decision later, because the immutable snapshots, exact spans, unresolved task list and machine readable traces are the artefacts an audit actually needs, and the evaluation commands turn that into reproducible comparison rather than an anecdote. Do not adopt it as a production fact checking service or as an accuracy claim, since the project labels itself a research preview and its classifier status is Alpha. Verify first the version you install, because the README directs you to the pinned v0.3.2 release, then confirm that the offline quickstart completes on your machine before enabling any model backed route, since the local tunnel is the default and the OpenAI route is an explicit opt in.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. superwesleyhys-ux/factcircuit on GitHub