# Hypoarena turns hypothesis pipelines into a testable mechanism layer

> Hypoarena is an offline Python workbench that decomposes generate, debate and evolve pipelines into separately testable stages, from synthetic corpora with planted causal chains to Elo tournaments and Bayesian accumulation. It proves the machinery works on synthetic data, and it is explicit that it says nothing about real scientific discovery.

**OpSafari/hypoarena** — Scientific hypothesis-discovery workbench: grounded hypothesis-evidence graphs, synthetic literature with planted causal chains, generate-debate-evolve loops over pluggable offline adapters, Elo tournaments recovering planted skill order, paraphrase dedup, Bayesian evidence accumulation, reproducible reports. NumPy core, CPU-only torch extra.

- Repository: https://github.com/OpSafari/hypoarena
- Stars: 541 · Forks: 29
- Language: Python
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/opsafari-hypoarena

## The pipeline is split into ten CLI stages that run in order

Hypoarena is a Python package, and its structure is visible in the command line tool rather than in a class diagram. The `hypoarena` command exposes one subcommand per pipeline stage, in pipeline order: `corpus`, `generate`, `verify`, `dedup`, `debate`, `rank`, `evolve`, `accumulate`, `report`, and `demo`. The console script is declared in pyproject.toml as:

```toml
[project.scripts]
hypoarena = "hypoarena.cli:main"
```

The package is installed from a source tree under `src/hypoarena`, which the build configuration points at directly:

```toml
[tool.hatch.build.targets.wheel]
packages = ["src/hypoarena"]
```

That one-subcommand-per-stage shape is the design decision that matters, because it means a stage can be run, inspected and re-run on its own rather than only as part of a single opaque loop. The README describes the package as a fully offline scientific hypothesis-discovery workbench that decomposes the mechanism layer of generate, debate and evolve pipelines into individually testable components, and the CLI is where that decomposition becomes usable. The alternative to this design is the full generate, debate and evolve pipeline the project refers to as AI co-scientist work, where agents call real models over real literature and the output is a set of candidate hypotheses for a person to judge. The difference in approach is not the loop shape, which is the same, but where the ground truth comes from. A model-backed pipeline has no planted causal chain to recover and no known skill order to recover, so its quality can only be argued about in prose, while a hypothesis returned by hypoarena can be scored against an answer the corpus generator put there on purpose. What hypoarena cannot tell you is whether the loop would survive contact with a real model, since its agents are scripted, replayed or loopback mocks. The two approaches are complementary in practice, because a mechanism layer that can be tested offline is the only way to check whether a change to the loop broke anything before spending a model budget on it.

## Installing it means a local venv and an editable pip install

The package is pure Python and the install is the editable kind, so a contributor works against the source tree in `src/hypoarena` rather than a published wheel. The README gives two commands, the first for the development environment and the second adding the optional CPU-only torch extra when the small trainable ranker is needed:

```bash
python -m venv .venv
.venv/bin/pip install -e ".[dev]"          # development env (hatchling, pytest, ruff, mypy)
.venv/bin/pip install -e ".[dev,torch]" \
    --extra-index-url https://download.pytorch.org/whl/cpu   # when the optional ranker is needed
```

The dev extra is where the tooling comes from, and the project pins it in pyproject.toml as build, hatchling, mypy, pytest and ruff. The first real use is one command:

```bash
.venv/bin/hypoarena demo --chains 2 --chain-length 2 --out /tmp/hya
```

That runs the complete pipeline offline over a synthetic corpus, prints whether each planted causal chain was recovered, and writes `report.md` and `report.html` to the output directory. A reader should expect a report that ends with an explicit note that this is a synthetic demonstration only. The same run is available through the Makefile, whose `demo` target invokes the module directly with the local venv interpreter.

## The corpus stage plants the causal chains it later asks you to recover

The corpus stage generates a deterministic synthetic literature with planted causal chains, competing hypotheses, paraphrase clusters and controllable noise in the form of distractors and contradictory findings. Every corpus carries a provenance hash. Determinism here is the point rather than a side effect. Once a chain of cause and effect is planted into the corpus, the rest of the pipeline can be scored against a known answer, and the documentation says that a typical run on the demo corpus reports every planted link, with the sample line reading `planted links: 6  recovered: 6  rate: 1.0`. That number is a property of a synthetic demonstration, and the documentation says so plainly, describing the run as a mechanism demonstration that makes no claim about real scientific discovery capability and ending with an explicit note that it is synthetic only. A reader should treat the recovery rate as a test of the machinery and nothing else, because the difficulty of the task is set by the generator that also created the answer.

## Grounding checks look for the failure modes a citation can hide

The verify stage performs span-level grounding checks with four named dimensions: citation existence, entity overlap, polarity consistency and numeric agreement. It produces graded flags for ungrounded or weakly grounded claims rather than a single boolean, which is what makes it usable in a loop where a proposal can be partially supported. The adversarial side is stated as a requirement, not a feature list. Fabricated citations, drifted numbers and negation flips are described as fixtures that must be caught, and a stage that quietly tolerates them would make every downstream number meaningless. Numeric agreement is the hardest of the four and the easiest to get wrong, because a number can be present in a span and still be wrong by a factor, while polarity consistency catches the case where a claim keeps its entities and its numbers but reverses its direction.

## Deduplication stacks four similarity signals and admits its error bars

The dedup stage combines normalized-text hashing, character and word n-gram Jaccard, TF-IDF cosine, and MinHash LSH. The documentation is unusually direct about this one: the false-positive and false-negative behavior of the method is documented, and planted paraphrase clusters are detected at measured recall and precision. The four signals do different jobs. Normalized hashing catches near-identical restatements cheaply, n-gram Jaccard is sensitive to word order and surface form, TF-IDF cosine weights rarer terms more heavily so that a shared common word does not create a false match, and MinHash LSH is what makes the comparison cheap enough to run over a large corpus rather than a sample. The MinHash LSH step is also where the failure modes live, since approximate matching trades exactness for speed, which is why the documentation publishes the rates rather than claiming correctness.

## The rank stage recovers a planted skill order with Bradley-Terry updates

The rank stage runs a round-robin pairwise tournament judged on rubric scores for novelty, testability, grounding and consistency, and applies Bradley-Terry and Elo updates with draw handling and K-factor decay. What makes this stage interesting is how it is validated. Property tests assert that the ratings recover a planted skill order within tolerance on seeded runs, meaning the corpus contains agents with a known ranking and the tournament machinery has to rediscover it from pairwise outcomes. A tournament that looks plausible but drifts from the planted order would still be a working Elo implementation, so the property test is the part that says something. The K-factor decay means the rating adapts as the tournament proceeds rather than freezing at the first few results. One of the three shipped examples is a full generate, debate and rank tournament showing Elo recovery of a planted skill order, with its own README and captured output.

## Byte-stable artifacts and a NumPy-only runtime

The artifact stability story is the quiet engineering achievement in this repository. All JSONL and JSON artifacts are written through a single canonical serializer with sorted keys, fixed separators, and floats quantized at their production points. The consequence is that run outputs, and the golden digests pinned over them, are byte-identical across platforms and interpreter versions. Checkpointed runs replay completed stages from the ledger and produce artifacts byte-identical to straight runs. That is a stronger reproducibility claim than most research tooling makes, and it is enforced by a test suite rather than a promise. The repository also pins a lock file, `requirements.lock.txt`, and ships a Makefile whose `test` target excludes the slow mark, so the fast suite is the one a contributor runs first. The cost of this strictness is that any change to a float at a production point or a key ordering becomes a visible diff in a pinned digest, which is exactly the point but slows careless edits. The dependency profile is unusually small for a project in this area. At runtime the package depends only on NumPy, and PyTorch is an optional extra used solely to demonstrate a small trainable ranker. Every default test and example runs without network access, without calling real models, and without downloading weights. The declared floor is Python 3.11, with classifiers for 3.11, 3.12 and 3.13, and the runtime dependency is pinned as:

```toml
dependencies = ["numpy>=1.26"]
```

The optional extra is separate and optional:

```toml
torch = ["torch>=2.2"]
```

That split has a practical effect on where this fits. A team without GPU access, without a model provider contract, or without a data-sharing agreement can still run the entire pipeline and its test suite, which removes the usual objection to evaluating research machinery. It also means the pluggable agent protocol is exercised through scripted, replay or loopback-mock agents, so the debate and evolve stages are testable without any model in the loop, though it also means the tool cannot tell you how the loop behaves against a real model unless you write that adapter.

## The limitation the project states first: it is alpha, and its evidence is synthetic

The honest limits are visible in the project description rather than buried. The package classifier marks the development status as 3 - Alpha, and the intended audience as Science/Research. The pipeline's correctness is established against synthetic corpora with planted answers, so the components can be wrong in ways that a synthetic check would not catch, since the generator and the checker share an author and an assumption set. The verification stage is the part most exposed to this, because grounding checks operate on spans from a corpus the same system wrote. The domain of use the project claims is a mechanism layer, not a research assistant, and the report stage is built to be honest about it, generating Markdown and self-contained HTML with a limitations section, hostile-text escaping and embedded run metadata. There is also a packaging oddity worth checking before you rely on links: the URLs in pyproject.toml point at a Netrixo/hypoarena repository, while this project sits under the OpSafari organisation.

## Conclusion

Hypoarena fits researchers and engineers who need the plumbing of a hypothesis pipeline tested without a model provider, a network connection or a GPU, since every default test and example runs offline and the runtime dependency is NumPy alone. It is the wrong tool for anyone hoping to discover something true about the world, because the corpus is synthetic and the ranking, grounding and dedup stages are validated against planted answers the same project planted. It is also the wrong shape for a team that needs a packaged multi-agent research assistant, since the repository classifies itself as a development status of 3 - Alpha. Before adopting it, read docs/cli.md for the subcommand options and exit codes, run `hypoarena demo` to see the recovered chain rate on your own machine, and check that the pinned golden digests in the test suite still match your Python version.

## FAQ

### How do I install hypoarena?

The README gives a virtual environment setup and a pip install of the package with the dev extra, which pulls hatchling, pytest, ruff and mypy. A second command installs the optional torch extra from the CPU-only wheel index when the optional trainable ranker is needed.

### Does hypoarena need network access or a GPU?

No. The package depends only on NumPy at runtime, PyTorch is an optional extra for a small trainable ranker, and every default test and example runs without network access, without calling real models and without downloading weights.

### How does hypoarena check that its hypothesis ranking works?

The rank stage runs a round-robin tournament with Bradley-Terry and Elo updates, and property tests assert that the ratings recover a planted skill order within tolerance on seeded runs. The demo command prints how many planted links were recovered and writes report.md and report.html.

## Sources

- [Issues](https://github.com/OpSafari/hypoarena/issues)
- [License: MIT](https://github.com/OpSafari/hypoarena/blob/main/LICENSE)
- [OpSafari/hypoarena on GitHub](https://github.com/OpSafari/hypoarena)
- [README](https://github.com/OpSafari/hypoarena/blob/main/README.md)
- [Releases](https://github.com/OpSafari/hypoarena/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/opsafari-hypoarena