ctx: a repository-aware fit test for AI coding configurations

The cost question ctx tries to answer

Most teams pick an AI coding setup by trying one, feeling that it works, and stopping. ctx starts from the opposite end: it asks which configuration is the cheapest one that still passes the repository's own tests. The README frames the goal as finding the cheapest AI coding setup that actually works on your repo, and it treats reliability as a requirement rather than a tie-break. If nothing beats the setup you already have, the tool says so.

The intended user is a developer or platform engineer who already has a coding agent running in a repository and wants to justify the spend. That person needs a comparison that survives review, not a leaderboard. The design reflects that: an LLM may explain a result, but per the README it never decides one. The decision is mechanical, which is the main reason to consider ctx over a vendor benchmark.

The scope is narrower than the name suggests. Release 1.0.21 compares capability configurations within one coding-agent harness. It does not compare Codex, Claude Code, or other harnesses against each other. If your actual question is which harness to standardize on, this tool does not answer it.

What the free profile actually inspects

Bare ctx fit is free, local, and read-only. According to the README it runs no model, spends nothing, and issues no git commands at all. It detects languages, lists the current AI coding setup (instruction files such as AGENTS.md and CLAUDE.md, tool config, installed skills), and derives the commands the repository uses to verify itself.

The verification detection is the interesting part. The example output in the README shows test, typecheck, lint, and build commands traced back to pyproject.toml sections, each with a confidence label: high confidence for pytest, mypy, and ruff, medium confidence for the build backend. That provenance matters because the selected test command becomes the verification authority for any later evaluation.

Alongside that, ctx scores agent readiness out of 100 across six categories: verification, instructions, environment, CI enforcement, tool safety, and context tractability. The README's own example scores 91/100 and flags a missing committed dependency lockfile as the highest-impact improvement at +9 points. Read that score as a checklist of static evidence, not as a prediction. The README says it plainly: the score shows the static evidence needed to plan an evaluation, and whether the tests can execute is checked only inside the campaign.

Install, doctor, dry run, then spend

The base install is one command:

pip install --upgrade claude-ctx

Then cd into the repository and run ctx fit. Requires Python 3.11 or newer. Add --json for machine-readable output, or --dry-run to see what a full evaluation would involve. Note that --dry-run does read history: it runs read-only git queries (log, show --name-only, ls-tree, rev-parse) to derive representative tasks, and writes nothing to the repository, the index, or any ref.

Spending requires both --test and --budget N. The README is explicit that --test without --budget only plans. Before that, run ctx doctor to see whether a real evaluation can run here. A real evaluation needs pip install "claude-ctx[harness]", Node.js with npx for the workspace-filesystem MCP, a matching provider credential, and Bubblewrap on Linux.

The failure modes around credentials are worth reading twice. Without a matching provider credential, --test runs in simulation, which proves the pipeline but not your repository. With a credential but a missing live prerequisite, ctx refuses the run before trial setup. A simulated result is refused as evidence for --apply and --pr, so you cannot quietly promote a simulation into a committed change.

Ubuntu 24.04 and the Bubblewrap profile

On Ubuntu 24.04 the sandbox needs host configuration. The distribution restricts unprivileged user namespaces, and installing bwrap does not prove it can start the network-disabled namespace ctx uses for repository commands. The README's fix is to install and load Ubuntu's packaged, scoped bwrap-userns-restrict profile for /usr/bin/bwrap:

sudo apt update sudo apt install bubblewrap apparmor-profiles apparmor-utils if [ ! -e /etc/apparmor.d/bwrap-userns-restrict ]; then sudo install -m 0644 \ /usr/share/apparmor/extra-profiles/bwrap-userns-restrict \ /etc/apparmor.d/bwrap-userns-restrict fi sudo apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict ctx doctor

The README advises keeping Ubuntu's global unprivileged-user-namespace restriction enabled and using the targeted profile instead of weakening the system-wide boundary. That is a defensible choice, but it is administrator-visible host policy affecting every /usr/bin/bwrap caller, not a private ctx setting. The conditional block preserves an existing local profile rather than overwriting it. ctx doctor proves the path with a bounded /bin/true probe in the same no-network namespace, and per the README it executes no repository code and calls no model. If you run on a managed fleet, that AppArmor change is the part your platform team will want to review first.

The selection rule and what the campaign can verify

The winner is chosen by a fixed rule, not a score: discard every candidate below the reliability floor, then minimize attributable cost, then break ties toward the simpler configuration. That ordering is the whole design argument. A cheaper configuration that fails the repository's test command is discarded before cost is considered at all.

The campaign environment constrains what that verification means. For an installable Python project, ctx builds a campaign environment and installs it without network access, so the build backend and dependencies must already be available without downloading them. In JavaScript/TypeScript, Go, Rust, and Make repositories, verification is supported only when the runtime is usable from the host PATH under an isolated home and the verification dependencies are already present in the repository. Final verification runs without network access, and the README states that a user's package caches are not a supported dependency source.

That is a real constraint, not a footnote. A project whose install step reaches the network, or whose test suite depends on a warm cache, will not verify under this model. The README also draws its own boundary: this is evidence for normal development, and it does not prove that deliberately hostile code cannot deceive its own test runner. If your threat model includes adversarial code in the repository, the test command is not a trustworthy authority.

apply writes to the tree, pr writes a pull request

The two output paths differ in what they touch. ctx fit --apply writes the winning configuration into your working tree, on whatever branch you are standing on. It prints every proposed change first and stops there unless you pass --yes. The write itself runs no git command: nothing is staged, committed, or pushed. Getting to that point does run git, because --apply is refused without evidence from ctx fit --test --budget N, and deriving the tasks for that evaluation uses the same read-only queries as --dry-run.

ctx fit --pr produces the change as a pull request instead. Each proposed change names the file and whether ctx is creating or modifying it. The README states that today every plan contains exactly one CTX-owned file, though the supplied text is truncated mid-sentence at that point, so treat the full shape of a plan as something to confirm against the current release rather than something documented here.

The practical difference is review surface. --apply leaves a diff in a tree you already control, which suits a single developer testing a change locally. --pr routes the same change through whatever review the repository already enforces, which suits a team that wants the configuration change to carry the same approval trail as code.

Where ctx is the wrong tool, and what to use instead

Three cases where ctx does not fit. First, harness comparison: if you are deciding between Codex and Claude Code, 1.0.21 explicitly does not do that. Second, repositories without a single reliable test command: the selected test command is the verification authority, so a project with flaky or absent tests has no floor to clear, and the reliability filter has nothing to work with. Third, air-gapped or cache-dependent builds, per the no-network install constraint described above.

For the harness question, the alternative is your own harness-level evaluation: install both, run the same repository tasks under each, and compare cost and pass rates yourself. That is more work than ctx fit, and it is the only way to get an answer ctx does not claim to provide. The difference in approach is structural. ctx holds the harness constant and varies the capability configuration inside it, which is what makes its fixed selection rule meaningful. A cross-harness comparison varies the harness, so the reliability floor and the cost attribution would need to be defined differently for each one.

There is a second alternative worth naming: doing nothing and keeping the current setup. ctx is built to return that answer when no candidate clears the floor, and the README says the tool will say so. If your goal is a written justification for the setup you already have, the free profile plus --dry-run may be enough, and you never need a provider credential.

Maintenance cost and licence

ctx is MIT licensed and published on PyPI as claude-ctx. MIT is permissive: it allows commercial use and modification, and it comes with no warranty. That is not legal advice; check the LICENSE file in the repository and your own organization's policy before redistributing or vendoring it.

The repository ships a large graph artifact, described as a 79,958-node graph with 68,494 skills, 467 agents, 10,790 MCPs, and 207 harnesses, alongside a separate graph-artifacts release tagged for v1.0.21. Those numbers describe the shipped inventory, not adoption or quality. If you use the shipped graph rather than your own inventory, you are depending on an artifact that is versioned separately from the code, so pin both.

Upgrade cadence is visible in the release list: v1.0.20 in June 2026, v1.0.21 in August 2026, with graph artifacts published the same day as the code release. The README also carries a CI badge reporting 8,803 inventory tests. Treat that as a signal that the inventory is covered by tests, not as evidence that your repository will verify cleanly. The operational cost that actually matters is environmental: Python 3.11 or newer, Node.js with npx for the workspace-filesystem MCP, a provider credential, and Bubblewrap with the AppArmor profile on Ubuntu 24.04. Budget for the host setup once, and re-run ctx doctor after any system upgrade that touches AppArmor or user namespaces.

Editorial conclusion

Adopt ctx if you already run a coding agent in a repository with deterministic, declared tests and you want a cost decision made by a fixed rule instead of by a model. Do not adopt it to compare Codex against Claude Code: release 1.0.21 compares capability configurations inside one harness only. Before authorizing spend, run ctx doctor on the target machine, then ctx fit --dry-run, and read the campaign scope it prints. If you are on Ubuntu 24.04, confirm the bwrap-userns-restrict profile is loaded so the probe can start the network-disabled namespace.

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. stevesolun/ctx on GitHub