benjamin-plus: a token-efficiency skill you inject, not install

The problem: an agent pays twice for every clumsy lookup

Every lookup an agent performs is billed twice. The step itself costs tokens, and then the growing conversation containing that step gets re-read on every subsequent turn. benjamin-plus targets that second, compounding charge. It does not touch what the agent builds; the README states plainly that it changes how the agent looks things up and waits.

The intended user is anyone running a coding agent on tasks long enough for context to accumulate: Claude Code sessions, Codex CLI sessions, or any harness where a system prompt can be extended. On short, single-turn tasks there is nothing to compound, so the skill has little to act on. The README also notes that on some agent platforms, polling alone turned out to be nearly half of all steps, which is the kind of waste a single rule can remove.

Five habits, and the reasoning behind each

The ruleset is short by design: RULESET.md is roughly 745 tokens injected, and the integration file is about 3 KB. Five rules carry the weight.

Recon in one pass replaces five separate pokes at a repository with one combined step, and requires two real examples before copying a format or convention. Keyhole reads cap inspection at 50 lines when the agent only needs to see something, with an explicit carve-out: data the agent will actually transform is never truncated. Environment probing checks every dependency in one command and installs whatever is missing in one go, rather than discovering gaps one crash at a time. The definition of done is the task's own check, and a check that fails twice means the approach is wrong, not the symptom; when it passes, the agent stops. Polling is treated as a step, so a build is checked every 30 seconds rather than every second.

That last rule has an interesting provenance. The README says results from other benchmarks and harnesses are how v6 got its polling rule, which suggests the ruleset is meant to keep absorbing outside evidence rather than stay fixed.

Injecting benjamin-plus into Claude Code and Codex CLI

The project's own framing is the install instruction: inject it, don't install it. Start by cloning the repository to a fixed path.

git clone https://github.com/JetBrains/benjamin-plus-skill ~/.benjamin-plus

For Claude Code, add a SessionStart hook to ~/.claude/settings.json. The README gives this exact block.

{ "hooks": { "SessionStart": [ { "matcher": "startup|resume|clear|compact",
  "hooks": [ { "type": "command", "command": "cat ~/.benjamin-plus/injected-instruction.md" } ] } ] } }

The README says to verify the hook with /hooks, or to simply ask Claude Code to add it. A per-project alternative needs no configuration at all: appending the file to CLAUDE.md.

cat ~/.benjamin-plus/injected-instruction.md >> CLAUDE.md

Codex CLI needs no hook because AGENTS.md is loaded into every session. Appending there is the whole setup.

cat ~/.benjamin-plus/injected-instruction.md >> ~/.codex/AGENTS.md

For any other agent, the README's instruction is to append injected-instruction.md to the system prompt. What you should see is the ruleset text present at session start, not a skill the agent has to discover.

Why the skill folder delivery saves nothing

This is the most useful negative result in the README. The same skill was tested two ways: injected into the prompt, and placed as a discoverable skill folder. Injected, it saved. As a folder, it saved nothing (−0.5%, not significant), because agents burned steps just finding SKILL.md. That cost is exactly the kind of step the skill exists to eliminate, so the delivery mechanism was paying for itself twice over.

If you are browsing an awesome-claude-code style list of skills and planning to drop this one into a skills directory, the README's measurement says that will not reproduce the headline number. The delivery method is not an implementation detail here; it is the difference between a measurable saving and none.

What the paired A/B actually measured, and what it did not

The measurement is a paired A/B: same agent, same model, same tasks, same container images, with the injected skill text as the only difference between arms. The README describes 80 paired SkillsBench tasks run in Claude Code 2.1.201 inside Docker sandboxes with Sonnet 5 at low effort, Wilcoxon on paired deltas, a sign test on rewards, and a per-trial adoption check that found the payload reached the model in 80 of 80 treated runs and 0 of 80 controls. Trials that failed on one side only were retried before anything was counted.

Quality came back unchanged: 7 better, 5 worse, 68 ties (sign p = 0.77), mean verifier reward 0.362 to 0.392. The README is careful to note this is not powered as an equivalence test, so large effects are ruled out but small ones are not. Savings scale with baseline bloat. An identical run a day earlier measured −10.0% median cost against a leaner baseline, and the treated arm stayed flat across both days while the control drifted +10.5%. The README's guidance is to expect roughly −10% to −18% cost depending on how bloated your sessions run.

On harder work the effect shrinks. On Java SWE-bench with Codex CLI and gpt-5.6-luna across 675 paired replicas, the hook-injected skill measured −4.4% cost [−7.5, −1.5], p = 0.003, with solve rate unchanged (p = 0.22) and tool calls down 20%. The README also warns that medians are the honest unit, because a few hard-task tails can give part of the aggregate back. If your workload is dominated by hard tasks, the −4.4% figure is the more relevant one.

Where benjamin-plus is the wrong tool

The skill is a behavioural prompt, not an enforcement layer. Nothing stops an agent from ignoring a rule, and the README does not describe a mechanism that verifies compliance beyond the adoption check that the payload reached the model. If you need hard guarantees about tool-call counts, this is not that.

The rules also assume a workload with repetition to remove. Recon in one pass only helps if the agent would otherwise poke at the repository repeatedly; keyhole reads only help when files are large relative to what the agent needs. On tasks where the check itself is ambiguous, the rule that a failing check twice means the approach is wrong could push an agent to abandon a correct approach because the verification command was misread. The README does not document a rollback path or a per-rule opt-out, so tuning means editing the ruleset text yourself.

Finally, the numbers are medians from specific harnesses and models. The README explicitly invites reports of workloads where it loses money, which is an admission that such workloads exist and have not all been catalogued.

Alternatives and how they differ

The closest alternative is writing your own project-level instruction file, such as a CLAUDE.md or AGENTS.md populated with your team's conventions. The difference in approach is evidence: benjamin-plus went through what the README calls auto research, mining roughly 1,200 old agent traces, drafting rules, running paired A/Bs, reading failing trajectories, and revising across six versions. Rules that traded quality for savings were deleted. A hand-written instruction file encodes what your team believes; this one encodes what survived a paired measurement.

The trade-off is specificity. A hand-written file can encode domain rules this skill knows nothing about, and it will not spend 745 tokens of your context budget on generic lookup discipline. If your agent already behaves efficiently, the ruleset is overhead. If it does not, the README's claim is that these five habits are the ones that paid for themselves.

Maintenance, licence and what a fork costs you

The repository is MIT licensed and is not archived. Its last push was on 2026-08-27, and there are no releases retrieved, so distribution is via git clone or by copying the files. That matters for upgrades: there is no package version to pin, so the practical unit is a commit. The README does not document rollback, so keeping your own copy of injected-instruction.md before overwriting it is the only version control you get.

Under MIT you may copy, modify and redistribute the ruleset, including inside a commercial product, provided the licence and copyright notice are preserved. That is a general description of the licence, not legal advice; check the LICENSE file and your own counsel for anything that matters.

Upgrade cost is low but not zero. The integration is a file append or a hook command, so updating means re-cloning or re-copying and re-appending. If you edited the ruleset for your stack, those edits need reapplying, and the README offers no merge path. The maintenance surface is the ruleset text itself, which is short enough to review by hand before you adopt a new revision.

Editorial conclusion

Adopt benjamin-plus if your coding-agent sessions run long and you are willing to append a 3 KB file to the system prompt or wire a SessionStart hook. Do not adopt it if you expect a drop-in skill folder to work: the README reports that delivery method saved nothing (−0.5%, n.s.) because agents burned steps finding SKILL.md. Verify three things first: that the payload actually reaches the model in your harness, that your workload resembles the 80 paired SkillsBench tasks rather than the harder Java/Codex setup where the saving was −4.4%, and that a failing check twice makes your agent change approach rather than retry the symptom.

Frequently asked questions

Official sources

1. Issues
2. JetBrains/benjamin-plus-skill on GitHub
3. License: MIT
4. README