# tokenjuice: compacting terminal output before it reaches an agent's context

> tokenjuice is a deterministic output compactor for agent workflows. It leaves the command you ran alone, observes what the command printed, and returns a smaller payload built from rule-driven reducers instead of dumping the whole terminal transcript back into the model's context. It ships as a native CLI and as thin hook wrappers for a long list of coding agents.

**vincentkoc/tokenjuice** — 🧃 Token weight loss. Lean output compaction for terminal-heavy agent workflows. Works as a native CLI tool or as an extension to popular coding and agent frameworks.

- Repository: https://github.com/vincentkoc/tokenjuice
- Stars: 527 · Forks: 51
- Language: TypeScript
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/vincentkoc-tokenjuice

## The problem and the shape of the fix

tokenjuice is a deterministic output compactor for terminal-heavy agent workflows, and the readme describes the situation it addresses in one dense paragraph.

Agents and harnesses run noisy commands like git status, pnpm test, docker build, rg, or pnpm --help. tokenjuice keeps the command semantics untouched, observes the output after execution, and returns a smaller payload built from rule-driven reducers instead of dumping the whole wall of terminal text back into context.

Three properties of that design are worth pulling out separately, because they are the difference between this and an ordinary summariser.

First, it observes rather than intercepts. The command runs exactly as it would have, with the same semantics and the same exit status. Nothing is wrapped or proxied, so there is no risk of a compaction layer changing what a build actually does.

Second, it is deterministic. The reducers are rules, not a model. That means the same output always compacts to the same result, which matters for a tool sitting in the path of a build loop, and it means the behaviour can be read and reviewed.

Third, the rules stay inspectable JSON instead of model behaviour, which points at a real property. If you want to know why a particular output was reduced a particular way, you read the rule file.

The claimed payoff is in the same paragraph: less transcript waste, fewer useless reruns, and cleaner handoff between tools without making the shell magical.

The escape hatches are equally explicit. Raw output stays available only when you explicitly ask for it through --raw or --full, or through opt-in artifact storage. So the default is compaction and the opt-in is the wall of text, which is the right way round for a tool whose whole job is to save context.

## The integration surface, and why it is so wide

The integration table is the most striking thing about the repository, and it says a lot about how agent tooling has fragmented.

The supported integrations are Claude Code, CodeBuddy, Codex CLI, Cursor, Droid from Factory, GitHub Copilot CLI, OpenClaw, OpenCode, pi, and VS Code Copilot Chat. Each has an install command and a hook file, and the hook file locations show how differently these tools are configured.

The majority use a JSON settings file in the user's home directory. Claude Code uses ~/.claude/settings.json, CodeBuddy uses ~/.codebuddy/settings.json, Codex CLI uses ~/.codex/hooks.json, Cursor uses ~/.cursor/hooks.json, Droid uses ~/.factory/settings.json and Copilot CLI uses ~/.copilot/hooks/tokenjuice-cli.json.

Then three of them are not JSON hooks at all. OpenClaw is configured by setting a plugin entry in ~/.openclaw/openclaw.json, OpenCode takes a JavaScript plugin file at ~/.config/opencode/plugins/tokenjuice.js, and pi takes a JavaScript extension at ~/.pi/agent/extensions/tokenjuice.js.

The readme states the principle behind the whole approach: host integrations stay thin wrappers around the same core reducer instead of becoming one-off adapter logic. So the compaction logic is one implementation and each integration is a small shim that knows where the host's hook file lives and what shape it expects.

That is why the beta integration list can be so long. It runs to more than twenty entries covering AdaL CLI, Aether, aictl, AI Memory Protocol, Aider, Agent Layer, AgentInit, Agentlink, Agentloom, agents-cli, the AGENTS.md convention, agents.ge, AgentsMesh, Amazon Q, Amp, Google Antigravity, anywhere-agents, Augment and Avante.nvim, with the list truncated in the readme.

And these do not all use a hook file. Many take a markdown instructions file instead: Aider uses CONVENTIONS.tokenjuice.md, Agent Layer uses .agent-layer/instructions/tokenjuice.md, agents-cli uses ~/.agents/memory/AGENTS.md, Amazon Q uses .amazonq/rules/tokenjuice.md, and AGENTS.md itself uses AGENTS.md. Several of them also need a sync step after install or uninstall, because those hosts keep their own copy of the instruction file.

So the tool has two installation mechanisms. A hook file for hosts with a real interception point, and an instruction file for hosts that only accept guidance to the model, which is a meaningfully weaker integration.

## What is in the package, and how it is built

The package manifest describes a well-organised TypeScript project rather than a single script.

The package is tokenjuice at version 0.8.5, ESM only, with the type field set to module. It requires Node 20 or newer, uses pnpm 10.11.0 as its package manager, and is MIT licensed with GitHub Sponsors funding configured.

The entry points tell you how the pieces are consumed. The main export is ./dist/index.js with types at ./dist/index.d.ts, and there is a single binary, tokenjuice, pointing at dist/cli/main.js. Then there is a second export path, ./openclaw, which resolves to a separate build under dist/hosts/openclaw/, with its own types. That is the one host that gets a bespoke export rather than a generic hook, and the presence of a dedicated hosts/ directory in the build output shows the structure is designed around per-host packaging.

The published files list is deliberately narrow: dist, README.md, LICENSE and SECURITY.md. The SECURITY.md being shipped is a small detail worth noting for a tool that sits between an agent and a shell.

The build script is a chain and it shows what the project has to do beyond compiling. It cleans dist, generates the builtin rules with a dedicated script, runs tsc, then builds a pi runtime and an OpenCode runtime with separate scripts, and finally copies the rules into place. So the rule set is generated rather than hand-written, and two of the hosts need a runtime bundle of their own built on top of the core.

The scripts section also shows the testing approach. There are e2e:local, which builds and then runs a local host end-to-end script, a bench:fixtures target and a bench:verify target for the benchmark, an analyze:codex-logs script for analysing real Codex logs, and a contracts target that runs publint in strict mode against the packed tarball. The contracts check is the one that catches packaging mistakes, and running it against the packed artefact rather than the source tree means it verifies what users actually install.

The repository layout follows the same logic. There is src/ for the source, test/ with a vitest.config.ts, docs/ for the documentation, packaging/ for the distribution artefacts, scripts/ for the build tooling, and an .agents/ directory at the root alongside AGENTS.md and a .oxlintrc.json for the linter.

## The rules, and the bet the project is making

The central bet of tokenjuice is stated in the readme and it is worth examining on its own terms.

The bet is that most of the output an agent sees from a shell command is noise, and that the noise can be removed by rules rather than by understanding. A test run that prints forty passing suites and three warnings can be reduced to the count and the failures, because the passing lines are implied. A git status showing thirty modified files can be reduced to the summary line plus the file names, because the per-file diff of unstaged and staged is redundant. A help text is mostly a list of flags the agent did not use.

That kind of reduction is not summarisation in the language-model sense. It is pattern matching against known output shapes, which is why it can be deterministic, why the rules can be inspectable JSON, and why the same input always produces the same output.

The strength of the approach is that it does not require understanding. The weakness is that it only works where the rules cover the output. A command whose output shape tokenjuice has not seen comes back either untouched or badly reduced, and the second case is the dangerous one, because a badly reduced output can hide the line the agent needed.

That risk is what the escape hatches are for. --raw and --full give you the original output, and opt-in artifact storage keeps the full output on disk so an agent can go and look at it rather than having it dropped entirely. A compaction tool that silently discards the thing you later needed is worse than no compaction tool, and the design here is clearly built with that in mind.

The generated builtin rules are the other half. Since the rules are generated by a script rather than hand-maintained, they can be produced from known command output formats at build time and regenerated as those formats change, which is how the tool keeps pace with the commands it targets.

## The release rhythm and the shape of the project

A few details about how the project is put together say more about its intentions than the feature list does.

The version numbers tell the story. The recent releases are v0.8.5 from 2026-09-15, v0.8.4 from 2026-09-13 and v0.8.3 from 2026-09-04, and the last push was on 2026-09-15. That is three releases in a fortnight at the 0.8 line, which is a project being changed fast and often, and the pre-1.0 version number is the honest signal that the API is still moving.

The repository topics are openclaw-plugin, optimization, token and tui-app. The first tells you that the OpenClaw plugin path is the one with a dedicated export in the package manifest, and the last tells you that the tool is aimed at terminal user interfaces, which is consistent with a tool whose whole purpose is to deal with terminal output.

The .agents/ directory at the repository root and the AGENTS.md file are worth noting on their own. A project whose job is to write instructions into other agents configuration files has a particular interest in the conventions those files follow, and having its own AGENTS.md means it consumes the same kind of file it writes.

There is a CHANGELOG.md and a SECURITY.md, and the latter is shipped in the published package alongside the code, the readme and the licence. For a tool that sits between an agent and the commands it runs, having a security policy in the published artefact rather than only on the repository page is a reasonable choice.

One more thing the readme says is worth repeating because it sets expectations about what the tool is not. It is not making the shell magical. The command still runs, the exit code still comes back the same way, and the compaction happens afterwards as a separate step. That is a deliberately unspectacular design, and for something that runs on every shell command in an agent loop, unspectacular is the right property.

## Conclusion

Adopt tokenjuice if you run a coding agent that constantly shells out, because the transcript waste from commands like git status or a test run is one of the largest predictable sources of context bloat. Do not adopt it if you need a summariser that understands what the output means, because the reducers are rule-driven and deterministic rather than model-driven. Before you commit, run it once with --raw so you can see exactly what it is dropping, since the value depends entirely on the rules matching the commands you actually run.

## FAQ

### What does tokenjuice do?

It is a deterministic output compactor for terminal-heavy agent workflows. The command runs with its semantics untouched, tokenjuice observes the output afterwards, and returns a smaller payload built from rule-driven reducers instead of dumping the whole terminal transcript back into the agent's context.

### How does tokenjuice avoid hiding output the agent needs?

Raw output stays available when you explicitly ask for it through the --raw or --full flags, or through opt-in artifact storage. The rules are also inspectable JSON rather than model behaviour, so you can read exactly what a given reducer keeps and discards.

### Which coding agents does tokenjuice integrate with?

Supported integrations include Claude Code, CodeBuddy, Codex CLI, Cursor, Droid, GitHub Copilot CLI, OpenClaw, OpenCode, pi and VS Code Copilot Chat, each installed with a tokenjuice install command that writes a hook file. A longer list of beta integrations, including Aider, Amazon Q, Amp, AGENTS.md-based hosts and others, relies on markdown instruction files rather than hooks.

### How is tokenjuice built and packaged?

It is an ESM TypeScript package built with pnpm and tsc, requiring Node 20 or newer. The build cleans dist, generates the builtin rules, compiles, then builds separate runtime bundles for pi and OpenCode and copies the rules into place. A contracts target runs publint in strict mode against the packed tarball to verify the published package.

### Does tokenjuice use a language model to summarise command output?

No. The compaction is done by rule-driven reducers, which is what makes it deterministic and inspectable. The rules are generated from a script rather than hand-written, with the builtin rules regenerated by the generate:builtin-rules build step.

## Sources

- [Issues](https://github.com/vincentkoc/tokenjuice/issues)
- [License: MIT](https://github.com/vincentkoc/tokenjuice/blob/main/LICENSE)
- [README](https://github.com/vincentkoc/tokenjuice/blob/main/README.md)
- [Releases](https://github.com/vincentkoc/tokenjuice/releases)
- [vincentkoc/tokenjuice on GitHub](https://github.com/vincentkoc/tokenjuice)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/vincentkoc-tokenjuice