billion-context: Compression for Long-Lived Coding Agents

Why billion-context Targets Long Coding Sessions

billion-context addresses a specific limitation in long-running coding-agent sessions: every exchange adds conversation material that must be sent to the model and counted against a finite context window. The README describes the problem as growing context, token charges, degraded sessions, and requests that exceed the model window. Its intended users are developers using coding agents such as Claude Code, Codex, OpenCode, Gemini CLI, Qwen Code, GitHub Copilot CLI, pi, and other listed clients.

The project presents two related goals. It says that a 100K context window can be sufficient after compression, and it targets sessions lasting for months with billions of cumulative tokens. Those are product claims in the README, not measurements produced in this review. The clearest documented deployment model is to place billion-context between an agent and its model API. That makes it relevant when an agent already supports a proxy or compatible integration point, rather than only when a user can change the agent itself.

The design also differs from simply shortening a request by deleting old messages. The README says that the model decides when to compress and what to compress into summaries. This preserves the intent of letting the agent decide which historical material can be represented in a compact form, while the plugin handles the stream and context-management layer.

The Model-Driven Compression Path

The documented architecture has three conceptual layers. First is the coding agent, which produces conversation messages and expects model responses. Second is billion-context, described as sitting between that agent and the model API and rewriting Anthropic and OpenAI streams. Third is `acp-kernel`, which provides the compression mechanism.

The compression decision is model-driven. According to the README, the model selects what has already been consumed and summarizes it into high-fidelity representations. This is different from applying a fixed token ceiling. A fixed ceiling can stop sending older material without deciding whether it contains a failed command, a modified file path, a user requirement, or an unresolved error. The project instead describes compression as a content-management operation performed during an ongoing exchange.

The plugin is intended to rewrite both Anthropic and OpenAI streams, while the package exposes several integration entry points. The root export and `dsh` export point to `dist/agent/dsh-native.js`, the `server` export points to `dist/agent/opencode-native.js`, and the `client` export points to `dist/agent/dsh-native-client.js`. The package also defines `bili` and `bili-proxy` as command-line binaries. These entries indicate that the project supports more than one client path, but the README does not explain how a user selects among them.

The README also gives cache-health guidance. It states that a healthy session has a 95 to 97 percent prefix-cache hit rate and that compression itself costs no more than 2 percent. It attributes sustained lower rates to cache expiry, a model switch, a suspected bug, or another unknown cause. Those figures are presented as project documentation, and the README points to `/acp` and `/acp-cache` for attribution. The repository also contains technical notes, tests, release notes, and a paper, but the README does not reproduce their full operational details.

Install the Package and Check Its Entry Points

The README gives one installation command. It installs the package globally under the user's local npm prefix, which is useful for keeping the command outside a system-level npm directory.

npm install -g billion-context --prefix=~/.local

After installation, the package manifest declares `bili` and `bili-proxy` as binaries. A reader can check whether npm made those commands available, but the README does not document a first invocation, required environment variables, proxy URL, model configuration, or client-specific setup command. The repository includes `CONFIGURATION.md`, `CLIENTS.md`, and `PLUGIN.md`, so those files are the next project-specific places to inspect before connecting it to an agent.

This missing first-use sequence matters. Installing the package is straightforward, but installing it is not the same as activating it. The package manifest confirms that the project can load native DSH and OpenCode-related modules, yet it does not explain whether a user must launch a proxy, edit an agent configuration, or select an export path. The absence of invented setup flags is deliberate: the README does not provide enough information to state a safe first-run command.

Where the Compression Trade-Off Appears

The central limitation is that compression changes representation. The README claims high compression quality and says the model creates high-fidelity summaries, but it does not define how fidelity is measured, which message classes are preserved, or how conflicts between old and new requirements are resolved. A long-lived coding agent can therefore accumulate a compact history whose token count is acceptable while omitting a detail that a later edit depends on.

The model's role creates another dependency. Because the model decides when and what to compress, the result can vary with model behaviour, prompt content, and the client's message format. The README identifies model switches as a cause of lower cache health, which confirms that model changes affect the system, although it does not explain whether switching models also changes summary quality.

The cache guidance is useful but not a full reliability guarantee. A 95 to 97 percent prefix-cache hit rate concerns request caching, not necessarily semantic accuracy. Compression may reduce tokens and preserve cache reuse while still producing a summary that omits an important instruction. The repository's tests and proof directory may provide implementation-specific evidence, but their contents are not included in the README, so no conclusion about measured accuracy can be drawn from their names.

The project is also not the right tool for a short, self-contained coding request. Adding a compression layer introduces another component, another stream-rewriting path, and another configuration dependency. If a session already fits comfortably inside the model's window, the stated need for month-long history and token reduction is limited.

billion-context Compared with Manual or Native Context Management

The practical alternative is manual session summarization, where the developer periodically asks an agent to summarize the conversation and then starts or continues with that summary. The approach differs because the human controls when the checkpoint occurs and can review the retained narrative before the next request. Its weakness is operational effort: every checkpoint needs to be scheduled, and the summary still has to fit the model's window.

Another alternative is relying on the agent or model provider's built-in context handling. That approach avoids adding `billion-context` as a separate stream-rewriting component, but the README does not describe the compression behaviour, cache handling, or long-session controls of those other systems. The comparison therefore cannot claim that one preserves more detail. It can only state that billion-context supplies an explicit model-driven layer across Anthropic and OpenAI streams and exposes package entry points for supported clients.

For teams that need an audit trail, manual summarization has an obvious advantage because the developer sees the checkpoint. For teams running an unattended agent for weeks, the plugin's automated decision process removes the need to remember every checkpoint. The trade-off is trust in the compression layer and its client integration.

Licensing and Maintenance Signals for billion-context

The repository is classified as using a custom licence that GitHub cannot classify, and the repository contains a `LICENSE` file. That makes licence review a prerequisite for commercial or redistributed use. The README separately says that the paper is open-sourced under the MIT License as part of the codebase. These are different assets with different terms, so the paper's licence should not be treated as the licence for the npm package.

The repository is not archived. The last push was on 2026-10-15, and the listed recent releases are `v0.1.195` on 2026-10-12, `v0.1.196` on 2026-10-13, and `v0.1.197` on 2026-10-14. That release cadence indicates recent project activity, but it does not by itself establish compatibility with every listed client or explain the upgrade procedure. The repository includes `CHANGELOG.md` and `release-notes/`, which are the relevant files for checking what changed between versions.

Because the package version is `0.1.197`, adopting it as infrastructure also means accepting a pre-1.0 interface boundary unless the project's own documentation says otherwise. The package manifest includes build, typecheck, test, and end-to-end scripts, but the README does not state which exact verification command should be run after an upgrade.

Editorial conclusion

Adopt billion-context when a coding agent needs to continue across a long session and the surrounding agent and model API can be configured to place the plugin in the request path. Do not adopt it without reviewing the custom LICENSE, confirming the documented client setup, and validating that its model-driven summaries preserve the task details that matter. The first concrete check is the client configuration and the exact behaviour of the installed `bili` and `bili-proxy` entry points.

Frequently asked questions

Official sources

1. Issues
2. ranxianglei/billion-context on GitHub
3. README
4. Releases