What is Context window?

How the context window works

A transformer model does not read text the way a person does. Text is split into tokens, each token is mapped to a vector, and the model computes attention over the whole sequence at once. The context window is the maximum sequence length the attention layers were built and trained to handle. If the token count of the prompt plus the expected reply exceeds that number, the request either errors out or the oldest content is dropped, depending on the API.

Two mechanics matter in practice. First, attention cost grows with sequence length, so a longer window is slower and more expensive per call even when the provider does not bill a separate fee for it. Second, the window is shared. A system prompt, a tool schema, retrieved documents, earlier turns and the model's own output all compete for the same budget. A 200k-token window is not 200k tokens of user conversation; it is 200k tokens of everything.

Providers differ in how they count and enforce the limit. Some count only input tokens, some count input plus a reserved output budget, and some apply a smaller effective limit when tools are enabled. The only reliable way to know is to read the provider's documentation for the model in use, or to measure it, which is what the tracing tools below do.

When you need a large context window and when you do not

A large window helps when the task genuinely requires many tokens at once: reviewing a long file, comparing several documents, or keeping a long agent session coherent without re-summarising. Retrieval-augmented generation can reduce the need, because only the relevant slice is inserted instead of the whole corpus. ref-tools/ref-tools-mcp follows that pattern: according to its description, it searches public and private documentation and returns only the relevant slice of a page, which is a way to spend fewer tokens rather than a bigger window.

A large window is the wrong tool when the problem is routing or repetition. If an agent re-reads the same tool output every turn, a bigger window just delays the overflow. mksglu/context-mode attacks that directly: its description says it sandboxes tool output and persists session memory, and the README claims a 98% reduction in tool output, a project claim rather than an independent measurement. Paritok-official/paritok-4b-v1 takes a different route, sitting between the agent and the API, filtering tool schemas, compressing tool results and summarising stale history; the README claims 25% savings on turn one rising past 85% in long sessions. Both are compression strategies, not window enlargements.

There is also a cost case for staying small. A short, well-structured prompt is cheaper and often more accurate than a long one, because irrelevant tokens can distract the model. The decision is not "bigger is better" but "what must be visible at once for this task".

Common pitfalls and limits

The most common mistake is treating the window as free memory. It is not. Every token in the window is processed on every turn, so a session that starts with a large system prompt pays for it repeatedly. Compaction, the act of summarising old turns to free space, is lossy: detail that seemed unimportant at turn ten may be needed at turn fifty, and once summarised it is gone.

A second pitfall is silent truncation. Some APIs drop the oldest messages without an error, so an agent can lose its instructions mid-session and behave strangely with no visible cause. A third is tool schema bloat. Each tool definition consumes tokens before any work happens, and agents with many tools can spend a noticeable fraction of the window on definitions alone. Paritok's filtering of tool schemas is aimed at exactly this.

Measurement is harder than it looks. Token counts from a tokenizer are an approximation of what the provider bills, and different providers count differently. gkamradt/needle-in-a-haystack is a measurement harness rather than a production library: its v2 rewrite uses YAML run files, a JSONL result store and a reconstruct command that rebuilds the exact prompt a model saw, which is useful for checking whether a model actually uses the middle of a long window. It does not tell you what your agent is doing in production; for that you need tracing.

How the context window shows up in open-source projects

Several projects exist mainly to make the window visible. matt1398/claude-devtools is an MIT-licensed Electron app that, per its analysis, parses the transcripts Claude Code already writes to ~/.claude and rebuilds tool calls, thinking, subagent trees and per-turn token attribution. It is a viewer for logs you already have, not a wrapper around the CLI. graykode/abtop is read-only and needs no API keys; its analysis says it reads local process and file state to show token usage, context window percentages, rate limits and orphan ports, installing from a release script or cargo. jmuncor/tokentap sits between an LLM CLI tool and the provider API, forwarding requests while a Rich dashboard shows cumulative token use against a configurable limit; it works today for Claude Code, Codex and MiniMax, while Gemini CLI support is blocked by an upstream bug.

Others try to change how the window is used. mksglu/context-mode is an MCP server that sandboxes tool output, indexes session events into SQLite FTS5 and routes tool calls across 17 client platforms; it is Elastic-2.0 licensed. Paritok-official/paritok-4b-v1 installs as a Python package and is described as a drop-in for Claude Code, Cursor, Codex, OpenHands and any BASE_URL agent. bassimeledath/dispatch takes the opposite approach to a single window: it turns a Claude Code session into a checklist writer and hands implementation to background agents with their own context windows, a mechanism the README describes clearly while being thin on everything else.

Two projects are more about discipline than tooling. mgechev/skills-best-practices is a guide plus a skill directory, not a runtime, offering rules for writing SKILL.md files that an LLM can route to, read and execute without bloating its context. s0xDk/ghostty-blackhole is a single GLSL fragment shader for the Ghostty terminal that beam-traces Schwarzschild geodesics live and grows with Claude Code's context fill; it is a visual gauge, not a monitor, and its pomodoro mode is hobbled by an unpopulated iDate uniform. None of these projects replaces reading your provider's token accounting.

Choosing between a bigger window and better context management

If a provider offers a larger window, using it is often the cheapest fix for an occasional long task. It is a poor fix for a structural problem. When an agent overflows every session, the cause is usually repeated or irrelevant content, and the same compression that saves tokens also improves reliability by keeping the important material near the end of the prompt.

A practical order of operations: measure first, then compress, then enlarge. Measuring means tracing actual token use per turn, which claude-devtools, abtop and tokentap do in different ways. Compressing means filtering tool schemas, trimming tool results and summarising stale history, which context-mode and Paritok claim to do. Enlarging means switching model or provider, which changes cost and latency and is the least reversible of the three.

Be sceptical of headline numbers. The 98% reduction in context-mode and the 25% to 85% savings in Paritok are project claims from their READMEs, not independent measurements, and they depend on workload. A short session with few tools will see far less benefit than a long saturated one.

In practice

The context window is a budget, not a feature. Treat it as shared, finite and costly, and decide per task what must be visible at once. To see what your own agent is spending, start with a read-only tracer such as matt1398/claude-devtools or graykode/abtop, then compare against a compression layer like mksglu/context-mode or Paritok-official/paritok-4b-v1. Read each project's README for its own claims and limits before adopting it.