distill: a local 1.7B model that rewrites shell output before your agent reads it

The token bill that comes from piping raw commands into an agent

An agent that runs rg, a test suite or a build and then feeds stdout back into the context window is paying for text nobody reads. A stack trace with forty frames, a test summary with two hundred passing lines, a grep that matched across a vendor directory: the model needs the one line that says what broke, and it receives everything. distill targets exactly that gap. It is a command line tool you place at the end of a pipe, and its job is to turn the left side of the pipe into a short answer for the right side. The README frames the problem in one sentence, that agent command outputs are one of the biggest sources of token waste, and the example it gives is a ripgrep search across a desktop codebase with a natural language question attached. The intended user is someone running a coding agent locally, on macOS, who has noticed that context fills up faster than the work justifies. It is not a logging tool, not a search tool, and not a general text summarizer.

One pipe, one prompt, one bundled model

The mechanism is a Unix pipe with an extra argument. You write your command, then a pipe, then distill followed by a quoted question describing what you actually want to know. The README example is rg -n with a glob exclusion, piped into distill with the question find where terminal and permission UI are implemented in chat screen. So distill reads stdin, treats the quoted string as the instruction, and produces the compressed answer on stdout. The compression is not a regex filter or a line-count heuristic. According to the README, distill uses its own Expert Language Model, published as distill-1.7B-MLX on Hugging Face, at 1.7B parameters in 4-bit quantization. That means the summarization step is a local inference call, not an API request, which is the reason the tool can claim token savings at all: the tokens are never sent to a hosted model. The README also mentions a Distill Language, described as teaching your LLM to talk and think in a more efficient way, illustrated by an image rather than a specification. That part is the least documented piece of the project and I would treat it as separate from the pipe.

Install and the RAM number that decides everything

Installation is a global npm package and a single onboarding command. The README gives npm i -g @samuelfaj/distill, then running distill with no arguments to start onboarding, followed by the claim that this is all that is required. The package name is scoped, so the binary and the package name differ: you install @samuelfaj/distill and invoke distill. The hardware guidance is stated plainly: the model is 1.7B at 4-bit, the safe recommendation is 8 GB of RAM or more, and 16 GB or more is comfortable. That is the constraint to read first. The model identifier carries MLX in its name, which is Apple's array framework, so the practical target is Apple Silicon. The README does not describe a CPU fallback, a CUDA path or a hosted option. If you are on a Linux CI runner or a Windows workstation, nothing in the supplied material says distill will start. The repository is TypeScript, but the inference weight is a separate downloaded model, so the npm install is not the whole story.

The 98.7% figure is one example, not a guarantee

The headline number in the README is up to 99% of tokens, and the worked example is more specific: before, 7648 tokens and 30592 characters; after, 99 tokens and 396 characters; saved, roughly 98.7%. Both files are linked in the repository under examples/1 as BEFORE.md and AFTER.md. Read that as a single measurement on a single command, not as a property of the tool. Compression ratio depends on how much of your output is repetition, and a grep across a large tree with a narrow question is close to the best case. A short command whose output is already dense will compress less, and there is no published table of ratios across command types. There is also a correctness cost that the README does not discuss. When a 1.7B model decides which lines matter, it can drop the line that mattered, and the agent downstream has no way to know something was removed. For interactive debugging where you are reading the output yourself, piping through distill is the wrong move: you are paying inference latency to lose information you wanted.

Where distill sits next to a general-purpose agent CLI

The obvious comparison is a coding agent that already manages its own context, such as Claude Code or Codex, both of which appear in this repository's topics. Those tools decide internally what to keep in the window and often run their own summarization when context grows. The difference in approach is where the compression happens. A general agent compresses after the output has entered its process, using whatever model it is already talking to, and the raw text is still available to it. distill compresses before the agent sees anything, at the shell boundary, using a small local model that has no knowledge of the task beyond the quoted question. That boundary placement is the whole design: it works with any agent, or with no agent at all, because the interface is stdin and stdout. The trade is that distill has no conversation context, no file access and no follow-up. If your question needs a second look at the output, the output is gone.

Maintenance, distribution and the licence gap

The release history shows three versions shipped on the same day in May 2026, v1.5.0 through v1.5.2, and the last push to the default branch is dated June 2026. That pattern, a cluster of patch releases followed by a quiet month, is common for a small tool and says nothing about whether it will be maintained next year. There is a more concrete cost: the model weights are a separate artifact hosted on Hugging Face, so a working install depends on that download remaining available and on the MLX runtime continuing to support the quantization format. Upgrading the npm package can change which model version it expects. On licensing, the repository metadata supplied here lists the licence as unknown, and the model card is a separate document with its own terms. Before shipping distill inside a commercial product, read both the repository licence file and the Hugging Face model card, and treat the absence of a stated licence as a question for your legal team rather than an assumption. The README also promotes a hosted client at remotecode.io at the top of the page, which is a separate product and not covered by anything else in this material.

Who should put distill in the pipe

The case for distill is narrow and clear. You run a coding agent on an Apple Silicon laptop with 16 GB of RAM, your context window is being consumed by test output and recursive greps, and you are comfortable with the agent seeing a paraphrase rather than the raw text. In that setup the install is one npm command and the usage is one pipe, and the savings are real because the alternative is sending those characters to a hosted model. The case against is equally clear. Linux and Windows users have no documented path. Machines under 8 GB are outside the stated safe range. Anyone who needs the complete output for a bug report, a compliance record or a postmortem should not pipe through a summarizer at all. And anyone who wants a supported, versioned, licence-clear dependency should note that the licence is listed as unknown here. The cheapest way to settle it is the README's own example: run your noisiest command twice, once raw and once piped into distill with a question, and diff what the model kept against what you needed. If the missing lines are the ones you would have grepped for anyway, the tool is doing its job.

Editorial conclusion

Adopt distill if your agent runs on an Apple Silicon machine with 16 GB of RAM or more and your token bill is dominated by ripgrep, test runner and log output that you never read yourself. Skip it if you are on Linux or Windows, if you are under 8 GB, or if you need the full output for auditing, because the compressed answer is what reaches the model and the original is not part of the pipeline. Before trusting it, run the exact command from the README on one of your own noisy commands and compare the AFTER file against the raw output to see what the 1.7B model dropped.

Official sources

1. Issues
2. README
3. Releases
4. samuelfaj/distill on GitHub