sammcj/ingest: Turn a Code Repo into One Markdown Prompt for an LLM

The gap sammcj/ingest fills between a repo and a chat window

Handing a codebase to a language model usually means a sequence of manual steps: list the directories, open the files that matter, paste them in some order, and hope the total fits. The README describes ingest as a tool that "parses directories of plain text files, such as source code, into a single markdown file suitable for ingestion by AI/LLMs." That single sentence is the whole product thesis. It is a packaging tool, not an analyser.

The intended user is an engineer who already knows which project they want explained and does not want to assemble the context by hand. The README's own example output shows what that looks like in practice: a traversal message, a token estimate such as 15,945, and a confirmation that the result landed on the clipboard. From there the text goes into a model, a ticket, or a file.

It is worth being clear about what it is not. There is no index, no embedding store, no retrieval step, and no server. Each invocation produces one document and exits. That constraint is the reason the tool is small enough to install with a single Go command, and it is also the reason it will not help you query a repository larger than your context window.

How ingest builds the prompt: traversal, tree, tokeniser, template

The data flow is linear. Ingest walks the paths you give it, applies include and exclude glob patterns, and builds a tree view of what it found. Each selected file is read as text and rendered into a markdown document alongside that tree. Git diffs and logs can be folded in. A template controls the final layout, and the finished document goes to the clipboard, a file, or stdout.

Two mechanisms in that pipeline deserve attention. The first is token counting. The README states that the o200k_base and cl100k_base vocabularies are compiled into the binary, so counting happens offline and nothing is downloaded or cached at runtime. For Claude models, which have no published tokeniser, ingest counts with o200k_base and then scales the result: a factor of 1.53 for Opus 4.7 and later, Opus 5, Sonnet 5 and Fable 5, and 1.18 for earlier Claude models. Non-Claude models are counted with their own vocabulary and no scaling. The README is explicit that both factors are approximations that vary with content.

The second is Tree-sitter compression. Passing --compress extracts package and module declarations, imports, function and method signatures without bodies, class definitions without method bodies, type definitions, and comments. The README marks the feature experimental and lists six supported languages: Go, Python, JavaScript, Bash, C and CSS. Anything outside that list is passed through uncompressed, which means a --compress run over a mixed repository will strip structure from some files and leave others at full length. The token estimate you get back reflects that mixture, not a uniform compression ratio. The go.mod file confirms the dependency is github.com/smacker/go-tree-sitter.

Installing sammcj/ingest and running it on a real project

The README recommends the Go toolchain route and notes that the curl script is harder to update. Either way the binary is named ingest and takes a path.

go install github.com/sammcj/ingest@HEAD

After that command completes, the ingest binary should be in your Go bin directory. Running it with no arguments defaults to the current working directory, which the README shows producing a traversal spinner, a token estimate, and a clipboard confirmation. If the clipboard is unavailable, the same content can be written to a file instead.

ingest -i "**/*.py" /path/to/project

The -i flag takes a glob and restricts the run to Python files, which is the quickest way to keep a large repository inside a context window. To capture the current state of a working tree rather than the committed state, the -d flag adds the git diff and log.

ingest -d /path/to/project

If you would rather keep the output as an artefact than paste it, -o writes it to a named file, and --save writes to a path under ~/ingest named after the directory. The --llm flag skips the clipboard entirely and sends the generated prompt to an OpenAI compatible API, with -p appending a prompt suffix.

ingest --llm -p "explain this code" /path/to/project

One inconsistency is visible in the README itself. The feature list says the tokeniser vocabularies are compiled into the binary and that ingest "never downloads or caches anything," while the token counting section says the first run downloads a small tokeniser file. Both statements cannot describe the same behaviour, and the README does not resolve which applies to the current release.

Where ingest breaks down or is the wrong tool

The token count is the weakest link. Without an API key you get an approximation with an admitted correction factor, and the README's own framing is that the multipliers vary with content. If you are deciding whether a prompt fits inside a hard context limit, an estimate that can drift by a few percent is exactly the number you cannot afford to be wrong about. The escape hatch is -a or --anthropic, which calls Anthropic's count_tokens endpoint with the same --model value. The README says the call is free but requires ANTHROPIC_API_KEY, and that a failed API call falls back to the offline tokeniser silently. That fallback is a design choice worth noticing: you may believe you got an exact count when you did not.

Compression has a narrower reach than the feature list implies. Six languages are supported, so a repository in Rust, Java, Ruby or TypeScript will not be compressed at all. The README labels the whole feature experimental, which is a fair signal that signatures-only output can drop the context a model needs to answer a question about a specific implementation.

Glob patterns cut both ways. Include patterns narrow a run, but a pattern that is too broad will happily sweep in configuration files, fixtures and anything else that is plain text. The README documents include and exclude patterns and mentions gitignore handling through a dependency, but it does not describe a secret-scanning step. If you point ingest at a repository root and send the result to a hosted API, that is a decision you are making, not one the tool is making for you.

Finally, the tool has no notion of a repository too large to process. There is no chunking, no summarisation pass, and no retrieval. A monorepo will produce a document that simply does not fit.

ingest compared with repomix and similar packers

The closest comparison in this space is repomix, a Node.js tool that packs a repository into a single AI-friendly file. The difference in approach is mostly about where the intelligence sits. repomix is built around the packing step and runs on Node; ingest is a single Go binary, which matters if you want it on a machine without a JavaScript runtime, and it ships token counting inside the binary rather than leaving that to the caller.

A second difference is the compression strategy. repomix offers its own tree-sitter based compression across a broader language set. Ingest's compression covers Go, Python, JavaScript, Bash, C and CSS, and the README calls it experimental. If your repository is mostly TypeScript, repomix compresses it and ingest does not, which is a concrete reason to pick one over the other rather than a matter of taste.

The third difference is what happens after packing. Ingest can post the prompt straight to an OpenAI compatible endpoint with --llm, and it can crawl web pages and convert PDFs, so it covers more than local source trees. repomix stays on the packing side. If you already have a script that pipes a file into a model, that extra capability is dead weight; if you do not, it saves you a step.

Neither tool solves context limits. Both produce one document. The choice is about language coverage, runtime, and whether you want the model call inside the same command.

Maintenance, licence and the cost of upgrading ingest

The repository is not archived, and the last push was on 2026-08-11, which is the same date as the v0.15.6 release. The two releases before it, v0.15.5 and v0.15.4, are dated 2026-01-19 and 2026-01-18. That pattern, a burst of releases in January and then a single release in August, is worth reading before you plan around frequent updates. The version numbering stayed in the 0.15.x line across the whole period, so the project has not declared a stable interface.

Upgrade cost is low if you install through go install, because the version is whatever HEAD points at and there is no lockfile to maintain. That is also the risk: go install github.com/sammcj/ingest@HEAD pulls the current default branch rather than a tagged release, so an upgrade can change flag behaviour without a version bump you chose. Pinning to a tag from the releases page and building with the Makefile is the more predictable route. The Makefile stamps the binary with git describe --tags --always and a build time, so ingest --version or the equivalent reports which build you are running.

Licensing is MIT, which is permissive and imposes no conditions beyond retaining the copyright notice. The dependencies listed in go.mod carry their own licences, and the README does not enumerate them. If you redistribute a built binary, check those separately; nothing here is legal advice.

Editorial conclusion

Adopt sammcj/ingest if you regularly paste directory trees and source files into a chat window or an OpenAI compatible endpoint and want the token budget known before you send. Skip it if you need exact Claude token counts without an API key, if your language is outside the six Tree-sitter grammars, or if your source lives in a monorepo where a glob mistake silently ships secrets. Verify first: run ingest --no-correction on a small directory and compare the reported count against your provider's own counter, and check that your .gitignore excludes .env files before you ever point it at a real repository.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. sammcj/ingest on GitHub