repo-to-prompt: pasting a repository into a chat without blowing the context window

Show & TellNico3 replies

NicoMember#1

/tools/repo-to-prompt takes a set of files (drop a folder, or paste) and produces one text block with a file tree and each file's content fenced and labelled, in the shape models handle well. It runs in the browser; the code does not leave your machine.

The useful part is what it leaves out. By default it skips lockfiles, build output, node_modules and friends, binary files and anything in .gitignore if you include it. Each of those is thousands of tokens of nothing. You can add your own exclusion patterns.

Pair it with /tools/token-counter before you paste: a mid-sized repository is often 300k tokens with everything and 40k with the noise removed, and the second number fits in most models while the first does not. If it still does not fit, the tree alone plus the five files you are asking about is almost always better than the whole thing truncated. /learn/context-window explains why "it fits" is not the same as "the model uses it".

If you use a coding agent, you probably do not need this; the agent reads files itself. It is for the chat window, for the review question, for "explain this codebase to me".

3 replies

KaceyMember#2

Pairs with /tools/gitignore-generator too: generate the ignore file for the stack first, feed it in, and the exclusion list writes itself. I had been maintaining the exclusions by hand.

MookMember#3

What happens with binary files that are not obviously binary by extension, like a .dat or a file with no extension?

NicoAuthorMember#4

In reply to Mook (#3)

It sniffs the first bytes: anything with null bytes or a high share of non-text bytes is listed in the tree with its size and skipped, regardless of extension. Extensionless text files (Dockerfile, Makefile, LICENSE) come through as text. If it gets one wrong, the tree entry tells you, and you can force-include.