entropy: hunting high entropy lines that turn out to be secrets

A signal that a line looks random, not a scanner that knows keys

entropy is a CLI tool that scans your codebase for high entropy lines, which are often secrets. The README states the problem in the first line, in the voice of someone who has seen it happen: paranoid about having secrets leaked in your huge codebase, and here to help find them.

The mechanism is a statistical one. A line of source code is mostly words, identifiers and punctuation, and it compresses well. A line containing a base64 encoded key, a random token or a hash is mostly noise, and it does not. Measuring that difference is what the tool does, and the name of the project is the name of the quantity.

What this approach buys is coverage without configuration. There is no rule file, no list of provider prefixes, no allow list of file types to teach it about AWS keys or GitHub tokens. It looks at every line you point it at and ranks by how random it looks.

What it gives up is specificity, and the project is honest enough that the reader can work this out. The tool knows nothing about what a valid credential looks like, so it cannot tell you that a 40 character string is a GitHub token rather than a base64 encoded certificate. It also cannot tell you whether a detected string is still live. Both of those need a scanner that knows key formats and provider revocation, which is a different kind of tool with a different cost.

The code is small in a way that suits the job. The repository is Go, with a main.go, a main_test.go, a testdata directory, a Makefile, a go.mod and a Dockerfile, and the only non-test dependency is golang.org/x/term, which is there for terminal handling. The module targets Go 1.23.0 with a toolchain directive of go1.24.0.

Installing with go install, brew or a container

There are three install routes, and the README marks the first as preferred.

From source with Go, the tool is installed and then run with no arguments, which scans the current directory.

go install github.com/EwenQuim/entropy@latest
entropy

If you do not want to install it, the same binary can be fetched and run in one step.

go run github.com/EwenQuim/entropy@latest

Homebrew is the second route, from the author's own tap.

brew install ewenquim/repo/entropy
entropy

The container image is the third, published on Docker Hub as ewenquim/entropy.

docker run --rm -v $(pwd):/data ewenquim/entropy /data

The volume mount is what makes the container useful, and the README explains the mechanics in one sentence: the -v option mounts the current directory into the container, and the /data directory is the default directory where the tool will look for files. Then it adds the warning that matters, and the warning is a real trap.

Don't forget to add /data at the end of the command, otherwise the tool will search inside the container, not your local filesystem. A container that mounts your repository and then scans its own empty layer will report nothing, and the natural reading of that result is that the code is clean.

The image itself is built in two stages, from a golang builder, with a scratch final stage that copies in the single static binary and the empty data directory.

FROM scratch

WORKDIR /bin

COPY --from=builder /go/src/entropy /bin
COPY --from=builder /data /data

ENTRYPOINT [ "entropy" ] 

A scratch image means no shell and no package manager inside the container, which keeps the image small and the attack surface small, and it is a good fit for a scan that only needs to read files.

Top N, extension filters, and the ignore list that carries the weight

The flags are few, and two of them do most of the work.

The -top flag sets how many results are shown. The default run in the README's examples is entropy -top 20 -ext go,py,js, which limits the output to the twenty highest entropy lines and restricts the scan to Go, Python and JavaScript files.

The -ext flag is therefore a scanner option as much as an output option. Restricting by extension means minified bundles, lock files, images and archives never enter the analysis, and it is the cheapest way to cut the false positive rate.

The -ignore-ext flag is the complement, and it is the one that carries the most weight in practice. The README's example is long.

entropy -top 5 -ignore-ext min.js,pdf,png,jpg,jpeg,zip,mp4,gif my-folder my-file1 my-file2

That single line shows the shape of a real run: a small top-N, an ignore list of minified JavaScript, documents, images, archives and video files, and then a set of paths to scan. The Homebrew section shows the same idea with a different ignore list, where the test file pattern _test.go replaces the media types, so the ignore list is where a project puts the things that look random but are not secret.

The paths at the end are positional arguments, and the README's examples pass a folder and individual files together, so both a directory tree and a specific list of files work.

The -h flag prints the options. Nothing in the README describes a configuration file, a baseline file, a rule set, an output format other than the printed lines, or a way to fail a build on a finding, so this is a reporting tool you read rather than a gate you wire in without modification.

A Makefile with three container targets and no test target

The Makefile is short, and its shape tells you something about how the project is developed.

docker-buildx:
	docker buildx build --platform linux/amd64,linux/arm64 --tag ewenquim/entropy:latest --push .

docker-run:
	docker run --rm -v $(pwd):/data ewenquim/entropy /data

docker-push:
	docker push ewenquim/entropy:latest

There are three targets and all three are about the container image. The build target is a buildx build for two platforms, linux/amd64 and linux/arm64, which is why the Dockerfile takes a TARGETARCH argument and passes it into the Go cross compile as GOARCH. That means the same image can run on an Apple silicon machine without a separate build.

What is missing is as informative as what is there. There is no test target, no lint target, no build target for the host, and no release target. The project has a main_test.go and a testdata directory, so tests exist and can be run with go test, and the CI configuration lives in the.github directory. But the Makefile is aimed squarely at producing and publishing the Docker image rather than at the Go development loop, which is consistent with a tool whose main distribution channel is the container on Docker Hub and the go install path.

The buildx push in the default target is worth a second look too. The docker-buildx target pushes as part of the build, so running make docker-buildx publishes an image tagged latest from a local checkout. That is a convenience for the maintainer and a sharp edge for anyone else who runs it, since a locally built binary pushed under the same tag as the published one is indistinguishable to a user pulling by tag.

Where a high entropy scan fits in a secret detection workflow

The honest way to position this tool is as a different layer from a scanner that knows credential formats.

A format-aware secret scanner works by pattern. It knows the shape of a GitHub token, of an AWS access key identifier, of a private key header, and of a connection string with a password in it. It can look in the places those things live, such as configuration files and environment variable assignments, and it can check whether a detected key has been revoked. The false negative is a key format nobody thought of, but the precision is high.

An entropy scan works the other way. It has no knowledge of any format, so it will surface a random looking string in a file nobody would think to check, in a language nobody thought to scan, in a format nobody anticipated. That is a genuine advantage, and it is why the tool is a reasonable sweep before a commit or before publishing a repository.

The false positive is the price, and the shape of the false positive is predictable. A base64 encoded PNG in a fixtures directory, a test vector that looks like a key, a minified bundle, a font file, a compressed archive. Every one of those is high entropy and none of them is a secret. The README's own examples show the author excluding exactly these categories by hand, which is the clearest possible signal that the tool's output needs curation before it means anything.

So the practical workflow is: run entropy across the whole tree first, read the top results, and treat them as a review queue rather than as findings. Then let a format-aware scanner handle the known key types. Running an entropy tool on a tree and believing the empty result is the failure mode, and it is the one the container volume warning is really about.

Editorial conclusion

entropy fits a developer who wants a fast, low-setup sweep of a tree for random looking strings before pushing code, and it does not fit a compliance pipeline, since it reports on entropy rather than on key format, and it does not fit a repository where false positives on minified assets and test fixtures outweigh the benefit. Before wiring it into a pre-commit hook, run it once over your whole tree with a top-N limit and an ignore list, look at what it actually reports, and confirm that your compiler and minifier outputs are excluded, because the README's own examples have to exclude minified JavaScript and test files by hand for the output to be useful.

Frequently asked questions

Official sources

1. EwenQuim/entropy on GitHub
2. Issues
3. License: MIT
4. README