klauspost/compress: pure Go zstd, S2 and drop-in deflate for Go services

What klauspost/compress solves that the Go standard library does not

The Go standard library gives you compress/flate, compress/gzip, compress/zlib and compress/lzw. It does not give you zstandard, and it does not give you a Snappy implementation at all. That gap is what this repository fills. The README lists zstandard compression and decompression in pure Go, S2 as a high performance replacement for Snappy, and optimized deflate packages described as a dropin replacement for gzip, zip and zlib. It also ships snappy as a drop-in replacement for github.com/golang/snappy, and lzw as a drop-in replacement for compress/lzw.

The audience is Go engineers who already know which codec they want and need it inside a Go binary. A service that has to decode zstd frames produced elsewhere, or that wants to stop shelling out to a C library, is the natural fit. The pure Go claim matters most for cross-compilation and for static binaries, because there is no cgo boundary to cross.

The repository is not one package. The top-level entries include separate directories for flate, fse, gzhttp, gzip, huff0, internal, lzw, s2, snappy, xpress, zip, zlib, zstd and dict, plus compressible.go at the root. You import the subpackage you need rather than a single facade.

How the packages are layered: entropy coders under format wrappers

The layout in the repository shows a deliberate split. huff0 and FSE are described in the README as implementations for raw entropy encoding. Those two are the low-level primitives. flate sits above them and provides the DEFLATE bitstream, and gzip, zip and zlib are format wrappers around flate. That is why the README can call the deflate packages a dropin replacement: the wrappers present the same surface as the standard library equivalents, so swapping the import path is the intended migration.

zstd is its own stack. The README points at a zstd subdirectory for the implementation, and the changelog records work at both the encoder and decoder level, including an arm64 decoder assembly path and true concurrent stream encoding. S2 is positioned as a faster Snappy, and snappy itself is kept as a drop-in replacement for the upstream golang/snappy package, which means the two coexist for different call sites.

Two packages sit outside the codec layer. gzhttp provides client and server wrappers for handling gzipped and zstd HTTP requests, and the changelog shows it gaining a zstandard option in the server handler wrapper. pgzip is a separate package, linked from the README, that provides a parallel gzip implementation. If your problem is HTTP response compression rather than file compression, gzhttp is the entry point, not flate.

Installing klauspost/compress and running a first zstd round trip

The README gives one install line, which adds the module at the latest version. Run it from the root of your own module.

The build tags are the real configuration surface

Most Go libraries have no build-time switches. This one has two, and both are documented in the README. The nounsafe tag disables all use of the unsafe package, and the noasm tag disables all assembly across packages. If your build environment forbids unsafe, or if you are targeting a platform where the assembly paths are not the ones you want, these tags are how you get a portable build.

That is a real trade-off rather than a footnote. The changelog for v1.18.0 mentions unsafe little endian loaders and a later entry mentions simplifying matchlen by removing assembly, and v1.19.0 adds an arm64 decoder assembly path for zstd. The fast paths and the safe paths are not the same code, so a build with nounsafe noasm is a different artifact from the default. The README does not state what the performance difference is, and this article will not guess at one.

The go.mod file declares go 1.25 and states that the package supports the current Go version and two versions back. If you are pinned to an older toolchain, that constraint applies before any of the codec questions do.

Retracted releases and the CVE note you have to read before upgrading

The go.mod retract block lists v1.18.1, v1.14.3, v1.14.2 and v1.14.1, each with a comment pointing at an issue or pull request. The changelog marks v1.18.1 as RETRACTED in its own heading. Go module tooling will refuse a retracted version unless you ask for it explicitly, so a normal go get will route around these, but a pinned go.sum in an older project may not.

The changelog also records that v1.18.3 was released to address a downstream CVE, referencing golang/go#77102. That entry does not describe the vulnerability in the repository text; it points at the Go issue. Anyone running a version before 1.18.3 should treat that pointer as the starting point rather than assuming the library itself was the vulnerable component.

This is the strongest argument for tracking releases here rather than vendoring once and forgetting. The library is a dependency of the toolchain's own compression surface in many projects, and the retract list shows that the maintainer uses Go's retraction mechanism rather than only publishing a new patch.

Where it is the wrong tool

If you need to compress a PDF, a JPEG or an MP4, this library does not do that. It implements general-purpose byte-stream codecs, not media-specific transforms. A JPEG is already entropy coded, and no amount of DEFLATE or zstd will shrink it meaningfully; the format wrappers here operate on whatever bytes you hand them.

If you want a command-line tool, this is also the wrong repository. It is a Go module. The README documents go get and build tags, not a binary you invoke. The one executable-adjacent item in the tree is gen.sh, which the repository layout places at the top level and which is not described in the README.

There is a third case worth naming. If your compression need is a single gzip stream and the standard library's compress/gzip already meets your throughput target, adding this dependency buys you a new import path and a new release cadence to track. The drop-in replacement argument cuts both ways: the API is the same, so the reason to switch has to be the implementation, not the interface.

How it compares with github.com/golang/snappy and pgzip

The closest comparison inside the Go ecosystem is github.com/golang/snappy, and the README addresses it directly: the snappy package here is described as a drop-in replacement offering better compression and concurrent streams. The difference is not a different algorithm. It is the same Snappy format with a different implementation, so the migration is an import path change and the on-disk or on-wire format stays compatible. That is a much smaller commitment than switching to zstd, which changes the format itself.

The second comparison is with pgzip, which the README lists as a separate package rather than a directory in this repository. pgzip provides a parallel gzip implementation. The distinction is that pgzip parallelizes gzip specifically, while this repository's flate and gzip packages are optimized single-stream implementations, and it is zstd that gets the concurrent stream encoding work per the v1.19.0 changelog. If your workload is many independent gzip streams, pgzip is the one the README points at. If it is one large stream and you want zstd, the zstd package here is the one.

The xpress package is a third case and an unusual one. It handles decompression of the Microsoft XPRESS (MS-XCA) plain LZ77 and LZ77+Huffman formats, and the README notes the LZ77+Huffman variant is the one used in WIM images and Windows Compact OS / WOF data. That is a read path for a Microsoft format, not a general-purpose codec choice.

Editorial conclusion

Adopt it when you need zstd or S2 in pure Go, or when you want a drop-in replacement for compress/flate, compress/gzip, compress/zlib or github.com/golang/snappy without changing call sites. Skip it if your only compression need is a single gzip stream and the standard library already meets your throughput target, since adding a dependency buys nothing there. Before upgrading, read the retract block in go.mod, which lists v1.18.1, v1.14.3, v1.14.2 and v1.14.1, and check whether your build depends on unsafe or assembly, because the nounsafe and noasm tags change which code paths compile in.

Frequently asked questions

Official sources

1. Issues
2. klauspost/compress on GitHub
3. README
4. Releases