ulikunitz/xz: pure Go xz compression with no C dependency

A pure Go xz implementation, which means no liblzma anywhere

ulikunitz/xz is a Go package that supports reading and writing of xz compressed streams, and the point of it is stated in the README's first paragraph: the package is completely written in Go and does not have any dependency on any C code. That is the entire pitch, and it is a pitch about deployment rather than about features. A Go service that needs to read a .xz archive can either bind to liblzma through cgo, which drags in a native build step and a system library version to manage, or shell out to the xz binary, which means a subprocess in the request path. This package removes both. The go.mod is correspondingly bare, declaring a module path and a Go 1.20 directive with no require block at all, since the standard library and the package itself are the whole dependency set.

The API is a Writer and a Reader over the standard io interfaces

The API example in the README is the whole surface, and it is short:

package main

import (
    "bytes"
    "io"
    "log"
    "os"

    "github.com/ulikunitz/xz"
)

func main() {
    const text = "The quick brown fox jumps over the lazy dog.\n"
    var buf bytes.Buffer
    // compress text
    w, err := xz.NewWriter(&buf)
    if err != nil {
        log.Fatalf("xz.NewWriter error %s", err)
    }
    if _, err := io.WriteString(w, text); err != nil {
        log.Fatalf("WriteString error %s", err)
    }
    if err := w.Close(); err != nil {
        log.Fatalf("w.Close error %s", err)
    }
    // decompress buffer and write output to stdout
    r, err := xz.NewReader(&buf)
    if err != nil {
        log.Fatalf("NewReader error %s", err)
    }
    if _, err = io.Copy(os.Stdout, r); err != nil {
        log.Fatalf("io.Copy error %s", err)
    }
}

The shape is what a Go programmer expects. `xz.NewWriter` returns a writer that takes bytes and produces an xz stream, `w.Close()` finalises the stream and flushes the trailing checksum and footer, and `xz.NewReader` turns a reader into a decompressor. Because both ends are ordinary `io.Writer` and `io.Reader` values, the package drops into any pipeline without adapters, and the example's `io.Copy(os.Stdout, r)` shows the read side being used exactly that way. The error handling in the sample is the part worth copying: every step checks its error and the writer is closed before the reader is created, which matters because an unclosed xz writer leaves an incomplete stream that the reader would reject.

The gxz command compresses a file in place

The package also ships a command line utility named gxz for compression and decompression, which is installed like any other Go program:

    $ go get github.com/ulikunitz/xz/cmd/gxz

    $ gxz bigfile

    $ gxz -d bigfile.xz

The behaviour is in-place rather than streaming to standard output. After some time a much smaller file bigfile.xz will replace bigfile, and to decompress it you use the second command. This is a different convention from gzip, where `gzip -d` typically writes to standard output, so a script that pipes xz output somewhere has to be rewritten. The file naming follows the `.xz` suffix convention. The command lives under `cmd/gxz` in the repository, alongside the `cmd/` directory that also holds the package's own test fixtures such as `fox.xz` and `fox-check-none.xz`.

The implementation is split into filters, LZMA and the container format

The repository layout shows how the format is decomposed, and it is a useful map for anyone reading the source. `lzma/` holds the LZMA2 codec, `lzmafilter.go` provides the filter layer that chains compression stages, `format.go` and `reader.go` and `writer.go` implement the xz container itself, `bits.go` and `crc.go` handle the low-level bit reading and the check types, and `none-check.go` covers the no-check variant. The `xio/` directory holds extra IO helpers, and `internal/` holds unexported pieces. `format_test.go` and `xz_test.go` sit alongside the source, and `testdata/` holds the corpus the tests read. The presence of a `none-check` implementation alongside the default one is a detail worth noticing, because xz streams can carry different integrity check types and a reader that only understood the default would reject valid files.

The README is unusually direct about where this falls short

The status paragraph is the part of the README worth quoting, because it is rare to see a compression library state its limits before you find them. The package is currently under development, there might be bugs, and the APIs are not considered stable. At this time the package cannot compete with the xz tool regarding compression speed and size, because the algorithms there have been developed over a long time and are highly optimized. The author adds that a number of improvements are planned and is optimistic about parallel compression and decompression. That is a fair description of the situation for any pure-language reimplementation of a C library: you inherit the format and the semantics, but not the two decades of tuning that the reference implementation carries, and the parallel work is the thing that would change the picture.

The supply chain story is the strongest argument for this package

The security section is unusually relevant given the history of the format. The README states that the software is not affected by the supply chain attack on the original xz implementation, CVE-2024-3094, and explains why: this implementation does not share any files with the original xz implementation. It also notes that no patches or pull requests are accepted without a review. That is a concrete property rather than a promise, because the whole point of the pure Go rewrite is that there is no liblzma code in the dependency graph to be backdoored. The security policy is documented in SECURITY.md, and all security advisories for the project are published on the GitHub advisories page for the repository, so the disclosure process is visible rather than private.

Where ulikunitz/xz is the wrong tool

Two limits are concrete and one is administrative. The compression limit is the one the README itself states: if you are archiving data, where the ratio is the point and the file is written once and read rarely, a smaller output wins and this package loses, so you want the reference xz tool, or a Go binding to liblzma, or a different format entirely. The second limit is API stability, since the APIs are not considered stable and the package is under development, which means a Go module version bump can break your build. The administrative limit is the licence. The repository's licence is classified as a custom one that GitHub cannot classify, which is unusual and means you have to read the LICENSE file before using this in a product, and a custom licence may carry terms the standard ones do not. The module also declares Go 1.20, which is fine for most projects but is a floor rather than a promise of newer language features.

The alternative is a cgo binding, with the trade-off stated plainly

The other common way to get xz into a Go program is a cgo binding to liblzma, and the difference in approach is exactly the one this package is arguing about. A binding inherits the reference implementation's compression ratio and its tuned performance, because it is the same code, and it inherits its build requirements: a C toolchain, a system library that must be present or statically linked, and cgo's consequences for cross compilation and for the pure-Go nature of the binary. The pure Go package inverts that: the ratio and speed are worse, the build stays simple, the binary stays static, and the dependency graph contains nothing that can carry a liblzma compromise. Between them sits the shell-out option, which gets the reference tool's output quality and adds a subprocess to the design. For a service reading archives produced elsewhere, the ratio usually matters less than the deployment story.

Editorial conclusion

ulikunitz/xz fits a Go service that must read or write .xz data without linking a C library or shelling out to the xz binary, and that accepts a smaller compression ratio in exchange for a single static binary and no supply chain exposure to liblzma. It is the wrong tool for archival storage where ratio matters most, since the README says plainly that the package cannot compete with the xz tool on compression speed and size. Before you adopt it, check the LICENSE file yourself, since the project is classified as a custom licence rather than a standard one, and read TODO.md, since parallel compression and decompression are planned improvements rather than current behaviour.

Frequently asked questions

Official sources

1. Issues
2. README
3. ulikunitz/xz on GitHub