Kanzi compresses in Go with Burrows-Wheeler, context modelling and block parallelism

A compressor that combines transforms instead of picking one paradigm

Kanzi is described as a modern, modular, portable and efficient lossless data compressor written in Go, and the four adjectives on that line each correspond to a design decision.

Modern means it implements state-of-the-art compression algorithms and is built to use multi-core CPUs through built-in multi-threading. Modular means entropy codecs and data transforms can be selected and combined at runtime to suit the specific data being compressed. Expandable means a clean, interface-driven design with no external dependencies, which makes it easy to integrate, extend and customise. Efficient means it is carefully optimised to balance compression ratio and speed for practical, high-performance usage.

The most consequential of these is modularity, because it is what makes the rest of the project different. Most mainstream lossless compressors are one paradigm with a level setting. Kanzi is a pipeline where a transform stage is chosen and an entropy coder stage is chosen, and the composition is the configuration. The README states the point directly: unlike most mainstream lossless compressors, Kanzi is not limited to a single compression paradigm, and by combining multiple algorithms and techniques it supports a broader range of compression ratios and adapts better to diverse data types.

The topic list gives the algorithmic vocabulary: asymmetric numeral systems, BWT for the Burrows-Wheeler transform, Huffman coding, and LZ77. That mix is what a BWT plus context modelling pipeline is built from, and it is the mechanism behind the ratio claims.

The licence is Apache-2.0, the module is github.com/flanglet/kanzi-go, the go.mod requires Go 1.19, and the last push was on 2026-10-18 with the newest release v2.6.0 from 2026-09-20.

Why the project argues against Lempel-Ziv

The Why Kanzi section is a positioning argument, and it is worth reading because it explains the whole design.

It starts from an acknowledgement: while excellent open-source compressors like zstd and brotli exist, they are primarily based on Lempel-Ziv algorithms, and zstd in particular is a fantastic general-purpose choice known for its speed. The criticism is about ratios, not about speed.

The first point is Beyond LZ: by incorporating the Burrows-Wheeler transform and context modelling, Kanzi can achieve compression ratios that traditional LZ methods cannot.

The second is speed where it counts. The argument is that LZ is ideal for the compress once, decompress often pattern, like software distribution, but that it slows down significantly at high compression settings. Kanzi is said to leverage multi-core CPUs to maintain performance, which makes it effective for backups, real-time data generation and one-off transfers. That is a deliberate positioning away from the distribution use case and towards the cases where you compress once and either keep the result or decompress it yourself.

The third is content-aware. Kanzi has built-in, customizable transforms for specific data types, for example multimedia, DNA and UTF text, said to improve efficiency where generic compressors fail. This is the part a general-purpose codec cannot offer, because it does not know what kind of data it is looking at.

The fourth is extensibility, with an architecture described as developer-friendly, making it straightforward to implement new transforms or entropy codecs for experimentation or niche data types.

The project also publishes a C++ implementation and a Java implementation, so the design is not confined to Go even though this repository is.

The benchmark table, read as a trade-off curve

The README publishes benchmark tables for Kanzi version 2.6.0 on two corpora, and they are more informative than the marketing text because they show the shape of the trade-off.

The silesia.tar table was measured on an AMD Ryzen 9950X on Ubuntu 26.04 with Go version 1.27.1, against a corpus of 211,957,760 bytes. Reading the Kanzi rows, level 1 encodes in 157 ms and decodes in 69 ms for 79,184,957 bytes, level 2 gives 137 ms and 84 ms for 68,627,321 bytes, level 3 gives 232 ms and 111 ms for 63,093,409 bytes, and by level 6 the figure is 1295 ms encoding and 1056 ms decoding for 49,472,110 bytes, with level 9 at 16397 ms and 16991 ms for 41,520,677 bytes.

Compare that with the other tools in the same table. zstd 1.6.0 at level 2 encodes in 57 ms and decodes in 25 ms for 69,443,247 bytes, and at level 13 it decodes in 26 ms for 57,843,283 bytes. brotli 1.1.0 at level 9 takes 23030 ms to encode. xz 5.8.1 at level 9 takes 43611 ms to encode and 931 ms to decode. bzip3 with 16 jobs lands at 42,962,926 bytes, essentially matching Kanzi level 8.

Two things follow. Kanzi level 2 beats zstd level 2 on size by about 800,000 bytes while encoding more slowly, and by the higher levels Kanzi reaches sizes that zstd only reaches at settings where zstd's own encoding time explodes. But the decode side tells the other half of the story: Kanzi's decode time rises with level, from 69 ms at level 1 to 16991 ms at level 9, while zstd's decode time stays flat at roughly 25 ms even at level 19. For a workload that compresses once and decompresses many times, that flat decode curve is exactly why zstd is the default answer, and the README's own framing concedes it by positioning Kanzi for backups and one-off transfers.

The enwik8 table was measured on an Apple M3 with 24 GB running Sonoma 15.7.3 with Go version 1.26.1, over a 100,000,000 byte corpus, where Kanzi level 1 gives 42,941,668 bytes at 169 ms and 51 ms, and level 2 gives 37,6 million bytes at 161 ms and 67 ms, with the table continuing past the visible text.

A compressor, not an archiver, and the seekable bitstream

The README is explicit about a boundary that is easy to misread, and it is worth quoting the shape of it. Kanzi is a data compressor, not an archiver. It includes optional checksums for verifying data integrity, but does not provide features like cross-file deduplication or data recovery mechanisms.

That removes a whole category of expectation. There is no container format, no directory structure, no per-file metadata, and no recovery if a block is lost. What you get is a compressed representation of a byte stream.

What you do get, and it is a genuinely useful property, is a seekable bitstream. The README explains that one or more consecutive blocks can be decompressed independently, without needing to process the entire stream. That is the property that makes Kanzi usable on formats where random access matters, and it is what lets the parallel block compression and the concurrent decompression work: if blocks were not independently decodable, you could not compress them in parallel and still produce a stream that can be read from the middle.

The concurrency claim is also specific. The README says most traditional compressors underutilise modern hardware by running single-threaded, even on machines with many cores, and that Kanzi is concurrent by design, compressing multiple blocks in parallel across threads for significant performance gains. It also states plainly that Kanzi is not compatible with standard compression formats, which is the flip side of the same design decision.

The module structure reflects the modularity claim. The repository top level is small, with a v2 directory holding the versioned source, a go.mod, a SECURITY.md and a LICENSE. The module file reads:

module github.com/flanglet/kanzi-go

go 1.19

retract [v1.4.0, v1.9.0]

The retract directive withdraws the version range from v1.4.0 to v1.9.0 from the module proxy, which tells you part of the v1 line had a problem worth hiding from users who resolve by a version range.

Where Kanzi is the wrong tool

Three limits, and the first one is about formats rather than about performance.

The first is interoperability. Kanzi is not compatible with standard compression formats, which means a Kanzi-compressed file can only be read by a Kanzi decompressor. If your data crosses an organisational boundary, if it is served to a browser, or if it has to be read by a service you do not control, this is a non-starter and no ratio or speed advantage compensates. The same applies to longevity: a stream written by a given Kanzi version is not a documented interchange format, and the project is at v2.6.0 with the module still carrying retract directives for v1 releases, which is a reminder that the format has changed before.

The second is the decode cost at high levels, and the benchmark table is explicit about it. At level 9 on silesia.tar, Kanzi decodes in 16991 ms while zstd at level 19 decodes in 23 ms. If anything in your system decompresses the same data more than once, a high Kanzi level can be a net loss even though the compressed size is smaller.

The third is that the modularity has a cost in predictability. The runtime selection of transforms and entropy coders is what makes Kanzi adapt to diverse data, and it is also what makes the behaviour of a given input depend on configuration rather than on a fixed algorithm, so a size you measure on one dataset is not a guarantee on another. For a project that needs reproducible output with an auditable algorithm, that is a real drawback.

The alternative approach, and the reason most people will not use Kanzi, is to use zstd or brotli, which the README acknowledges are excellent. The difference in approach is that a mainstream codec offers one well-understood paradigm with predictable behaviour and a format everyone can read, while Kanzi offers a configurable pipeline that can beat those codecs on ratio and can use your cores, at the cost of a private format and a per-dataset configuration decision. If your data is genuinely LZ-hostile, for example highly redundant structured or numeric data, that trade can be worth it. If you are not sure, it is not.

Editorial conclusion

Adopt Kanzi if you compress and decompress in the same ecosystem, you can accept a non-standard bitstream, and your data compresses poorly with Lempel-Ziv methods, since the transform and entropy stages can be combined at runtime and the block parallelism uses the cores your machine has. Do not adopt it if you need an interchange format, because the output is not compatible with standard compression formats and the decompressor must be Kanzi itself. Verify first that the version you pin is v2.6.0 or later, since the benchmark table in the README is measured on that release, that the Go version you build with satisfies the module requirement, and that your workload tolerates the decode cost at the compression level you choose, since the benchmark shows decode time rising sharply at the higher levels.

Frequently asked questions

Official sources

1. flanglet/kanzi-go on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases