aircompressor: Snappy, LZ4 and Zstandard for Java with native and pure Java paths

The same API over a native library or a Java implementation

aircompressor provides a set of compression algorithms implemented in pure Java and, where possible, native implementations. The two approaches use different mechanisms for memory access, and that is the interesting technical detail. The Java implementations use sun.misc.Unsafe to provide fast access to memory, and the native implementations use java.lang.foreign to interact directly with native libraries without the need for JNI.

That last part is the notable change. JNI has been the default route for calling a C library from the JVM for two decades, and it brings a native toolchain, a build step and the fragility of the boundary. The Foreign Function and Memory API is the platform's replacement for it, and using it means the native codecs are reachable without a compilation step in the consumer's build.

The API is deliberately small. Each algorithm provides a simple block compression API using the io.airlift.compress.v3.Compressor and io.airlift.compress.v3.Decompressor classes. Block compression is the simplest form, compressing a small block of data provided as a byte array or, more generally, as a java.lang.foreign.MemorySegment. Each algorithm may also have one or more streaming formats, which typically produce a sequence of block compressed chunks.

The byte array path looks like this:

byte[] data = ...

Compressor compressor = new Lz4JavaCompressor();
byte[] compressed = new byte[compressor.maxCompressedLength(data.length)];
int compressedSize = compressor.compress(data, 0, data.length, compressed, 0, compressed.length);

Decompressor decompressor = new Lz4JavaDecompressor();
byte[] uncompressed = new byte[data.length];
int uncompressedSize = decompressor.decompress(compressed, 0, compressedSize, uncompressed, 0, uncompressed.length);

And the MemorySegment path allocates from an Arena and reslices to the actual size afterwards, which is the pattern the Foreign API forces on you:

Arena arena = ...
MemorySegment data = ...

Compressor compressor = new Lz4JavaCompressor();
MemorySegment compressed = arena.allocate(compressor.maxCompressedLength(toIntExact(data.byteSize())));
int compressedSize = compressor.compress(data, compressed);
compressed = compressed.asSlice(0, compressedSize);

Decompressor decompressor = new Lz4JavaDecompressor();
MemorySegment uncompressed = arena.allocate(data.byteSize());
int uncompressedSize = decompressor.decompress(compressed, uncompressed);
uncompressed = uncompressed.asSlice(0, uncompressedSize);

The two code paths are near-identical in shape, which is the point: switching from an array to a segment does not change the algorithm, only the memory handling.

Zstandard and LZ4 are the recommended codecs

The library covers four compression algorithms and is opinionated about which to use.

Zstandard is the recommended algorithm for most compression. The stated reason is that it provides superior compression and performance at all levels compared to zlib, and it is described as an excellent choice for most use cases, especially storage and bandwidth constrained network transfer. The native implementation is ZstdNativeCompressor and ZstdNativeDecompressor, and the Java implementation is ZstdJavaCompressor and ZstdJavaDecompressor. The Zstandard streaming format is supported by ZstdInputStream and ZstdOutputStream.

LZ4 is described as an extremely fast compression algorithm that provides compression ratios comparable to Snappy and LZO, and it is the choice for applications that require high performance compression and decompression. Here too there is a native and a Java variant, Lz4NativeCompressor and Lz4JavaCompressor, and there is a separate pair for the LZ4 frame format, which is the format the lz4 command line tool produces. The framing itself is always performed in Java, and the implementations differ only in the raw block codec used for the block contents.

That last detail is a sensible design decision rather than a limitation. Framing is a small amount of logic and doing it once in Java removes a class of bug where the native and Java paths disagree about the container.

Snappy sits alongside them as a third option, and the trade-off is stated clearly. It is not as fast as LZ4, but it provides a guarantee on memory usage that makes it a good choice for extremely resource-limited environments such as embedded systems like a network switch. If your application is not highly resource constrained, LZ4 is the better choice. The native and Java variants are SnappyNativeCompressor and SnappyJavaCompressor, and the framed format is handled by SnappyFramedInputStream and SnappyFramedOutputStream.

LZO and Deflate are there for compatibility, not for new work

Two of the four algorithms are explicitly marked as legacy, and the library is candid about that.

LZO is only provided for compatibility with existing systems that use LZO, and the recommendation is to rewrite LZO data using Zstandard or LZ4. There is a Java implementation provided by LzoCompressor and LzoDecompressor, and due to licensing issues the LZO only has a Java implementation, which is based on LZ4. That last part is worth understanding: if you are reading LZO data today, you are not running the LZO algorithm, you are running an LZ4 based decoder that consumes LZO formatted data.

Deflate is in the same category. It is the block compression algorithm used by the gzip and zlib libraries, provided for compatibility with existing systems that use Deflate, with the recommendation to rewrite Deflate data using Zstandard. The implementation is DeflateCompressor and DeflateDecompressor, and it is implemented in the built-in Java libraries, which internally use the native code.

So the two legacy codecs are legacy in two different senses. LZO is legacy because of a licensing constraint on the original library and is emulated here. Deflate is legacy because zlib is already in the JVM and there is no reason to reimplement it.

That leaves a clean decision for a new project: use Zstandard or LZ4, and treat the other two as read-only paths for data you did not create. The naming of the classes follows the same pattern throughout, with a Native or Java infix in the class name telling you which path you are on, and the fact that the LZO classes have no infix at all is the visible reminder that there is only one path for that codec.

XXHash3, with a measurement of its own overhead

Beyond compression, the library provides hash functions, and the newest one comes with a note about its cost that you would rarely find in a readme.

XXHash3 is described as the latest generation of the XXHash family, providing faster hashing than XXHash64 at all input sizes, with support for both 64-bit and 128-bit hash outputs. It is only available as a native implementation via XxHash3Native, and there is no Java implementation.

The 128-bit variant has approximately 12 nanoseconds of constant overhead, due to Java FFM pulling the 128-bit result back into Java. At small inputs, under 512 bytes, this overhead is noticeable, and at larger sizes of 8KB and up it becomes a rounding error as the hash computation dominates. The measurement was taken on an M4 Apple Silicon, and the documentation says so.

Publishing the number is the useful part. A hash function used for a hash map key, a content hash or a sharding decision has a latency budget, and a fixed 12 nanosecond floor on every call is a real design constraint for a service doing many small hashes. A reader can now decide whether to use the 64-bit variant, accept the 128-bit one, or measure it on their own hardware.

The API covers one-shot and streaming in both widths:

// One-shot hashing (64-bit)
long hash = XxHash3Native.hash(data);

// One-shot hashing (128-bit)
XxHash128 hash = XxHash3Native.hash128(data);

// Streaming hashing (64-bit)
try (XxHash3Hasher hasher = XxHash3Native.newHasher()) {
    hasher.update(chunk1);
    hasher.update(chunk2);
    long hash = hasher.digest();
}

XXHash64 is the older extreme-fast non-cryptographic hash with excellent distribution properties, and here both a native and a Java implementation exist, XxHash64NativeHasher and XxHash64JavaHasher. The XxHash64Hasher interface provides static methods that automatically select the best available implementation, which is the convenient way to use it, since the caller does not have to know whether a native library was found on the machine.

A Maven project releasing every few days

The packaging and the release cadence are worth a short note, because the versioning tells you something about how the library is meant to be used.

It is a Maven project, with a pom.xml at the root, an mvnw wrapper along with mvnw.cmd, a.mvn/ directory for the wrapper configuration, and a license.txt and notice.md in the style Apache projects use. The published artifact is io.airlift aircompressor-v3, and the repository description is a port of Snappy, LZO, LZ4 and Zstandard to Java.

The version numbering is the interesting part. The recent releases are 3.13 from 2026-10-20, 3.12 from 2026-10-19 and 3.11 from 2026-10-16, which is three releases in five days, and the artifact name carries the major version, aircompressor-v3. The io.airlift.compress.v3 package name carries it too.

Releasing that often with a major version in the artifact name looks less like a stable library and more like a rolling delivery of codec fixes, where the codecs themselves are stable and the library is the moving part. That is a reasonable pattern for a component that sits under a data pipeline and needs to pick up a fix to a native library promptly.

The repository is not archived, the default branch is master, and the last push was on 2026-10-20, the same day as the newest release. The licence is Apache 2.0, with the standard notice file alongside.

There is a testdata/ directory at the root, which for a compression library is the fixture set that the round trip and interoperability tests run against, and a bin/ directory. The src/ directory holds the implementations, and the fact that a single repository contains both the pure Java codecs and the native bindings is what makes the fallback path coherent: the two are tested against each other rather than being separate projects.

Editorial conclusion

aircompressor suits a Java service that already speaks one of these compression formats and needs to read or write it without pulling in a native toolchain, and it suits a team that wants the same code path to work with or without native libraries. It does not suit a project that needs one algorithm and a small dependency tree, since the library covers four codecs and three hash functions, and the pure Java LZO path is a compatibility shim rather than a recommendation. Before adopting it, decide which algorithm you actually need, note that Zstandard and LZ4 are the recommended ones, and check whether the 128 bit XXHash3 overhead of roughly 12 nanoseconds matters at your smallest input sizes.

Frequently asked questions

Official sources

1. airlift/aircompressor on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases