density: compression built on four-byte work units

The work unit is four bytes, and that explains everything

Density is a free, pure Rust, open source compression library, formerly C99, and the project describes itself as a superfast compression library focused on high-speed compression at the best ratio possible. The design decision that drives everything else is stated up front. Unlike most compression algorithms, its work unit is not a byte but a group of four bytes, so where other libraries consume one byte of data to apply their algorithmic processing, density consumes four. That is why its algorithms were designed from scratch rather than adapted, because they cater for four-byte work units and still provide interesting compression ratios. The claim about position is that the algorithms are currently at the pareto frontier of compression speed versus ratio, with a link to an independent benchmark on the quixdb squash benchmark site as the evidence offered. The licence is dual, MIT or Apache-2.0, the crate is `density-rs`, and the last push was on 2026-07-12 with v0.16.6 released on 2025-08-27.

The five speed pedigree traits

The README lists the traits that produce the speed, and they are worth reading as a design brief rather than as a list. Four-byte work units come first. Then heavy use of registers as opposed to memory, which is the direct consequence of the work unit size, since a four-byte word is what a register holds naturally. Then avoidance of, or minimal use of, branching where possible, because a branch misprediction costs more than the instruction it skips. Then use of low memory data structures to favour processor cache Lx storage, so the working set stays in cache rather than spilling to main memory. And then library-wide inlining. Every one of those is a memory or branch cost decision, which is the part that distinguishes a compression library designed for throughput from one designed for ratio. Blowup protection is described separately, and it is a correctness feature as much as a speed one, dramatically increasing the processing speed of incompressible input data with the aim of never exceeding the original data size even for incompressible inputs.

Three algorithms, ranked on speed and on ratio

The library ships three algorithms and the README tabulates their trade-offs. Chameleon ranks first for speed and third for ratio, with one dictionary unit, zero prediction units and a signature size of 8 bytes. Cheetah ranks second on both, with two dictionary units, one prediction unit and an 8 byte signature. Lion ranks third for speed and first for ratio, with two dictionary units, five prediction units and a 6 byte signature. Chameleon is a dictionary lookup based algorithm designed for absolute speed, on the order of GB/s, for both compression and decompression. Cheetah, developed with inputs from Piotr Tarsa, is derived from chameleon and uses swapped dual dictionary lookups with a single prediction unit. Lion is derived from chameleon and cheetah, using different data structures, dual dictionary lookups and five prediction units, which gives it a compression ratio advantage on moderate to highly compressible data. So the choice is a dial, and the dial is set by how compressible your data is.

Building it, and running the benchmark on your own data

The build is the standard Rust path, and the README assumes rustup is used to install rust. You clone and build:

git clone https://github.com/g1mv/density.git
cd density

cargo build
cargo test

The benchmark is where this library makes its case, and you can point it at your own files rather than trusting the published numbers. The integrated benchmark is single-core and in-memory, and it also covers lz4 and snappy for comparison using the dickens file from the silesia corpus:

RUSTFLAGS="-C target-cpu=native" cargo bench
FILE=... cargo bench

The second form replaces the file path to benchmark your own data, and the README points at the silesia corpus files and enwik8 as popular test files. A sample run log is committed at `benchmark.log` in the repository. The manifest backs this up, with a `[[bench]]` entry for density, snappy and lz4, each with `harness = false` because the benchmarks use divan rather than the built-in harness, and the dev dependencies pin divan, snap and lz4_flex.

The API is a struct with three methods

Using density from Rust is a matter of adding the dependency and calling methods on the algorithm struct. The dependency goes in your `Cargo.toml`:

[dependencies]
density-rs = "0.16.6"

And the round trip is this:

use std::fs::read;
use density_rs::algorithms::chameleon::chameleon::Chameleon;

fn main() {
    let file_mem = read("file_path").unwrap();

    let mut encoded_mem = vec![0_u8; Chameleon::safe_encode_buffer_size(file_mem.len())];
    let encoded_size = Chameleon::encode(&file_mem, &mut encoded_mem).unwrap();

    let mut decoded_mem = vec![0_u8; file_mem.len()];
    let decoded_size = Chameleon::decode(&encoded_mem[0..encoded_size], &mut decoded_mem).unwrap();
}

The three things to notice are the shape of the API, the ownership of the buffers and the method names. `Chameleon::safe_encode_buffer_size` asks the algorithm how large a buffer to allocate, which is how you avoid guessing at the worst case, and both `encode` and `decode` return a `Result`, so a buffer that is too small is an error rather than a memory overrun. The buffers are caller-owned, which means you can reuse allocations across many calls instead of allocating per message, and that is where most of the throughput advantage over a convenience API comes from.

Where the four-byte work unit costs you

The documentation is direct about the trade-off. Density's algorithms are general purpose, very fast algorithms, and they empirically exhibit strong performance on the pareto frontier of speed and ratio on voluminous datasets and text-based datasets, and they are less performant on very small datasets. The reason given is that the four-byte work unit approach makes them slower learners than Lempel-Ziv based algorithms, albeit being much faster theoretically. That is the whole limitation in one sentence. A Lempel-Ziv compressor finds a match, copies it, and then immediately recognises the repeated phrase, so a small payload of repetitive text compresses well almost at once. Density has to accumulate enough data to fill four-byte units and learn its dictionaries, so a short record pays the learning cost without getting the benefit. If your workload is many small records, a conventional LZ codec will beat it on both axes, and the ratio rankings in the table are measured on large corpora.

Licensing and the release line

The project ships two licence files, `LICENSE-MIT` and `LICENSE-APACHE`, and the Cargo manifest states the licence as `MIT OR Apache-2.0`, so you can take it under whichever suits your project. The package is `density-rs` at version 0.16.6 on edition 2024, authored by Guillaume Voirin, with the repository URL pointing at github.com/g1mv/density. The release profile is tuned for throughput, with `opt-level = 3`, fat LTO, a single codegen unit, no incremental compilation, overflow checks off and debug assertions off. That is the right configuration for a library you intend to ship in a hot path, and it also means the crate is compiled with the trade-off that arithmetic overflow is not checked, so any assumption in your own code about the sizes you pass in has to hold. The published releases are v0.16.6 on 2025-08-27, v0.16.5 on 2025-03-26 and v0.16.4 on 2025-03-24, and the repository has no GitHub releases beyond the crate versions.

Editorial conclusion

Adopt density if you are compressing large buffers in Rust and want speed at a comparable ratio, since the four-byte work unit is what buys it and the blowup protection keeps incompressible input from growing. Skip it if you compress small records, because the documentation says the algorithms are less performant on very small datasets and their four-byte approach makes them slower learners than Lempel-Ziv based algorithms. Before you commit, run `cargo bench` on your own data with `FILE=... cargo bench` rather than trusting the published comparison, and note that the crate is published as density-rs under a dual MIT or Apache-2.0 licence, with the 0.16 line being the current one.

Frequently asked questions

Official sources

1. g1mv/density on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases