FSST compresses distinct-value strings and keeps them searchable

The target is data with many distinct values, where dictionaries fail

FSST is a compression scheme aimed at string and text data, and the README is specific about which kind. It is built for distributions with many different values, which is exactly the case where dictionary compression does not work well. A dictionary compressor wins when the data repeats; a column of unique identifiers, log lines, URLs or product codes has little internal repetition for a dictionary to exploit, yet still tends to draw its characters from a small alphabet.

The second requirement is random access. FSST is not block-based, so an individual string can be decompressed without touching the surrounding data in a compressed block. That is the property that separates it from the comparison point the README names, LZ4, which is block-based. The README states that compared with LZ4, FSST achieves similar decompression speed and compression speed, and a better compression ratio. Those are the project's own claims, and they are claims about a specific distribution rather than a general guarantee.

Together the two properties define the audience: engineers writing a database engine, a columnar file format, or a log store, who need to compress text and still read individual values out of it.

Symbols become single-byte codes, with 255 reserved as an escape

The encoding mechanism is a symbol table, but not the kind you would guess from the name. FSST maps symbols, which are byte sequences between 1 and 8 bytes long, onto codes, and a code is a single byte. A byte that has no code of its own can still appear: 255 is reserved, and it is followed by the original byte as an exception. Compression therefore turns a sequence of bytes into a sequence of codes and escaped bytes, and the result is meant to be shorter.

The interesting property of the output is that it is still a string. The compressed byte sequences fit back into whatever program manipulates strings in your system, and an optional 0-terminated mode exists for C-string style data. That matters more than it sounds, because it is what lets FSST slot into an existing system with few structural changes. The authors put it plainly: after all, FSST compressed strings still remain strings.

The design is the work of Peter Boncz at CWI, Viktor Leis at FSU Jena and Thomas Neumann at TU Munchen, and the reasoning behind it is written up in a PVLDB paper included in the repository as fsstcompression.pdf.

Equal strings stay equal after compression

This single guarantee is the reason FSST is usable where other compressors are not. FSST ensures that strings that are equal are also equal in their compressed form. Equality comparisons can therefore be performed without decompressing the strings.

Think about what that removes. Without the guarantee, a comparison engine has to decompress both sides of every comparison, or maintain a second compressed representation for comparison, or give up and compare a hash. With the guarantee, the compressed bytes can go straight into a hash table, a sort, or a comparison function, and the decode step happens only once for the value that survives.

It also means a compressed column can be compared against a compressed literal without constructing a temporary. The property is a consequence of how the symbol table is built, and it holds for strings of the same content regardless of how each was encoded. A compressor that could not promise this would still be a compressor, just not one that can be embedded in a query engine's data path.

Decompression can be pushed past hash tables and across the network

The database payoffs follow from the two properties above rather than from the compression ratio. The README describes FSST as allowing fine-grained decompression of values when selection predicates are pushed down into a scan operator, so a query that only needs to test a column can read and test compressed bytes.

More often, FSST allows decompression of string data to be postponed entirely. That shrinks hash tables in joins and aggregations, since the hashed representation is smaller, and it reduces network communication in distributed query processing. Nothing about the surrounding system has to change much, because the compressed form is still a string and the rest of the pipeline can treat it as one.

The practical consequence for an implementer is that the decoding step moves from per-value to per-surviving-value. A scan of a million rows that filters on one column decodes only the rows that pass. That is where the value of the scheme comes from in a query engine, and it is also why the comparison with a block compressor is not quite like the comparison with another block compressor: the two schemes are not trying to do the same job.

FSST12 widens the symbol table and pays for it in speed and memory

FSST12 is the alternative variant, and the trade is stated in detail. It uses 12-bit symbols and can therefore encode up to 4096 symbols of up to 8 bytes each, against 256 in the 8-bit version. It also needs no escaping mechanism, because the first 256 codes are single-byte symbols consisting of only that byte.

Those single-byte symbols guarantee that FSST12 can always find a symbol matching the next input, but the cost is not zero: a code is 1.5 bytes and those symbols are 1 byte, so there is still compression loss in that case. In FSST8 the penalty for an escape is heavier, at 2x compression loss.

The other costs are memory and time. FSST12 lookup tables are 16x bigger than for 8-bit FSST, roughly 8KB on average in storage and 32KB in memory, so a larger granularity of encoding volume is needed before the scheme pays off. Encoding is slower because of the memory pressure from 4096x4096 counters, and there is no AVX512 path for x86 in FSST12. To reach the same compression ratio, FSST12 needs symbols 1.5x longer on average than FSST.

What you get in exchange is reach. The larger table holds many more symbols, including less frequent and therefore longer ones that would not make the worthwhile cut in FSST8, so FSST12 handles distributions that are less focused than natural text. JSON and XML are the examples the README gives.

A small C++ core with SIMD variants in the same tree

The repository layout says a lot about the intended integration. There is a single header and source pair, fsst.h and fsst.cpp, and a separate implementation of the 12-bit variant in fsst12.h with libfsst12.cpp and libfsst12.hpp. A C++ wrapper with a C-compatible interface sits in libfsst.cpp and libfsst.hpp, and the x86 acceleration lives in fsst_avx512.cpp with several unroll variants, fsst_avx512_unroll1.inc through unroll4.inc, plus a makefile for Linux, Makefile.linux.

The build is CMake based, and the README states the implementation is quite portable and has been verified to work on 64-bits x86 computers running Linux, Windows and MacOS, the latter also using arm64. That portability claim is about the platform coverage of the project, and the separate arm64 mention suggests the Mac support path is not purely an x86 story.

The repository also carries the research trail alongside the code: the PVLDB paper, a paper directory, and a set of presentation files in mp4, pdf, png and pptx formats. For a project whose design decisions are the interesting part, that is a reasonable thing to have in the tree.

Where FSST is the wrong tool

The scheme has clear failure cases, and the README lets you infer most of them. If your data is already long and highly repetitive, a general purpose compressor will do better than a scheme that spends one byte per symbol occurrence. If your values are binary with a wide spread of byte values, there is little symbol structure to find and the escape path in FSST8 gets expensive, at 2x compression loss per escaped byte.

The other two constraints are about resources and about integration. FSST12 needs roughly 32KB of lookup table in memory per instance, and encoding it is slower, so a workload that decodes constantly and encodes rarely is a good fit, while the reverse is not. And the scheme has to be embedded: a consumer that needs the original bytes still needs the symbol table, and the table is specific to the data that was encoded. The README does not document a format for storing the symbol table alongside the data, so a self-contained file format is something an implementer has to design.

There is also the simple matter of the project itself. It has no GitHub releases, so there is no packaged artefact to depend on, only a source tree you vendor or build yourself. The last push was on 2026-04-06, the default branch is master, and the code is MIT licensed, so the terms for use and redistribution are permissive. The upstream reference is the PVLDB paper rather than a release changelog.

Editorial conclusion

FSST fits database systems and file formats where the same small set of characters repeats across millions of distinct values, where a block compressor such as LZ4 cannot help because values are read one at a time, and where a column must still be scannable. It does not fit data where values are already long and repetitive, where every byte differs, or where you need the highest compression ratio and can afford slower encoding, since FSST12 trades encoding speed and memory for its larger symbol space. Before you build on it, check three things: that your values really are text-like rather than binary, that the symbols your distribution produces fit the 256-code table of FSST8 or justify the 4096-symbol table of FSST12, and that your consumers can hold a symbol table per column, because decompression needs it. The code is MIT licensed, the repository has no GitHub releases, and the last push was on 2026-04-06.

Frequently asked questions

Official sources

1. cwida/fsst on GitHub
2. Issues
3. License: MIT
4. README