c-blosc2: a compressed, persistent data store with an open format

A compressor that is also a data format

Blosc2 is described as a high-performance compressor and data format optimised for binary data, including numerical arrays, tensors and other structured formats. What makes it more than a codec is the framework around it: a flexible set of codecs and filters, so a developer can trade compression speed against compression ratio for a particular use case, from high-throughput data pipelines to persistent storage.

It is the successor to the original Blosc library, which was released in 2010, and the project describes the new version as built on a mature foundation. That maturity shows in the adoption: Blosc2 is integrated into widely used scientific computing libraries including PyTables, h5py and Zarr, which means the format is already load-bearing in other people's stacks.

The C-Blosc2 library is the latest major version of C-Blosc, and it is backward compatible with both the C-Blosc1 API and its in-memory format. There is a Python wrapper too, officially supported thanks to the Python Software Foundation, which carries the features of the older python-blosc package plus most of what is new in C-Blosc2, including 64-bit and multidimensional containers.

The C library is also available in binary form through that wrapper, which is the easiest route if you are not writing C but still want the library on a machine.

The compatibility only runs one way

This is the constraint to understand before you store anything. C-Blosc2 is backward compatible with the C-Blosc1 API and with its in-memory format, so an application written against the old library can generally keep working. The reverse is not true: buffers generated with C-Blosc2 are not format compatible with C-Blosc1, so forward compatibility is not supported.

That is a normal trade for a format that adds capabilities, and it is the kind of thing that bites when a downgrade is needed. If you want full API compatibility with the C-Blosc1 API, the project notes say to define the BLOSC1_COMPAT symbol, which is the switch that keeps the older surface available.

The broader point is about persistence. A compressed buffer written by this library is a data file, and its readability is governed by the format, not by the library version you happen to have installed. That is why the format documentation, rather than the API documentation, is the thing to keep.

There is also a file name extension document in the repository, alongside separate documents for the chunk format, the frame format, the superframe format, the metalayer and the threaded configuration, so the on-disk conventions are written down piece by piece rather than in one opaque file.

B2ND, and why a pineapple is the right shape

The feature that distinguishes Blosc2 from a general purpose compressor is the Blosc2 NDim layer, called B2ND, which allows creating and reading n-dimensional datasets efficiently through an n-dimensional two-level partitioning. That partition is what makes it possible to slice and dice arbitrarily large compressed data in a fine-grained way, and the project has a blog post and a short explanatory video about it, with the slicing described as pineapple-style.

The intuition is worth having. A flat array compressed as one long run of bytes can only be read as that run. A two-level structure puts chunks inside larger blocks, so reading a slice touches the blocks that intersect it and skips the rest, in a way that generalises across every axis of a multidimensional array. For a dataset where most reads are slices, that is the difference between reading a megabyte and reading a terabyte.

The Python wrapper is where this is easiest to see, and the project shows the performance of reading slices orthogonal to different axes of a four-dimensional dataset there. And because the wrapper ships wheels and binary versions of the C libraries, you can install the whole stack with a single command:

  pip install blosc2

The repository also ships examples for the NDim layer, including reading and writing slices, so the C API for the same operations is not hidden.

The format spec is under 4000 words

The project makes a point of its format documentation, and the argument is about long-term survival rather than about conformance testing. The Blosc2 format is open and fully documented, and the format specs are defined in less than 4000 words, so they should be easy to read and understand. The project calls that critical for the long-term success of the library, because it allows third-party implementations of the format, and it allows users to understand what is going on under the hood.

That is an unusual position for a performance library to take, and it is a coherent one. A compression format that only its own library can read is a vendor lock-in with a performance benefit; a format described in a few thousand words is a format that other people can implement, and then the competition is on the implementation rather than on the access.

The repository reflects that attitude. There are separate documents for the chunk format, the C frame format, the superframe format, the B2ND format, the metalayer, the extension file names, the threaded configuration, plus a fuzzer note, a development guide, a release process document, a changelog and a what is new file, and the source is organised into a library directory, a compatibility directory, a plugins directory, an include directory, tests, benchmarks and examples.

Building, testing and installing with CMake

The build is a standard CMake flow, and the project describes it step by step. You clone the repository, create a build directory inside the sources and move into it:

  git clone https://github.com/Blosc/c-blosc2
  cd c-blosc2
  mkdir build
  cd build

Then you run the configuration, optionally specifying where the library should be installed:

  cmake -DCMAKE_INSTALL_PREFIX=your_install_prefix_directory ..

Configuration can also be done with the graphical and curses interfaces that CMake provides, which is worth knowing because the codec options are numerous. The build, test and install steps are:

  cmake --build .
  ctest
  cmake --build . --target install

The static and dynamic versions of the library, together with the header files, are installed into the prefix you chose, and the repository carries a package configuration file template and a pkg-config template, so a dependent build can find the library the usual way. There is also a Guix channel and a Guix package definition in the repository, and the project publishes a guide on compiling with wheels for the Python side.

Codec dependencies: bundled by default, external on request

C-Blosc2 can either fetch pinned upstream codec releases through CMake FetchContent or use codec libraries already installed on the system, and the top-level switch is BLOSC_DEPENDENCY_MODE with two documented values.

BUNDLED is the default. It fetches and builds the bundled dependency versions, which preserves the traditional self-contained C-Blosc2 build behaviour. The important property is that bundled dependencies are private: their headers and libraries are not installed separately, and static builds fold them into libblosc2.a. That is what makes the library easy to ship, because the only thing a consumer has to link is the Blosc2 library itself.

EXTERNAL requires the codec libraries to be present on the system, which is the mode to use when you want to link against the system versions of a compression library, or when your distribution policy requires it. The codec set covered is LZ4, LZ4HC, Zstd, Zlib and ZFP.

The two modes are a genuine fork in the road rather than a detail, because they determine whether your build is reproducible on a machine with no compression libraries installed, and whether upgrading a system codec can change your output.

Frequent releases, and two roadmaps

The project moves quickly. The recent releases include v3.3.3 on 2026-08-31, v3.3.4 on 2026-09-10 and v3.3.5 on 2026-09-24, and the last push to the repository was on 2026-10-22. So a new patch version roughly every two weeks, which for a library that sits underneath scientific tooling is a sensible rate, and it means the changelog and the what is new file are worth reading when you upgrade.

The direction of travel is written down too, in separate roadmap documents for the 3.0 and 4.0 versions, alongside a refactorisation task list, a release notes file, a releasing guide, a funding configuration and a thanks file. Having a roadmap for the next major version is a good sign, because it means the format changes that a major version implies can be anticipated rather than discovered.

The project sits under the Blosc development team, is part of NumFOCUS, follows a contributor covenant, and is built and tested in continuous integration with a CMake workflow. The licence file and a directory of third-party licences are in the repository root, which is what you would expect from a library that bundles codec sources and ships plugins.

Editorial conclusion

c-blosc2 suits a scientific computing project that stores large numerical arrays and wants compressed storage with cheap partial reads, because the two-level partition is what makes a slice of a huge array cost a fraction of the whole, and the open format means you are not locked into the library to read your own data. It is the wrong choice if you need a stable on-disk format that older software can still read, since the compatibility is one-directional and buffers written by this version are not readable by the previous one. Before you commit to it, read the format specification in the repository, since the project is explicit that reading it is the point, and decide the dependency mode deliberately, because the default bundled mode is convenient and the external mode hands codec selection to your system. The releases are frequent, v3.3.5 on 2026-09-24, and the last push was on 2026-10-22.

Frequently asked questions

Official sources

1. Blosc/c-blosc2 on GitHub
2. Issues
3. Project website
4. README
5. Releases