Roaring bitmaps in Go, and an unusually honest README about when not to use them

A set of integers that compresses without giving up bitwise operators

The README builds the case from first principles, which is unusual and useful. Sets are fundamental, and in databases and search engines they show up constantly as the set of all documents or rows matching some property, represented by numeric identifiers. What you need from that set is not just membership but fast intersection, union and difference.

A plain bitset does this well. Using n bits you can represent any set of integers from the range zero to n, where bit i is set when integer i is present, and processors work in words of 32 or 64 bits. Intersections, unions and differences then become bitwise AND, OR and ANDNOT, which is exactly the argument for the format: when a bitset applies, it can be orders of magnitude faster than a hash set while using several times less memory.

The catch the README raises immediately is density. If you set a single bit at position 1,000,000 in an uncompressed bitset you spend over 100kB to store one bit, and an intersection with another bitset has to walk all those zeroes. Roaring's answer is to chunk the key space and pick a different representation per chunk, so a dense chunk is stored as a bitmap and a sparse one is stored as a sorted array of values, with the choice made per chunk rather than globally.

The data structure is not new and not Go-specific. The README notes there are Java and C/C++ versions, that the Java, C, C++ and Go implementations are binary compatible so a bitmap saved from a Java program loads in Go and vice versa, and that a separate format specification document defines the layout. Apache License 2.0 applies, with copyright running from 2016 onward.

Adoption is the strongest argument, and the README makes it first

The first substantive paragraph of the README is a list of who runs this format in production, and the names carry the weight. Roaring bitmaps are used by Apache Lucene and its derivatives Solr and Elasticsearch, by Apache Druid, LinkedIn Pinot, Netflix Atlas, Apache Spark, OpenSearchServer, anacrolix/torrent, Whoosh, Redpanda, Pilosa, Microsoft Visual Studio Team Services and eBay's Apache Kylin. It also names the YouTube SQL Engine, Google's Procella, as using them for indexing.

The Go-specific list is separate and just as telling: InfluxDB, Bleve, Weaviate, lindb, Elasticell, SourceGraph, M3, NetApp's trident, Datadog's Husky, Polarsignals' FrostDB and Disney Quanta. The repository description on GitHub names InfluxDB, Bleve and DataDog as the headline users, and the topics are the vocabulary of a database index: bitmap-compression, bitset, databases, indexing and roaring-bitmaps.

That is a reference implementation in the practical sense. When a format is already the on-disk interchange between Lucene, Druid and a Go service, interoperability is not a feature you are hoping for, it is the reason the Go port exists. It also sets expectations: this is infrastructure code that has to be correct and fast rather than pleasant to use, which is what the repository layout reflects.

Where the README tells you not to use it

Two sections of this README are about when to avoid the library, and they are more concrete than the section explaining when to adopt it.

The first is the sparse scenario. If you have a thousand random-looking integers, the README says a simple array is probably the best representation. The reasoning is arithmetic rather than empirical: for random data you cannot get much below 32 bits per integer, so attempts at compression can be counterproductive.

The second is small universes. If your bitmaps represent sets of integers from a range where n is 64 or 128, and an uncompressed bitset does not blow up your memory, then compressed bitmaps are probably not useful to you. If you do not need compression at all, the README makes a stronger claim than you might expect: a bitset offers remarkable speed.

There is a comparison section too, aimed at the run-length-encoded bitmap family that most alternatives belong to, formats that identify long runs of ones or zeroes and store them with a marker word. It describes Oracle's BBC as effectively obsolete because of excessive branching, WAH as a patented variation on BBC with better performance, Concise as a variation on the patented WAH that can compress up to twice as well in specific instances but is generally slower, and EWAH, which the material leaves unfinished.

The published guidance from that SIGMOD 2017 paper is quoted in the README as a single sentence, and it is the clearest statement of the whole design: use Roaring for bitmap compression whenever possible, and do not use other bitmap compression methods.

Go packaging details that decide whether it compiles on your machine

The module path carries a major version suffix, so the import path is not what someone might first guess:

module github.com/RoaringBitmap/roaring/v2

go 1.24.0

toolchain go1.24.4

That also means the package sits on a v2 line, and the release history in the repository matches a v2 series, moving through v2.26.0, v2.27.0, v2.28.0 and v2.29.0 during 2026.

The dependency list is short and tells you what the library actually needs. The direct requirements are bits-and-blooms/bitset at 1.24.4, google/uuid, mschoch/smat for streaming cardinality estimation, stretchr/testify for tests, and golang.org/x/sys. Everything else in `go.mod` is indirect, including the tooling that `unconvert` brings in.

There is a `Makefile` with a single real target, and it is a lint rather than a build step:

# Find unnecessary type conversions
unconvert:
	go tool unconvert -apply ./...

So the project registers `unconvert` as a Go tool and gives you a command to apply its fixes. Small thing, but it is the kind of detail that tells you how a contributor is expected to work in the tree.

Assembly for population count and bit filling, on three architectures

The repository tree is a map of the implementation. `arraycontainer.go` and `bitmapcontainer.go` are the two container representations, and the test files beside them make clear that behaviour is pinned per type. `fastaggregation.go` and `bitsetops.go` cover the set operations, `iter.go` and `manyiterator.go` cover iteration, and `parallel.go` covers the concurrent paths.

What stands out is the assembly. There is `popcnt_generic.go` alongside `popcnt_avx2_amd64.go` with its `.s` file, `popcnt_avx512_amd64.go`, and `popcnt_neon_arm64.go`. Population count is the inner loop of nearly every interesting Roaring operation, and SIMD implementations of it exist per architecture with a generic fallback. `fillbits_generic.go`, `fillbits_vbmi2_amd64.go` and `fillarray_vbmi2_amd64.go` do the same for setting ranges of bits, using AVX-512 VBMI2 where available.

There is also a `BitSliceIndexing/` directory, which is a separate line of work from the core bitmap, and the 2026 release notes show both areas receiving attention: one release finding min and max by bit plane rather than by row in BitSliceIndexing, another fixing sign-plane preservation when cloning BSIs, and another adding a portable 64-bit serialization API for the roaring64 variant.

A top-level `AGENTS.md` is present in the tree, along with `AUTHORS`, `CONTRIBUTORS` and both `LICENSE` and `LICENSE-2.0.txt`. GitHub reports 2943 stars, 262 forks, 82 open issues, and a last push on 2026-09-21 that matches the v2.29.0 release.

Editorial conclusion

Roaring is one of the rare libraries whose README argues against itself where it should. It explains that a plain array beats it on sparse random data, that an uncompressed bitset wins on small universes, and that a hash set is sometimes the honest answer. That candour matches the adoption list: this format is already inside Lucene, Solr, Elasticsearch, Druid, Pinot, Spark, Netflix Atlas and the YouTube SQL engine, so the interoperability argument is settled and your real questions are about Go specifics. Those are answered by the module path carrying a v2 suffix, the go 1.24.0 floor, and a repository that carries hand-written AVX-512, AVX2 and ARM NEON assembly for population count and bit-filling. Add roaringbitmap.org/roaring to a Go service, measure it against a map[int]struct{} on your own access pattern, and keep whichever wins.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. RoaringBitmap/roaring on GitHub