bitarray: eight booleans per byte, with the machine representation exposed

A list of booleans at one eighth the memory

The library provides an object type which efficiently represents an array of booleans. Bitarray objects are sequence types and behave very much like usual lists, but eight bits are represented by one byte in a contiguous block of memory and all functionality is implemented in C. The design consequence is that a bitarray is homogeneous in a way a list of Python bools is not, since each element is one bit rather than an object reference plus a bool. The API is deliberately list-like: slicing including slice assignment and deletion, the + and * operations and their in-place variants, the in operator and len. On top of that come the full bitwise set, ~ & | ^ and the shifts, along with their in-place forms &= |= ^= <<= >>=. For a workload holding millions of flags or membership bits, the difference between eight bits per element and a pointer plus a bool object is the difference between a container that fits in a cache and one that does not.

Bit-endianness is per object and it will bite you

The user can select between two representations, little-endian and big-endian, and this is the single most consequential detail in the library. The default bit-endianness reported by the test output in the README is big, while the machine byte-order of the same machine is little, which shows the two are not the same thing. The README is clear that the bit-endianness is of importance when accessing the machine representation. If you are exporting a bitarray to bytes, or mapping one onto a file, the order in which bits are laid out within a byte determines what the data means. The upside of the choice is that a bitarray can be built to match an external format, such as a C structure or a network protocol, and then read without any per-byte conversion. The downside is that mixing two bitarrays with different endianness silently produces wrong data rather than an error, so the setting has to travel with the data.

List semantics, with two places where it differs

The usage section states that bitarray objects behave very much like lists, so there is not too much to learn, and it names the differences. Besides the obvious homogeneity, indexing a single item always returns an integer, whereas indexing a slice always returns a bitarray, even when the slice length is just one. That is a small rule that removes a class of bug, because a slice cannot accidentally be a list of mixed types. A second difference is that slices can be assigned booleans directly, which the documentation says is easier and faster than assigning a bitarray in which all values are the same, since you avoid having to create a temporary bitarray whose length must be known or calculated. The example is stark: a[1:15:3] = True against a[1:15:3] = 5 * bitarray('1'), which produce the same result but where the second form needs a temporary you have to size yourself.

The buffer protocol is what makes it interop

The library's methods for accessing the machine representation include the ability to import and export buffers, which is what allows creating bitarrays that are mapped to other objects, including memory-mapped files. Bitarray objects support the buffer protocol, both importing and exporting buffers, and there is efficient packing and unpacking for interoperability with byte-oriented objects such as NumPy arrays. The examples directory shows how far this goes, with files for mmapped-file.py, a Bloom filter in bloom.py, Hamming distance in hamming.py, a sieve in sieve.py, UTF-8 and UTF-16 handling, an IPv4 address space in ipv4.py, a Mandelbrot renderer in mandel.py, and both a Huffman and a deflate implementation in the huffman/ and puff/ directories. The practical test for whether you want this library is whether your boolean data already has a byte-oriented home. If it does, the buffer protocol makes bitarray the right intermediary rather than an extra copy.

A utility module that covers the awkward conversions

bitarray.util exists because converting between representations is the tedious part of working with bits. It handles conversion to and from hexadecimal strings, generating random bitarrays, pretty printing, conversion to and from integers, and creating Huffman codes. It also covers compression of sparse bitarrays, run-length encoding, serialization and deserialization, various count functions and other helpers. There is a whole sub-module for bitfields, packing and unpacking fixed-width bit-field structures, which is a different use case from the sequence type: it is for laying out C-style packed structs in a byte buffer. The library also documents fast methods for encoding and decoding variable-length prefix codes, and separate pages for the free-threading support for GIL-disabled CPython 3.14 and later, the platforms supported by CPython and PyPy, and the buffer and ndarray interoperability.

Installing is a wheel, and the source build is the fallback

Python wheels are available on PyPI for all major platforms and Python versions, so the install is the ordinary one:

python -m pip install bitarray

The project then suggests testing the installation, and the test() function is part of the API, returning a unittest.runner.TextTestResult object, so you can verify that all tests ran successfully with a plain assertion:

import bitarray
assert bitarray.test().wasSuccessful()

The output of that call in the README shows the environment it ran in, the bitarray version, the platform, the Python version, whether the GIL is enabled, the pointer size, whether a builtin 64-bit byte swap is available and the default bit-endianness, followed by the test run itself with 723 tests and two skipped. For a C extension, the environment lines are the interesting part, because a bitarray built for one configuration is not interchangeable with another. Building from source is handled by the Makefile, where the bitarray/_bitarray.so target runs setup.py build_ext --inplace, the test target runs setup.py test, and the install target runs pip install. The setup.py also reads the version straight out of bitarray/bitarray.h with a regular expression matching BITARRAY_VERSION, so the version has one source of truth.

Editorial conclusion

Adopt bitarray when you are holding large boolean sets in Python, such as adjacency structures, feature flags or bitmap filters, and the memory saving matters. Do not adopt it for small collections, where a list of bools or a Python set is simpler and the C extension is build overhead for nothing, and be careful with the bit-endianness, because it is set per object and has to be tracked when you exchange data with other systems. Before you commit, install it from PyPI, run the bitarray.test() self-check that the project ships as part of its API, and confirm your build environment can compile the extension if no wheel matches your platform.

Frequently asked questions

Official sources

1. ilanschnell/bitarray on GitHub
2. Issues
3. README