Comdb2: Bloomberg's clustered RDBMS with optimistic concurrency control

Optimistic concurrency control, not a lock manager

The first line of the overview is the design statement: Comdb2 is a clustered RDBMS built on optimistic concurrency control techniques. Read and write transactions run on any node. Isolation is offered in multiple levels, and the two named ones are Snapshot and Serializable Isolation. That pairing is unusual in an open source clustered database and it is the reason an architect looks at this project rather than a mainstream engine. Work started at Bloomberg LP in 2004 and the README says it has been under heavy development since, with the architecture described in a VLDB 2016 paper and usage documentation hosted at bloomberg.github.io/comdb2.

The client library chooses the node, and reconnects when it fails

Two client library behaviours do real work. First, connections are negotiated to the lowest cost node available, where cost is latency, so an application is written against one endpoint and the library decides where each transaction lands. Second, the library reconnects transparently. For a cluster whose nodes go away, that is the difference between a failover the application handles and a failover the application never notices. The codebase backs this up: cdb2api/ is the client code, net/ and sockpool/ hold the network and socket pooling, and protobuf/ is described as the API to communicate with the server. There is also cdb2jdbc/ for the JDBC driver.

Building is cmake into /opt/bb, and prerequisites are long lists

The build is a conventional cmake project with a root CMakeLists.txt, and the quick start installs it system wide:

mkdir build && cd build && cmake .. && make && sudo make install

Afterwards the binaries live under /opt/bb/bin, and the quick start adds them to the path:

export PATH=$PATH:/opt/bb/bin

The prerequisites are the part that tells you what kind of machine this expects. On Debian based systems the README lists bison, build-essential, cmake, flex, libevent-dev, liblz4-dev, libprotobuf-c-dev, libreadline-dev, libsqlite3-dev, libssl-dev, libunwind-dev, ncurses-dev, protobuf-c-compiler, tcl, uuid-dev and zlib1g-dev, and the RPM list is longer still with rpm-build and openssl-devel among them. macOS is marked experimental and takes a shorter route:

brew install cmake lz4 openssl protobuf-c readline libevent

with coreutils, bash, gawk and jq for the tests, and a recommendation to raise open file limits in the sessions that start pmux and the server.

pmux is what actually runs the database

Once the path is set, the first command is the process multiplexer:

pmux -n

The database is then started against a configuration file, and this is the step that surprises people coming from a service manager world:

comdb2 --lrl ~/db/testdb.lrl testdb

When a node is ready it prints I AM READY. Databases are created with comdb2 --create, and the quick start deliberately creates one in the home directory to keep the steps short, since the default root is /opt/bb/var/cdb2/ and the README notes that root has to exist with the right permissions before you invoke --create. Reading those two paths side by side is the whole mental model: one place holds the data files, one holds the logs, and both default to system directories.

Cluster membership is a line in an lrl file you copy by hand

Nodes identify each other by hostname, which is the first thing to get right. If a node's hostname does not resolve from the others, you write the fully qualified name into a per node file:

vi /opt/bb/etc/cdb2/config/comdb2.d/hostname.lrl

The cluster membership itself is a directive inside the database's configuration file:

cluster nodes machine-1.comdb2.example.com machine-2.comdb2.example.com

and the file is pushed to the other machines with a dedicated tool:

copycomdb2 mptest-1.comdb2.example.com:${HOME}/db/testdb.lrl

That is the whole orchestration story in the quick start. There is no discovery protocol to configure and no membership service: you edit a text file and copy it. Fine for a cluster whose membership changes rarely, tedious for one where nodes come and go often.

The error you will hit is a permission denied on the log directory

The README calls out the failure that the quick start produces for most people and then tells you to ignore it. The default log directory is /opt/bb/var/log/cdb2/, and if your user cannot create files there you get:

[ERROR] error opening '/opt/bb/var/log/cdb2/testdb.longreqs' for logging: 13 Permission denied

The README says this condition will not impact operation of the database for the purposes of the quick start. True, but it means the long running request logging that a production cluster wants is silently absent while you evaluate the system, and discovering that after load is the bad way to find out. Create the directory with the right ownership before you start, or treat that first ERROR as the first configuration step rather than a footnote.

A layered C codebase with a Lua stored procedure VM

The directory table in the README is the clearest map of the system. bdb/ is the table layer and berkdb/ the btrees layer, db/ holds the types layer and the overall glue, and sqlite/ is the embedded SQLite VM acting as the SQL engine. Language support is unusually broad for a C database: lua/ is described as everything pertaining to the Lua VM used for stored procedures, and tcl/ holds Tcl language bindings, with cson/ as the JSON library and crc32c/ for checksums. Memory is a feature rather than an accident, with mem/ as a memory accounting subsystem and dlmalloc/ a local malloc version. Rounding it out are schemachange/ for table create, alter and truncate, plugin/ and plugins/ for the plugin subsystem, and pkg/ for deb and rpm packaging rules. If you are evaluating whether to carry this code, that table is the staffing estimate.

Against a mainstream clustered RDBMS

The alternative is a well known relational database with its own clustering story, and the difference in approach shows up in three places. First, access model: here a client connects to any node and the library routes by latency, with no connection pooling tier in front of it. Second, isolation: optimistic concurrency control with snapshot and serializable levels is the design centre, not a configuration option added later. Third, engineering burden: this is a C codebase you build, install to /opt/bb and configure through lrl files, with no tagged releases at all. What you give up is documentation written for your category of user, a managed upgrade path, and the community support network that comes with them. The trade is only worth it if the isolation model or the node routing is something you actually need, not if you merely want a database.

Editorial conclusion

Comdb2 fits an organisation that wants a clustered relational database where clients connect to any node and the system routes them, and where snapshot or serializable isolation is a requirement rather than a hope, since those isolation levels and the client library's node negotiation are the reason the project exists. Skip it if you want a container image and a managed upgrade path: this is a C codebase built with cmake into /opt/bb, run through pmux, with configuration living in.lrl files you copy between nodes by hand. Before you start, confirm every machine in the cluster can ssh to every other, because step one of the quick start says so, and decide whether you will carry the code yourself, because the last push was on 2026-10-16 and the repository has no GitHub releases. The architecture paper is the VLDB 2016 one the README links, and it is a better use of an hour than the quick start.

Frequently asked questions

Official sources

1. bloomberg/comdb2 on GitHub
2. Issues
3. README