Jocko Replaces ZooKeeper with Serf and Raft but Has No Consumer Groups

The single binary goal moves ZooKeeper inside the process

The stated goals are narrow and concrete: implement Kafka in Go, stay protocol compatible so existing Kafka clients and services work against Jocko, make operating simpler, and distribute a single binary. The coordination decision follows from the last two. Instead of an external ZooKeeper ensemble, Jocko uses Serf for discovery and Raft for consensus, so the dependency list carries hashicorp/serf v0.8.5, hashicorp/raft v1.1.1 and hashicorp/memberlist v0.1.3, alongside hashicorp/consul v1.0.3 and hashicorp/raft-boltdb for persistence. What that buys is operational: one artifact to deploy, no separate coordination cluster to size. What it costs is failure domain. A broker now carries its own discovery and consensus state. The image reflects the extra surface, exposing 9092 through 9095 rather than the single client port a plain Kafka broker would need, and its default command is the broker itself:

EXPOSE 9092 9093 9094 9095
VOLUME "/tmp/jocko"
CMD ["jocko", "broker"]

The task list is the real feature matrix, and consumer groups are missing

The README keeps a checklist, and it is more informative than the goals list. Marked complete are producing, fetching, partition consensus and distribution, discovery, and, inside the protocol bucket, produce, fetch, metadata, create topics and delete topics. Three items are not complete. Consumer group is labelled as the current task. API versioning is noted as needing more API versions to implement. Replication has a first draft and is described as being tested heavily now. Read against the compatibility goal, the picture is precise: a client that produces, consumes by direct partition assignment, reads metadata and manages topics has every operation it needs, while anything built on group coordination does not. That gap is structural rather than a tuning issue. The same applies to replication: partition consensus is done, replication is not, so the three-broker topology below is a test rig rather than a replicated production shape.

Retention is a share of the disk, so it depends on the node

Two configuration goals go beyond parity with Kafka. The first is retention expressed as a percentage of disk space rather than only as bytes and time kept, which is a real operational improvement when disks differ in size, and a real coupling when they do not. The second goal is handling size configs when the number of partitions changes or topics are added, which is the part that bites on a live cluster: a size value computed for one partition count is not automatically correct after you add another, so the configuration has to be revisited at exactly the moment you are least inclined to revisit it. Neither goal comes with a documented migration procedure in the README, so treat both as features to validate on your own disk layout before you rely on them.

go get, dep and go.mod all appear in the same build

The build instructions belong to three different eras of Go tooling at once. The local path is written in GOPATH style:

$ go get github.com/travisjeffery/jocko
$ cd $GOPATH/src/github.com/travisjeffery/jocko
$ make build

The Makefile that make build invokes has a deps target that installs the pre-modules dependency manager and runs dep ensure -v before compiling cmd/jocko/main.go into cmd/jocko/jocko, which is why the README tells you to put $GOPATH/bin on your PATH if dep is not found. Meanwhile the repository ships a go.mod declaring module github.com/travisjeffery/jocko with go 1.12 and a long pinned require block, so a module-aware toolchain resolves dependencies a completely different way and never consults dep. The Dockerfile adds one more mismatch by building on golang:1.9-alpine, a version older than the one the module file declares.

The compose file is the only documented three broker topology

Cluster startup is documented by example rather than by prose, and the example is three services where exactly one bootstraps:

command: jocko broker --id 0 --raft-addr=jocko_a:9093 --bootstrap --bootstrap-expect=3
command: jocko broker --join=jocko_a:9094 --id 1 --raft-addr=jocko_b:9093 --bootstrap-expect=3
command: jocko broker --join=jocko_a:9094 --id 2 --raft-addr=jocko_c:9093 --bootstrap-expect=3

Two details decide whether this works on your machine. The join address points at port 9094 while the Raft address uses 9093, so discovery and consensus travel on separate ports and both have to be reachable between nodes. And the bootstrap flag appears only on the first broker, which is joined by the other two expecting a three node cluster, so bringing up two of the three services leaves the cluster waiting rather than forming with the members present. Each service also builds from the checkout and tags the result travisjeffery/jocko:latest, the same tag the standalone docker build command produces, so a local image and a compose image are indistinguishable by name. That is fine for experiments and worth avoiding once you need to know which build a broker is running.

The layout diagram names directories the root no longer has

The project layout in the README is a directory tree, and comparing it against the root of the repository shows how much has moved:

├── broker        broker subsystem
├── cmd           commands
│   └── jocko     command to run a Jocko broker and manage topics
├── commitlog     low-level commit log implementation
├── examples      examples running/using Jocko
│   ├── cluster   example booting up a 3-broker Jocko cluster
│   └── sarama    example producing/consuming with Sarama
├── protocol      golang implementation of Kafka's protocol
├── prometheus    wrapper around Prometheus' client lib to handle metrics
├── server        API subsystem
└── testutil      test utils
    └── mock      mocks of the various subsystems

The root holds cmd/, commitlog/, protocol/ and testutil/ as advertised, plus a top-level mock/ that the diagram places under testutil, but the broker, server and prometheus directories are not there, and jocko/ and log/ appear instead. The examples directory is present with an underscore in front of it, which is what stops the Go toolchain from treating example programs as packages, and it also means the example paths in the diagram do not resolve as written. For a reader trying to find where the broker logic lives, the diagram sends them to three directories that no longer exist, so the commitlog and protocol directories are the reliable starting points.

One release from 2017, a goreleaser config, and an author's stated hiatus

The packaging story is thin. The only published release is 0.0.1, dated 2017-11-15, while the last recorded push to the repository is 2026-05-20, so there is a long gap between the newest tag and the newest commit with no intermediate artifact. The tooling for releasing is wired up anyway: a .goreleaser.yml at the root, a Makefile release target that installs goreleaser and runs it, and a FUNDING.yml. That combination means the path to a versioned binary depends on tags that do not exist, and the practical route for anyone evaluating the code is the build instructions rather than a download. The README also carries a section titled Hiatus, in which the author says he is writing a book for PragProg called Building Distributed Services with Go and that it walks through building a distributed commit log from scratch, with the stated hope that it helps contributors and people interested in distributed services. Two long-form write-ups on Medium cover the built-in discovery and consensus design and the storage internals, which is where the architecture reasoning actually lives.

Editorial conclusion

Jocko is interesting to read and to fork if your goal is a single binary log with discovery and consensus inside it, and its commitlog, protocol and examples directories are the parts worth studying. It is not a drop-in Kafka replacement today, because the author's own task list leaves consumer groups unimplemented, API versioning incomplete and replication still under test. Before you plan around it, verify three things: which Kafka client features your application actually uses, since produce, fetch, metadata and topic administration are the only protocol operations marked done; whether your build environment can satisfy the Go tooling the instructions assume, given go get, dep and go.mod all appear in the same build; and where you would get a binary, since the only published release is 0.0.1 from 2017-11-15.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. travisjeffery/jocko on GitHub