# Jocko: a Kafka-compatible commit log in Go with Raft and Serf instead of ZooKeeper

> Jocko aims to implement Kafka's protocol in Go while replacing ZooKeeper with Serf discovery and Raft consensus, shipped as a single binary. The protocol work is unfinished, so it is worth reading before it is worth running.

**travisjeffery/jocko** — Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

- Repository: https://github.com/travisjeffery/jocko
- Website: https://twitter.com/travisjeffery
- Stars: 5,009 · Forks: 371
- Language: Go
- License: MIT
- Published: 2026-09-23 · Updated: 2026-09-23 · Language: en
- Canonical page: https://hysenlabs.com/projects/travisjeffery-jocko

## What Jocko replaces, and who the README is written for

The stated goals are explicit: implement Kafka in Go, stay protocol compatible so Kafka clients and services work with it, make operating simpler, and distribute a single binary. The operational argument is the interesting one. A normal Kafka deployment carries ZooKeeper alongside the brokers, which means a second cluster to configure, monitor and upgrade. Jocko's README says it uses Serf for discovery and Raft for consensus "and remove the need to run ZooKeeper". That is one process per node instead of two, and the Dockerfile reflects it: a single image, one binary at /usr/local/bin/jocko, ports 9092, 9093, 9094 and 9095 exposed.

The audience is narrow. This is for Go engineers who want a commit log they can read end to end, and for anyone evaluating whether a Kafka-compatible broker without ZooKeeper is viable. The README also points at a book the author was writing for PragProg, Building Distributed Services with Go, which tells you the project doubles as teaching material. That framing matters: the goal list ends with "Learn a lot and have fun", which is not the language of a product with an on-call rotation.

## Raft for consensus, Serf for discovery, and what the layout says about the split

The repository layout separates concerns cleanly. commitlog holds the low-level log implementation, protocol holds the Go implementation of Kafka's wire protocol, server is the API subsystem, broker is the broker subsystem, and prometheus wraps the Prometheus client library for metrics. cmd/jocko is the single command that runs a broker and manages topics, so the binary covers both roles.

The consensus and discovery layers show up in go.mod: hashicorp/raft and hashicorp/raft-boltdb for the Raft log store, hashicorp/serf and hashicorp/memberlist for membership, boltdb for local storage. Broker metadata that Kafka would push into ZooKeeper lives in the Raft group instead. That is a real architectural difference, not a packaging detail. It means the failure modes you inherit are Raft's: leader election, log replication to a quorum, and the requirement that a majority of brokers be reachable before the cluster can make decisions.

The docker-compose.yml makes the quorum concrete. Three brokers, each given an explicit --id and --raft-addr, with --bootstrap-expect=3. The first node bootstraps; the other two join through jocko_a:9094. Ports are split by function: 9092 for the Kafka client protocol, 9093 for Raft, 9094 for Serf gossip, 9095 also exposed in the image. Reading a three-line compose file tells you more about the design than the feature list does.

## Installing Jocko and starting a three-broker cluster

The README gives two build paths. The local one fetches the module and builds through the Makefile, which depends on dep for vendoring. If dep is missing, the Makefile's deps target installs it, and the README notes that an error about dep not being found means $GOPATH/bin is not on your PATH.

```bash
go get github.com/travisjeffery/jocko
cd $GOPATH/src/github.com/travisjeffery/jocko
make build
```

The build target writes the binary to cmd/jocko/jocko. The Docker path is one command, and produces the image the compose file expects.

```bash
docker build -t travisjeffery/jocko:latest .
```

For a first real use, the compose file is the shortest route to a running cluster. Each service runs the same image with a different broker command; the flags below are copied from docker-compose.yml, including the distinct Raft addresses and the shared bootstrap expectation.

```yaml
services:
  jocko_a:
    image: travisjeffery/jocko:latest
    command: jocko broker --id 0 --raft-addr=jocko_a:9093 --bootstrap --bootstrap-expect=3
  jocko_b:
    image: travisjeffery/jocko:latest
    command: jocko broker --join=jocko_a:9094 --id 1 --raft-addr=jocko_b:9093 --bootstrap-expect=3
```

Once the three containers are up, the Kafka client port is 9092. The repository ships an example under examples/sarama that produces and consumes with Sarama, and go.mod pins github.com/Shopify/sarama v1.13.0 for it. That example is the honest test: if a stock Sarama producer and consumer work against the cluster, the protocol compatibility claim holds for those request types. If they do not, the README's TODO list tells you why.

## The TODO list is the real specification

The README's TODO section is the most useful part of the document. Producing, fetching, partition consensus and distribution, discovery, metadata, create topics and delete topics are checked off. Consumer group is marked as the current task and unchecked. API versioning is unchecked, with the note that more API versions remain to implement. Replication is unchecked, described as "first draft done - testing heavily now".

That is a clear boundary. Any workload that relies on consumer groups, which is to say most Kafka usage outside of simple produce and fetch, is outside what the README claims. Replication being a first draft means the durability story is not settled either: a broker that accepts writes without tested replication to peers is not the same guarantee a Kafka operator expects. The release history reinforces the point. The only release listed is 0.0.1 from 2017-11-15, so there is no versioned artifact signalling a stable protocol surface.

None of this is hidden. The README states it plainly, which is more than many projects do. But it means Jocko should be evaluated as an implementation to study, not as a drop-in broker. The last push to the repository was on 2026-05-20, so the code has moved since the 0.0.1 release even though the TODO list has not been reconciled in the README.

## Configuration goals that go beyond Kafka's defaults

Under the goals list, Jocko names two configuration behaviours it wants that Kafka does not offer in the same form. The first is retention by percentage of disk space, rather than only bytes and time. For an operator running on a fixed volume, that is a meaningful difference: you express the policy in terms of the resource you actually run out of, instead of translating disk size into a byte count by hand and updating it when the volume changes.

The second is handling size configs when you change the number of partitions or add topics. This is a real operational annoyance in Kafka deployments, where per-topic settings have to be revisited as topology changes. Both items are listed as goals, not as finished features, and the README does not document the config keys that implement them. Treat them as design intent. If you need either behaviour today, the README does not tell you how to enable it, and there is no configuration reference in the repository's top-level files to check against.

## Where Jocko is the wrong tool, and what to use instead

If you need a Kafka broker in production this week, Jocko is the wrong tool. The README marks consumer groups and replication as unfinished, and the only release is 0.0.1. A team that depends on consumer group rebalancing, offset commits through the group coordinator, or replicated partitions will not find those guarantees documented here.

The obvious alternative is Apache Kafka itself, and the difference is not just maturity. Kafka delegates cluster metadata, controller election and membership to ZooKeeper, which is a separate system with its own operational surface. Jocko collapses that into Raft and Serf inside the broker process, so a Jocko cluster is three processes rather than three brokers plus a ZooKeeper ensemble. The trade is that Jocko inherits Raft's quorum requirements for metadata operations and Serf's gossip for membership, and it gives up the years of protocol coverage Kafka has accumulated. If you want a Kafka-compatible broker without ZooKeeper and cannot wait, the honest answer is to run Kafka with KRaft, which is not discussed in this repository. If you want to understand how a commit log and its coordination layer fit together in Go, Jocko's commitlog, protocol and broker packages are the reason to look.

## Licence, maintenance and what an upgrade actually costs

Jocko is MIT licensed, per the LICENSE file and the README's licence section. MIT is permissive: you can fork, modify and redistribute, including in closed products, provided the copyright notice and permission notice are preserved. That matters here because the realistic use of this repository is as a starting point for your own broker rather than as a dependency you track. This is not legal advice; read the LICENSE file before you redistribute anything.

On maintenance, the last push was on 2026-05-20, and the only release in the list is 0.0.1 from 2017-11-15. There is no changelog in the top-level entries, so an upgrade path between versions is not documented. The Makefile does define a release target that shells out to goreleaser, and .goreleaser.yml is present, so releases are mechanised even though the release list is thin. Practically, adopting Jocko means tracking master and reading the TODO list yourself, because the README is the closest thing to a status document and it has not been reconciled with the current state of the code.

## Conclusion

Jocko is worth adopting only as a codebase to read or fork, since the README marks consumer groups and replication as unfinished and the release list stops at 0.0.1. Do not put it in front of production traffic that depends on Kafka consumer groups or replication. Before anything else, check whether the master branch has moved past the README's TODO list, then run the docker-compose.yml three-broker cluster and point a Kafka client at port 9092 to see which protocol requests are actually answered.

## FAQ

### What is Jocko?

Jocko is a Kafka-compatible distributed commit log service written in Go, per the README's own description. Its goals are to implement Kafka in Go, stay protocol compatible with Kafka clients, and distribute a single binary.

### How do you install Jocko?

The README gives two paths: clone the repository and run make build after fetching it with go get, or build the Docker image with docker build -t travisjeffery/jocko:latest . The Makefile's build target writes the binary to cmd/jocko/jocko.

### Does Jocko need ZooKeeper?

No. The README lists using Serf for discovery and Raft for consensus as a goal specifically to remove the need to run ZooKeeper, and go.mod carries hashicorp/raft, hashicorp/raft-boltdb and hashicorp/serf as dependencies.

### Which Kafka protocol features does Jocko implement?

The README's TODO list marks Produce, Fetch, Metadata, Create Topics and Delete Topics as done. Consumer group is unchecked and described as the current task, API versioning is unchecked, and replication is described as a first draft being tested.

### What ports does the Jocko broker use?

The Dockerfile exposes 9092, 9093, 9094 and 9095. In docker-compose.yml, brokers pass --raft-addr on 9093 and join each other through Serf on 9094, while 9092 is the Kafka client protocol port.

### What licence is Jocko under?

Jocko is under the MIT licence, stated in the README's licence section and in the LICENSE file in the repository root.

## Sources

- [License: MIT](https://github.com/travisjeffery/jocko/blob/master/LICENSE)
- [Project website](https://twitter.com/travisjeffery)
- [README](https://github.com/travisjeffery/jocko/blob/master/README.md)
- [Releases](https://github.com/travisjeffery/jocko/releases)
- [travisjeffery/jocko on GitHub](https://github.com/travisjeffery/jocko)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/travisjeffery-jocko