# ruby-kafka: a pure Ruby Kafka client now superseded by rdkafka-ruby

> ruby-kafka was a Ruby client for Apache Kafka written with no C dependency, focused on operational simplicity, good logging and metrics. The project itself states it is no longer actively developed and recommends rdkafka-ruby, which binds to librdkafka, instead.

**zendesk/ruby-kafka** — A Ruby client library for Apache Kafka

- Repository: https://github.com/zendesk/ruby-kafka
- Website: http://www.rubydoc.info/gems/ruby-kafka
- Stars: 1,284 · Forks: 332
- Language: Ruby
- License: Apache-2.0
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/zendesk-ruby-kafka

## The deprecation notice is the first thing in the README

The repository opens with a deprecation notice, and any review that skips it will get the story wrong. The library is no longer actively developed and has been superseded by librdkafka via the rdkafka-ruby bindings. The notice says the library may still receive security patches and bug fixes, but that it is no longer recommended for production usage.

The stated reason is about the shape of the work rather than about the maintainer's availability. The project argues there needs to be a concerted effort to keep up with Kafka features, and that there is no point trying to keep up with Kafka development when other languages use a well-established C binding and get more official support.

That is a specific technical argument. Kafka's client protocol surface has grown, and a pure Ruby implementation has to reimplement every feature in an interpreted language, while a binding inherits the behaviour of the C library that the broker project itself uses. The repository topic list includes deprecated, which is the same signal in machine readable form.

The dates matter for a decision. The last push was on 2026-10-15, so the tree is still receiving commits, but the newest tagged release is v1.4.0, and v1.3.0 and v1.1.0 were all tagged on 2021-12-27. Commits and releases are not the same thing here, and the release line is the one that reaches users through RubyGems.

## Operational simplicity was the design goal, not raw speed

ruby-kafka was a client for Apache Kafka, described as a distributed log and message bus, with the stated focus on operational simplicity, with good logging and metrics that make debugging issues easier.

That is a narrower and more honest pitch than being the fastest Ruby client. The documentation is organised around the things that go wrong in production: buffering and error handling, message durability, delivery guarantees, consumer checkpointing, shutting down a consumer, and understanding timeouts. There is a whole section on monitoring, with subsections on what to monitor and on reporting metrics to Statsd and to Datadog, plus a separate instrumentation section.

This matters when reading the library today. Even if you move to rdkafka-ruby, the mental model this documentation builds, about a producer that buffers and retries, a consumer that checkpoints offsets and shuts down cleanly, and a client that is expected to be monitored rather than assumed, is the model you carry to the replacement.

The design section is unusually explicit about the internals, with separate subsections for producer design, asynchronous producer design and consumer design. That is the part of the library that cannot be lifted into a C binding, and it is why the documentation remains the durable asset here.

## Installing the gem and running against a local cluster

Installation is a Gemfile entry followed by a bundle, or a direct gem install.

```ruby
gem 'ruby-kafka'
```

```bash
bundle
```

```bash
gem install ruby-kafka
```

For local development the repository ships a docker-compose.yml that stands up a ZooKeeper and three Kafka brokers from the confluentinc images at version 7.2.3. The brokers advertise separate listeners, with kafka1 reachable on localhost:9092, kafka2 on localhost:9093 and kafka3 on localhost:9094, and an internal listener on port 19092 for inter-broker traffic. The environment sets the replication factors for the offsets topic and the transaction state log, and enables topic deletion.

That three-broker layout is not decoration. Consumer group rebalancing, partition assignment and replication behaviour cannot be observed on a single broker, so the compose file exists to make those code paths testable locally.

The examples directory shows the range of use cases the library was built for: simple-consumer.rb, simple-producer.rb, consumer-group.rb, firehose-consumer.rb, firehose-producer.rb and ssl-producer.rb. The firehose examples are for consuming from the beginning of a topic rather than from a checkpoint, and the ssl one covers encrypted producer connections.

The Rakefile, the .rspec file and the .rubocop.yml file in the repository root show a conventional Ruby gem layout with RSpec for tests and RuboCop for style, and there is a benchmarks/ directory for the performance work the library's focus on operational behaviour was balanced against.

## The compatibility table stops at Kafka 2.7 and admits its limits

The README carries a Kafka compatibility table that is worth reading carefully because it is honest about the gaps.

Full producer and consumer support exists for Kafka 0.9 and 0.10. For Kafka 0.10 the documentation notes that you must run version 0.10.1 or higher because of limitations in 0.10.0. Kafka 0.11 gets full producer support with limited consumer support in the v0.7.x series, and the missing feature is called out specifically as dirty reading of the Consumer API. Kafka 0.11 also introduced the record batch format along with idempotent and transactional production.

From Kafka 1.0 onward the table records limited support, and the prose repeats the pattern with a formula: everything that works with the previous version should still work, but no features specific to the new version have been added. That line runs for 1.0, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6 and 2.7.

So the library functioned against the Kafka protocol of its era and tracked the broker's core producer and consumer paths, but did not add new broker features as they appeared. That is precisely the maintenance burden the deprecation notice describes, and the table is the evidence for it rather than an assertion.

The library targets Kafka 0.9 with the v0.4.x series and Kafka 0.10 with the v0.5.x series, with limited support for Kafka 0.8. Anyone reading this today against a current broker should assume that behaviours added after 2.7, whatever they are, are outside what this client was ever tested for.

## Security and monitoring are the documented surface area

Two documentation areas deserve particular attention for anyone evaluating a Kafka client for an existing Ruby stack.

Security is covered in its own section with two subsections, encryption and authentication using SSL, and authentication using SASL. The presence of the ssl-producer.rb example in the examples directory confirms that the TLS path is exercised in code rather than only described.

Monitoring is covered in two separate places, instrumentation and monitoring, which is a distinction worth keeping. Instrumentation is the mechanism the library exposes to emit its own measurements. Monitoring is the guidance on what to watch, plus the two concrete integration paths for shipping those measurements out, to Statsd and to Datadog.

For a distributed log, the failure modes are quiet. A producer that silently buffers, a consumer that rebalances and restarts, a partition that moves between brokers: none of these raise an exception in application code, and all of them show up as lag or as a delivery guarantee that quietly weakens. A client that gives you the numbers is not a convenience feature here, it is the difference between detecting a stalled pipeline and discovering it from a user.

The consumer API sections cover the mechanics that follow from that: consumer groups, consumer checkpointing, topic subscriptions, shutting down a consumer, consuming messages in batches, balancing throughput and latency, and customizing the partition assignment strategy. The last two are the ones that separate a client used in production from one used in a demo.

## Ruby Kafka clients split along a C binding line

The choice facing a Ruby team is not between many equal options but along one axis: whether the Kafka protocol is implemented in Ruby or delegated to the C library that the broker project itself uses.

ruby-kafka took the pure Ruby path, which removes the native build step and the librdkafka runtime dependency, and buys you a codebase you can read, step through in a debugger and reason about without a C toolchain in the build image. That was the trade it was making, and the deprecation notice is essentially the argument that the cost of reimplementing Kafka features in Ruby eventually outweighed it.

rdkafka-ruby takes the other path, binding to librdkafka through the AppSignal-maintained bindings. The trade is the reverse: you accept a native extension and the need to compile or ship a binary, and in exchange you inherit the behaviour of the C client, which is where Kafka feature work lands first and where the other language clients get their support from.

The README also points to higher level libraries for teams that do not want to handle brokers directly: message processing frameworks and message publishing libraries. The names are not given in the visible documentation, so it is worth reading that section rather than assuming which ones are meant.

For a team already running Kafka with librdkafka behind another service, the migration from ruby-kafka is mostly a change of API shape. For a team that valued pure Ruby for deployment reasons, the practical move is to check whether a native build actually blocks them, because that assumption is the main thing that kept ruby-kafka in place in the first place.

## Conclusion

ruby-kafka is worth reading as a source of design ideas rather than as a dependency: its logging, instrumentation and consumer lifecycle documentation remains the clearest Ruby explanation of how Kafka clients are supposed to behave. You should not adopt it for new work, because the project states plainly that it is no longer actively developed, that the newest tagged release is v1.4.0 from 2021-12-27, and that rdkafka-ruby through librdkafka is the recommended path. If you already depend on it, the practical step is to plan a migration to rdkafka-ruby and check the compatibility table in the README, which records limited support for every Kafka version from 2.0 through 2.7.

## FAQ

### Is ruby-kafka still maintained?

The repository states in its deprecation notice that the library is no longer actively developed and is not recommended for production usage, though it may still receive security patches and bug fixes. The newest tagged release is v1.4.0 from 2021-12-27, and the project points to rdkafka-ruby via librdkafka as the replacement.

### How do I install ruby-kafka?

Add gem 'ruby-kafka' to your Gemfile and run bundle, or install it directly with gem install ruby-kafka. The gem is documented at rubydoc.info/gems/ruby-kafka.

### Which Kafka versions does ruby-kafka support?

The compatibility table gives full producer and consumer support for Kafka 0.9 and 0.10, limited support for Kafka 0.11 with the v0.7.x series, and limited support for every version from Kafka 1.0 through 2.7, since no features specific to those versions were added.

### What should I use instead of ruby-kafka?

The repository recommends rdkafka-ruby, the bindings to librdkafka, on the grounds that other languages use a well-established C binding and receive more official support. It argues there is no point keeping up with Kafka development separately in Ruby.

## Sources

- [License: Apache-2.0](https://github.com/zendesk/ruby-kafka/blob/main/LICENSE)
- [Project website](http://www.rubydoc.info/gems/ruby-kafka)
- [README](https://github.com/zendesk/ruby-kafka/blob/main/README.md)
- [Releases](https://github.com/zendesk/ruby-kafka/releases)
- [zendesk/ruby-kafka on GitHub](https://github.com/zendesk/ruby-kafka)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/zendesk-ruby-kafka