kafka-rust: Rust Client APIs for Kafka Producers and Consumers

kafka-rust targets Rust applications that produce or consume Kafka records

kafka-rust solves the problem of connecting Rust applications to Apache Kafka. Its documented users include client applications that send records to Kafka and applications that receive records as part of a consumer group. The project provides both high-level modules, `kafka::producer` and `kafka::consumer`, and a lower-level `kafka::client::KafkaClient` abstraction.

The producer API handles partition assignment through partitioners. The consumer API handles offsets on behalf of a specified group. Applications that need lower-level Kafka operations can use `KafkaClient` for metadata loading, topic offset retrieval, message sending, message fetching, and committing or fetching consumer-group offsets.

This is a client library, not a Kafka broker. It does not create a Kafka cluster, manage ZooKeeper, or provide a complete application framework. The README also warns that not everything works correctly at present, testing coverage could be better, and production use is at the user's risk. That warning directly affects adoption decisions. A team evaluating kafka-rust should treat compatibility testing and failure handling as part of the integration work rather than assuming that a successful compile proves production readiness.

The project is not archived, and its last push was on 2026-07-02. The README states that maintenance was being brought up to date, including newer dependencies, removal of deprecated Rust code, and test adjustments. That maintenance status does not remove the compatibility and testing limits described by the project itself.

Cargo dependency selection controls compression and TLS backends

The README presents Cargo as the installation path and says the crate is available on crates.io as `kafka`. It gives a basic dependency declaration using version `0.10`, while the repository manifest declares package version `0.11.0`. That difference matters for dependency review. The README also says that the API is under heavy movement, although it follows semantic versioning and expects version numbers to grow quickly.

The default feature set includes `snappy`, `gzip`, and `security`. According to the README, the `security` feature now uses rustls as the default TLS backend. The dependency configuration enables `rustls`, `webpki-roots`, `rustls-pemfile`, and `rustls-native-certs`. The project describes rustls as requiring no additional system dependencies and targeting TLS 1.2 or later.

OpenSSL remains available as a deprecated backend. The README says it will be removed in the next major version, so a new application has a clear reason to prefer rustls unless an existing dependency chain requires OpenSSL. A build without TLS support can disable default features and retain `snappy` and `gzip`.

The manifest also exposes `producer_timestamp`, `nightly`, and `integration_tests` as optional features. The README does not give detailed behavior for each of these features, so their operational effects should be checked in the project documentation and generated API docs before they enter a build configuration. The feature list makes compile-time trade-offs visible, but it does not by itself prove that every Kafka protocol feature is implemented.

Installing kafka-rust and selecting a TLS configuration

A Rust application can add the client through `Cargo.toml`. The following is the README's basic dependency declaration, and it should be copied into the application's dependency table when version `0.10` is the intended API target.

[dependencies]
kafka = "0.10"

The project also documents a rustls configuration with the `security` feature. Since `security` is included in the manifest's default feature list, the explicit form is useful when the security choice should remain visible in configuration.

[dependencies]
kafka = { version = "0.10", features = ["security"] }

For OpenSSL, the README gives this deprecated configuration. It should be treated as a compatibility path rather than the preferred setup for a new project.

[dependencies]
kafka = { version = "0.10", default-features = false, features = ["snappy", "gzip", "security-openssl"] }

To build without TLS, use the configuration below. The command that follows is the documented build command.

[dependencies]
kafka = { version = "0.10", default-features = false, features = ["snappy", "gzip"] }

cargo build

The README says the usual `cargo build` should suffice. A reader should verify the resolved dependency versions and inspect compiler errors when changing Cargo features, because the README does not document every feature interaction.

Producer and consumer APIs map to Kafka communication patterns

The producer is the appropriate API when an application sends messages to Kafka. The README states that `kafka::producer` provides automatic partition assignment capabilities through partitioners. That means the application can work with the producer abstraction rather than implementing Kafka partition selection itself.

The consumer is intended for receiving messages. The README identifies `kafka::consumer` as a higher-level API with convenient offset management for a specified group. This is useful for applications that consume records as members of a consumer group, but the documentation excerpt does not specify retry policies, rebalance behavior, commit timing, or delivery guarantees. Those details must be established from the API documentation, examples, and application tests.

`KafkaClient` sits below these modules. Its listed responsibilities include loading metadata, fetching topic offsets, sending and fetching messages, and committing or fetching consumer-group offsets. This division creates a useful boundary between ordinary application code and protocol-level operations, but it also means that choosing `KafkaClient` directly increases the amount of behavior an application must coordinate.

The repository contains producer and consumer examples, including `console-producer.rs`, `console-consumer.rs`, `example-produce.rs`, and `example-consume.rs`. The README points to the `examples` directory for standalone compilable programs and to docs.rs for API examples. Those files are the appropriate starting points for verifying the actual call shape, since the README does not include complete producer or consumer programs.

Topic creation is an external Kafka operation

kafka-rust does not document topic creation as part of the client library workflow. The README instead gives commands from the Kafka distribution. The first documented command uses ZooKeeper:

kafka-topics.sh --topic my-topic --create --zookeeper localhost:2181 --partitions 1 --replication-factor 1

The README notes that ZooKeeper will be removed in the next major Kafka release and recommends using `--bootstrap-server` to be more ready. It then provides this command:

kafka-topics.sh --topic my-topic --create --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

This separation is operationally important. The client application can send or fetch records, but the deployment process still needs a reachable Kafka broker and a topic with suitable partitions and replication settings. The documented commands use `localhost:2181` for ZooKeeper and `localhost:9092` for Kafka, so those addresses are local examples rather than universal deployment defaults.

The README also states that the library ignores requests for topics it does not know about and will not attempt to retrieve messages from nonexistent or unknown topics. A missing topic can therefore appear as an application-level absence of data rather than an automatic topic-creation request. Applications should treat topic provisioning as a separate deployment concern.

Compatibility, tests, and maintenance limits define the adoption boundary

The README says kafka-rust is tested for compatibility with a select few Kafka versions from `0.8.2` to `3.1.0`. It immediately qualifies that statement by saying that not all features from Kafka `0.9` and newer are supported yet. The version range therefore describes a tested selection, not a promise that every Kafka feature across that range is implemented.

The project explicitly reports that testing coverage could be better and that some behavior does not work correctly. Its recommendation to use it in production at the user's own risk is unusually direct. For a service that depends on delivery, retries, offsets, or protocol features introduced after Kafka 0.8, the missing feature list is a concrete evaluation boundary.

Integration tests require Docker `1.10.0+` and docker-compose `1.6.0+`. The README says to run the included `run-all-tests` script in the `tests` directory and inspect that script for usage details. This gives contributors a documented integration path, but the README does not provide its command syntax or enumerate the Kafka features covered.

The manifest identifies the Rust edition as `2024` and sets `rust-version` to `1.85.0`. It declares the package license as `MIT OR Apache-2.0`, while the repository summary identifies the project license as MIT. That licensing difference should be checked against the repository's `LICENCE` file and the dependency licenses before redistribution. The repository is not archived, and the last push was on 2026-07-02, but a recent push does not resolve the README's stated test coverage warning.

rust-rdkafka offers a different Rust-to-Kafka integration path

The README names `rust-rdkafka` as an alternative. Its documented difference is architectural: kafka-rust is a Rust client library, while rust-rdkafka provides a safe Rust interface to `librdkafka`. The README describes rust-rdkafka as based on `librdkafka`, while kafka-rust exposes producer, consumer, and client APIs.

The choice is therefore not simply between two Rust syntaxes. It is a choice between kafka-rust's native Rust client model and rust-rdkafka's safe wrapper around `librdkafka`. The README does not provide a feature comparison, performance comparison, or compatibility matrix for rust-rdkafka, so it cannot support claims that one option is faster or more complete.

For an engineer choosing kafka-rust, the relevant comparison points are the Kafka protocol features required by the application, the documented Kafka version selection, the required compression features, the preferred TLS backend, and the amount of integration testing available for the chosen API level.

Editorial conclusion

Adopt kafka-rust for Rust applications that need native producer or consumer APIs and can accept its documented compatibility limits. Do not choose it without checking required Kafka features, TLS deployment, and test coverage first. The concrete first step is to compile the crate with the intended feature set and run the repository examples against the Kafka versions your service supports.

Frequently asked questions

Official sources

1. Issues
2. kafka-rust/kafka-rust on GitHub
3. License: MIT
4. README
5. Releases