# librdkafka is the C library many Kafka clients are built on

> librdkafka implements the Apache Kafka protocol in C, with Producer, Consumer and Admin clients, exactly-once support, a preview share consumer, and the compression, SSL and SASL options a production deployment needs. It is the layer you link against, and the licence it ships under is not the one GitHub shows by default.

**confluentinc/librdkafka** — The Apache Kafka C/C++ library

- Repository: https://github.com/confluentinc/librdkafka
- Stars: 1,070 · Forks: 3,274
- Language: C
- License: NOASSERTION
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/confluentinc-librdkafka

## A protocol implementation in C, and who actually uses it

librdkafka is a C library implementation of the Apache Kafka protocol, providing Producer, Consumer and Admin clients. It was designed with message delivery reliability and high performance in mind, and the README quotes current figures that exceed 1 million messages per second for the producer and 3 million messages per second for the consumer.

Those numbers are the project's own, measured on hardware the README does not specify, so they are best read as an order of magnitude rather than a promise for your deployment. What they do show is that the library targets the throughput end of the range.

The reason so many teams end up here is indirect. A lot of the popular Kafka clients for other languages are not independent protocol implementations, they are bindings over this library. If you write a client in a language that has no first party Kafka client, or you are writing in C or C++ and want something you can link rather than something you run out of process, this is the layer underneath.

The project also has a legal footnote worth reading, because it clarifies the relationship with the Apache Kafka project itself. KAFKA is a registered trademark of The Apache Software Foundation and has been licensed for use by librdkafka, and the project states plainly that it has no affiliation with and is not endorsed by The Apache Software Foundation. It is a separate implementation of an open protocol, and it says so.

The code is copyright 2012-2022 Magnus Edenhill, with 2023 and after Confluent Inc.

## Installing from a package manager on every major platform

Prebuilt packages are the documented route, and the README covers each platform by name.

On MacOS it is Homebrew.

```bash
$ brew install librdkafka
```

On Debian and Ubuntu the package comes from the Confluent APT repositories, and the library itself is installed as the development package.

```bash
$ apt install librdkafka-dev
```

On RedHat, CentOS and Fedora the same pattern applies through the Confluent YUM repositories, with the development package named differently.

```bash
$ yum install librdkafka-devel
```

On Windows the answer is not a package manager but a Visual Studio reference, using the librdkafka.redist NuGet package in the project. For other platforms the README points you to building from source.

There is also a vcpkg route, and the instructions include bootstrapping the tool itself if you do not have it.

```bash
$ git clone https://github.com/Microsoft/vcpkg.git
$ cd vcpkg
$ ./bootstrap-vcpkg.sh
$ ./vcpkg integrate install
$ vcpkg install librdkafka
```

The project notes that the librdkafka package in vcpkg is kept up to date by Microsoft team members and community contributors, and that if the version is out of date you should open an issue or pull request on the vcpkg repository. That is a small thing, but it tells you where to look when the packaged version lags the release you want.

## Building from source, and what the optional dependencies buy

The source build is autotools style. The requirements list is short, and the optional entries are the interesting part because each one unlocks a specific feature.

The mandatory items are the GNU toolchain, GNU make and pthreads. Beyond that, zlib-dev is optional and gives gzip compression, libssl-dev is optional and gives SSL and SASL SCRAM support, libsasl2-dev is optional and gives SASL GSSAPI support, libzstd-dev is optional and gives ZStd compression, and libcurl-dev is optional and gives SASL OAUTHBEARER OIDC support.

So the security features of the client are opt-in at build time. A build without libssl-dev has no SSL and no SCRAM, and a build without libsasl2-dev has no Kerberos. That is the kind of thing to check in CI if you consume a prebuilt binary from a distribution rather than building it yourself.

The build itself is three commands.

```bash
./configure
make
sudo make install
```

The configure script has two extra options documented alongside it, --install-deps to pull dependencies through the system package manager and --install-deps --source-deps-only to build them from source instead. Windows has its own instructions in README.win32, and there is an experimental CMake build that is explicitly marked as unsupported, with the note telling you to look at the packaging/cmake README.

One tuning detail is worth copying. Statically linking ZStd, with zstd 1.2.1 or newer, lets the producer encode the original size in the compression frame header, which the README says will speed up the consumer.

## What the client supports, and what is still marked preview

The feature list in the README is long enough to be worth sorting by how safe it is to depend on.

Solid ground first. There is full exactly-once semantics support, a high-level producer including idempotent and transactional producers, a high-level balanced KafkaConsumer, a simple legacy consumer and an admin client. Compression covers snappy, gzip, lz4 and zstd. Security is covered by SSL support and SASL support covering GSSAPI with Kerberos and SSPI, PLAIN, SCRAM and OAUTHBEARER. The API stability guarantee is stated as guaranteed for the C and C++ APIs, with ABI safety guaranteed for C.

Broker support reaches back to 0.8, and the balanced high-level consumer requires a broker of 0.9 or newer.

Then there is the feature to be careful with. The share consumer for Queues for Kafka, which is KIP-932, is labelled Preview, it is available through the C API only, and it requires a broker of 4.2 or newer. A preview feature with a floor on broker version is a two-sided constraint, and an evaluation should start by checking what your brokers actually run.

Statistics metrics are exposed through a dedicated document, which matters more than it sounds, because the metrics are how you tell a healthy client from a quietly stuck one.

## No built-in Schema Registry serializer, and that is deliberate

One gap in the library is called out in the usage section, and it is worth reading carefully because it changes what your application has to own.

The README says that production applications should use Schema Registry, and explains why, saying that producing raw bytes leads to data-quality issues, broken consumers, and ungovernable data. Then it says that librdkafka has no built-in Schema Registry serializer. The example, producer_schema_registry.cpp, exists for this reason and it does not use the library for the job. It uses libschemaregistry and avro-cpp.

The practical consequence is that schema handling is your code's responsibility. librdkafka moves bytes reliably, and it does not decide what those bytes mean or guarantee that producer and consumer agree on the format. If your data pipeline depends on a schema contract, that contract lives in a separate library that you integrate and that you then have to keep in step with the broker side.

The same pattern appears elsewhere in the feature list. Consumer group administration is exposed through the admin client and through the example programs, alter_consumer_group_offsets.c, delete_records.c, describe_topics.c, elect_leaders.c, list_offsets.c and others, but the operational work is still yours.

Linking is simple. You link your program with -lrdkafka for C or -lrdkafka++ for C++, and the examples directory has working programs for producers, idempotent producers, transactional producers, basic consumers, batch consumers, share consumers, and performance and latency testing tools.

## The build generates its own configuration reference

The Makefile is unusual in a way worth knowing, because it means the configuration reference in the repository cannot drift from the code.

The CONFIGURATION.md file is a build target. The rule runs examples/rdkafka_performance with the -X list option, rewrites the pipe characters in the output, compares the result against the checked-in file, and replaces it if it differs. The configuration properties you read in CONFIGURATION.md are therefore produced by running the library and asking it to list its own properties, not written by hand.

The rest of the build is a custom make system called mklove, included through mklove/Makefile.base, which supplies the shared rules for the src and src-cpp subdirectories, the tests and the examples. The version is not hardcoded either, it is extracted by running packaging/get_version.py over src/rdkafka.h, so the header and the package metadata cannot disagree.

Documentation is generated with Doxygen, using the Doxyfile at the root, and the build has a doc-install and doc-uninstall target alongside the usual install paths. There is a copyright check with an explicit skip list covering tests, packaging, src/nanopb and src/opentelemetry, which is a hint that the tree vendors or adapts a few third party components that keep their original headers.

The continuous integration runs on Semaphore, with a .semaphore/ directory, and the tree also carries a debian/ directory for Debian packaging, a win32/ directory, and a vcpkg.json manifest.

## Master is not the branch to deploy, and the licence needs a closer look

The README puts a note near the top of the documentation section that is easy to skim past, and it is the one line to internalise before anything else.

The master branch is actively developed, and for production use you should take the latest release.

The release history is regular. v2.16.0 was published on 2026-10-06, v2.15.1 on 2026-09-09 and v2.15.0 on 2026-06-30. The repository is not archived, and the last push was on 2026-10-19, so the difference between master and the newest release is a matter of days here, and in a deployment that difference is still the difference between a tested artefact and a moving target.

The licence needs a closer look than a badge. The README states that librdkafka is licensed under the 2-clause BSD licence, which is permissive and short. But the repository listing shows a LICENSE file for the project plus a separate LICENSE file for each bundled component, including cjson, crc32c, fnv1a, hdrhistogram, lz4, murmur2, nanopb, opentelemetry, pycrc, queue, regexp, snappy and tinycthread and wingetopt, plus a LICENSES.txt that collects them.

That is the ordinary arrangement for a library that vendors parts of other projects, and it is why GitHub cannot classify the repository under a single standard licence. The obligations that matter to you are in the individual files, so a compliance check should read LICENSES.txt rather than stop at the README.

## Conclusion

Adopt librdkafka when your application is written in C or C++, since it is the layer many Kafka clients in other languages build on. Do not adopt it if you need a managed client with a rich higher level API, and do not expect it to serialise your schemas for you, because the project states it has no built-in Schema Registry serializer. Verify first which branch you build from, since the documentation explicitly recommends a release rather than master for production, and check the licence position, because the README describes a 2-clause BSD licence while the repository ships a LICENSE file per bundled dependency alongside LICENSES.txt.

## FAQ

### What is librdkafka used for?

It is a C library implementation of the Apache Kafka protocol providing Producer, Consumer and Admin clients, with exactly-once semantics, idempotent and transactional producers, compression, SSL and SASL support. Many clients in other languages are built on top of it.

### How do I install librdkafka on Linux or MacOS?

Use brew install librdkafka on MacOS, apt install librdkafka-dev on Debian and Ubuntu from the Confluent APT repositories, and yum install librdkafka-devel on RedHat, CentOS and Fedora from the Confluent YUM repositories. On Windows, reference the librdkafka.redist NuGet package from Visual Studio.

### How do I build librdkafka from source?

Run ./configure, make and sudo make install. The build needs the GNU toolchain, GNU make and pthreads, and the optional libraries enable features: zlib-dev for gzip, libssl-dev for SSL and SCRAM, libsasl2-dev for GSSAPI, libzstd-dev for ZStd and libcurl-dev for SASL OAUTHBEARER OIDC.

### Does librdkafka handle Schema Registry?

No. The project states that librdkafka has no built-in Schema Registry serializer and that production applications should use Schema Registry, since producing raw bytes causes data quality issues. The example producer_schema_registry.cpp uses libschemaregistry and avro-cpp instead.

### What does it take to link a program against librdkafka?

Link with -lrdkafka for C or -lrdkafka++ for C++. The examples directory contains working programs for producers, idempotent and transactional producers, basic and batch consumers, share consumers and performance testing tools.

### Which broker versions does librdkafka support?

Broker version support goes back to 0.8, the high-level balanced consumer requires broker 0.9 or newer, and the share consumer for Queues for Kafka is marked Preview, C API only, and requires broker 4.2 or newer.

## Sources

- [confluentinc/librdkafka on GitHub](https://github.com/confluentinc/librdkafka)
- [Issues](https://github.com/confluentinc/librdkafka/issues)
- [README](https://github.com/confluentinc/librdkafka/blob/master/README.md)
- [Releases](https://github.com/confluentinc/librdkafka/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/confluentinc-librdkafka