# confluent-kafka-go: a librdkafka wrapper for Go, and the CGO cost that comes with it

> Confluent's Go client for Apache Kafka wraps the C library librdkafka through cgo. That buys protocol behaviour shared with Confluent's other clients, and it charges you a build toolchain and a binary size.

**confluentinc/confluent-kafka-go** — Confluent's Apache Kafka Golang client

- Repository: https://github.com/confluentinc/confluent-kafka-go
- Stars: 5,167 · Forks: 703
- Language: Go
- License: Apache-2.0
- Published: 2026-09-23 · Updated: 2026-09-23 · Language: en
- Canonical page: https://hysenlabs.com/projects/confluentinc-confluent-kafka-go

## The problem confluent-kafka-go actually solves

Writing a Kafka client means implementing a wire protocol that changes, handling partition assignment, rebalances, offset commits, retries and delivery reports. The README is explicit about the strategy: rather than reimplement that in Go, confluent-kafka-go is "a lightweight wrapper around librdkafka, a finely tuned C client." The stated reason is that these details get fixed once, in librdkafka, and that work is shared with confluent-kafka-python and confluent-kafka-dotnet.

The audience follows from that. If your stack already runs Confluent's Python or .NET clients against the same brokers, the Go client gives you the same configuration keys and the same failure semantics, because the same C library is underneath. If you are a Go team that has never touched librdkafka, the trade is different: you inherit a C dependency and a cgo build in exchange for protocol behaviour you did not have to write.

The README also states that commercial support is offered by Confluent, and that keeping client features in step with Apache Kafka and the Confluent Platform is a stated priority. Treat that as vendor positioning rather than a technical property, but it does explain why the repository ships a schemaregistry package alongside kafka: the client is aimed at people using Confluent's wider platform, not only at people running bare Apache Kafka.

## High-level producer and consumer over a C core

The repository splits into two importable packages under the module path github.com/confluentinc/confluent-kafka-go/v2: kafka for the client itself, and schemaregistry for serialization. The layout reflects the split. Top-level entries include kafka/, schemaregistry/, examples/, kafkatest/ and soaktest/, so there is an in-repo test harness and a long-running soak suite rather than only unit tests.

The consumer API is a poll loop. You construct a consumer from a ConfigMap, subscribe to topics (the README's example passes both a literal topic name and a regex, "^aRegex.*[Tt]opic"), then call ReadMessage with a timeout. The README's comment is worth reading closely: timeouts are not treated as errors because ReadMessage raises them when no message arrives, and the client "will automatically try to recover from all errors." That is librdkafka's recovery logic, not Go code you can inspect and step through. When a rebalance misbehaves, your debugging surface includes a C library.

The producer is asynchronous. Produce enqueues, and delivery outcomes arrive on the Events channel, which the README's example drains in a goroutine and inspects for TopicPartition.Error. Shutdown is explicit: the example calls Flush with a 15 second budget before exiting. That pattern matters because dropping the process without flushing loses queued messages.

On the serialization side, the README is opinionated: "Production applications should serialize with Schema Registry. Producing raw bytes leads to data-quality issues, broken consumers, and ungovernable data." The example registers a JSON Schema serializer against a Schema Registry URL and calls Serialize before producing. Avro and Protocol Buffers serializers exist too, with example directories named avrov3_producer_example and protobuf_producer_example.

## Installing confluent-kafka-go and producing a first message

The README states the version floor plainly: Go 1.26+ and librdkafka 2.15.0+. The go.mod in the repository declares go 1.26.0, so a stale toolchain will fail before you reach any Kafka code.

Add the dependency with the module path the README gives:

```bash
go get -u github.com/confluentinc/confluent-kafka-go/v2/kafka
```

Then import it. The README's own import line is the one to copy, including the /v2 segment:

```golang
import "github.com/confluentinc/confluent-kafka-go/v2/kafka"
```

Build the project. The README shows the plain form first, then the Alpine variant, because musl needs a build tag:

```bash
go build ./...
```

```bash
go build -tags musl ./...
```

A consumer that connects to a broker on localhost, joins the group myGroup and reads from the start of the topic looks like this, following the README's high-level consumer example:

```golang
c, err := kafka.NewConsumer(&kafka.ConfigMap{
	"bootstrap.servers": "localhost",
	"group.id":          "myGroup",
	"auto.offset.reset": "earliest",
})
if err != nil {
	panic(err)
}
if err := c.SubscribeTopics([]string{"myTopic"}, nil); err != nil {
	panic(err)
}
msg, err := c.ReadMessage(time.Second)
```

What you should see is a message printed with its TopicPartition and value. A timeout is normal and is not an error, per the README's note. The librdkafka binaries are bundled, so you do not install librdkafka separately on the build or target system for the supported platforms: macOS x64 and arm64, glibc Linux x64 and arm64, musl Linux amd64 and arm64, and Windows amd64. The bundled builds on Linux and Windows omit GSSAPI/Kerberos support.

## CGO_ENABLED=0, Kerberos and the platforms the bundle skips

The hardest constraint is stated in one line: "CGO_ENABLED must NOT be set to 0 since the Go client is based on the C library librdkafka." If your release pipeline builds static binaries, runs distroless or scratch images, or cross-compiles without a C toolchain for the target, this client is the wrong choice and no build flag fixes it. That is a deployment-architecture decision, not a tuning detail.

The second constraint is Kerberos. The bundled librdkafka binaries ship without GSSAPI/Kerberos support on glibc Linux, musl Linux and Windows. The README's remedy is to install librdkafka separately on both the build and target system and then build your Go application with -tags dynamic. Note what that means operationally: the target system now needs a compatible librdkafka present at runtime, so the binary is no longer self-contained. The README does not document rollback or downgrade procedures for that arrangement.

Alpine is a smaller version of the same problem. You must pass -tags musl to go get and go build, and the README states this for Alpine specifically. A build that works on Debian and fails on Alpine is usually a missing tag, not a missing package.

There is also a dependency-weight question. The go.mod requires a long list including the AWS, Azure and Google SDKs, HashiCorp Vault, Tink, cel-go and several Avro and JSON Schema libraries. Those serve the schemaregistry package's serializer and key-management paths. If you only use kafka and never touch the registry, you are still resolving that graph.

## Sarama and segmentio/kafka-go: what changes without cgo

The most common comparison is with Sarama, a pure-Go Kafka client. The difference is not feature parity in the abstract; it is where the protocol logic lives. Sarama implements the Kafka protocol in Go, so you can read the code, set CGO_ENABLED=0, and produce a static binary with no C toolchain. confluent-kafka-go delegates that logic to librdkafka, so you get whatever librdkafka does, including its automatic error recovery, without a Go-level equivalent to step through. The README frames this as the reliability argument: the details are got right in one place and shared across clients.

segmentio/kafka-go is the other frequently named alternative. It is also pure Go and is often reached for in smaller services where a single dependency and a simple reader/writer API are enough. The trade is the same shape: no cgo, no bundled C library, and none of librdkafka's shared configuration surface across languages.

The practical way to choose is to look at what you already run. If you have a fleet of confluent-kafka-python consumers and you want the Go service to behave identically on rebalance and offset handling, the shared librdkafka core is the point of the library. If your Go service is standalone and your build pipeline cannot host a C compiler, a pure-Go client removes a whole class of build problems. Neither answer is universal, and the README does not attempt a comparison.

## Versioning, licence and what an upgrade actually costs

The module path carries /v2, and the recent releases listed for the repository are v2.15.1, v2.15.0 and v2.14.2. The README ties the client to librdkafka 2.15.0+ and Go 1.26+, so an upgrade can require moving both your Go toolchain and, if you use -tags dynamic, the librdkafka installed on your target hosts. That second move is the one teams forget: with bundled binaries the upgrade is a go.mod change, with a dynamic build it is a go.mod change plus a system package change on every host.

Because the client is a thin wrapper, most behaviour changes arrive through librdkafka rather than through Go code you can diff. The repository keeps a CHANGELOG.md at the top level, and that is the file to read before bumping the version, especially when you depend on delivery-report timing or rebalance behaviour.

The licence is Apache-2.0, per the repository. That is a permissive licence and it is the same licence family as Apache Kafka itself. This is not legal advice, but the practical implication is that the licence does not by itself force you to publish your application source. The separate question is the commercial support Confluent offers, which is a contract, not a licence term, and the README does not describe its scope.

## Conclusion

Adopt confluent-kafka-go when you want the same librdkafka behaviour across Go, Python and .NET services, and when your build images can carry a C toolchain. Stay with a pure-Go client if your deployment target is a scratch container, a static binary, or a platform the bundled librdkafka builds do not cover. Before committing, confirm three things: that your Go toolchain satisfies the go.mod requirement of go 1.26.0, that your target platform appears in the prebuilt librdkafka list, and whether you need GSSAPI/Kerberos, which forces a separate librdkafka install and a -tags dynamic build. The schemaregistry subpackage is the other decision point: if you do not run a Schema Registry endpoint, you are carrying dependencies you will not call.

## FAQ

### How do I install confluent-kafka-go?

Add the module with go get -u github.com/confluentinc/confluent-kafka-go/v2/kafka, then import github.com/confluentinc/confluent-kafka-go/v2/kafka in your code. The README requires Go 1.26+ and librdkafka 2.15.0+, and on Alpine you must pass -tags musl to go get and go build.

### How does confluent-kafka-go compare with a pure-Go client like Sarama?

confluent-kafka-go wraps the C library librdkafka, so it cannot be built with CGO_ENABLED=0, while Sarama implements the Kafka protocol in Go and can produce a static binary. The payoff for the cgo dependency is that librdkafka's protocol handling and configuration surface are shared with Confluent's Python and .NET clients.

### Does confluent-kafka-go need librdkafka installed separately?

No on the supported platforms: prebuilt librdkafka binaries ship with the Go client and cover macOS x64 and arm64, glibc Linux x64 and arm64, musl Linux amd64 and arm64, and Windows amd64. You do need to install librdkafka yourself if you require GSSAPI/Kerberos support, which the bundled Linux and Windows builds omit, and then build with -tags dynamic.

### Does confluent-kafka-go work with Schema Registry?

Yes. The repository has a schemaregistry package with JSON Schema, Avro and Protocol Buffers serializers, and the README's producer example registers a serializer against a Schema Registry URL before producing. The README advises that production applications should serialize with Schema Registry rather than producing raw bytes.

### Why does my confluent-kafka-go build fail on Alpine Linux?

Alpine uses musl libc, and the README states that -tags musl must be specified when building for it, for go get and go build alike. Without the tag the bundled librdkafka binary for musl is not selected.

## Sources

- [confluentinc/confluent-kafka-go on GitHub](https://github.com/confluentinc/confluent-kafka-go)
- [Issues](https://github.com/confluentinc/confluent-kafka-go/issues)
- [License: Apache-2.0](https://github.com/confluentinc/confluent-kafka-go/blob/master/LICENSE)
- [README](https://github.com/confluentinc/confluent-kafka-go/blob/master/README.md)
- [Releases](https://github.com/confluentinc/confluent-kafka-go/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/confluentinc-confluent-kafka-go