KafkaFlow, a .NET layer over the Confluent Kafka client

What problem KafkaFlow solves for .NET teams

The Confluent Kafka client for .NET is a thin, capable layer over the broker protocol. It is also low level by design: a consumer is a poll loop, a producer is a delivery callback, and any behaviour you want across all topics has to be wired by hand in each place you touch Kafka.

KafkaFlow is a library from Farfetch that sits on top of that client and describes itself as designed to build .NET applications on top of Apache Kafka in a simple and maintainable way. It is extensible by design, and it is built on top of the Confluent Kafka Client rather than replacing it. The repository description calls it an Apache Kafka .NET Framework for building applications that are simple to use and extend.

The audience is .NET backends that publish and consume at a level of volume where boilerplate starts to hurt: order processing, event pipelines, audit trails. Teams that only occasionally send a handful of messages from a small service are unlikely to feel the difference, and would be better served by the client directly.

One practical signal about who uses it: the repository publishes a Slack workspace, a Stack Overflow tag, and a set of samples under samples/ that cover batch operations, consumer throttling, cooperative sticky assignment, flow control, a dashboard, schema registry use, a web API, wildcard consumption, pause-on-error, OpenTelemetry instrumentation, and a plain consumer sample. That spread is a good map of the problems the library is built to absorb.

The middleware pipeline as the core abstraction

KafkaFlow's central idea is that cross-cutting behaviour becomes middleware in a pipeline. The sample list makes the shape of that clearer than the README does: there is a dedicated sample for flow control, another for consumer throttling, another that pauses a consumer on error, and another for OpenTelemetry. These are concerns you would otherwise repeat in every consumer.

Because the pipeline is the extension point, the same mechanism covers producers and consumers. A middleware that adds logging, tracing or a retry policy sits in one place and applies wherever it is registered, which is exactly the maintenance argument the README makes with the phrase maintainable way.

The pipeline is also why the library feels heavier than the client it wraps. Ordering becomes meaningful, since a middleware placed after another observes different state, and a change to shared middleware order affects every topic that uses it. The samples directory, with separate projects for cooperative sticky assignment and flow control, suggests that these configurations are real tuning work rather than a single happy path.

The Confluent client remains underneath, so client-level configuration such as broker addresses and security settings is still expressed in client terms. KafkaFlow does not hide the wire protocol behind a different model of messaging; it adds a programming model around the same client.

Getting a broker and building the solution

The README points at the quickstart for creating a first application and at the documentation site for installation. The repository itself ships a docker-compose.yml for a local Confluent stack, and the Makefile wraps the common commands around it.

Starting a local broker is a single make target:

make init_broker

That target runs docker compose -f docker-compose.yml up -d, which brings up a Zookeeper node, a broker on port 9092 with a metrics port on 9101, and a schema registry on 8081, using the cp-server 7.8.0 and cp-zookeeper 7.8.0 images. The same file sets single-replica factors for the offsets topic and the transaction state log so a single node can run.

The build and test targets follow a familiar .NET shape:

dotnet restore KafkaFlow.sln
dotnet build KafkaFlow.sln

Both are wrapped by Makefile targets named restore and build. The integration test target is the interesting one, because it starts the broker first and stops it afterwards:

dotnet test tests/KafkaFlow.IntegrationTests/KafkaFlow.IntegrationTests.csproj -c Release --logger "console;verbosity=detailed"

That command runs after make init_broker and before make shutdown_broker. The unit test target runs the unit test project twice, once for netcoreapp2.1 and once for netcoreapp3.1. Seeing those older target frameworks pinned in the test target is a fair warning: the library is expected to work across a wide range of .NET versions, but the test matrix in the repository is not tracking the newest ones.

Where KafkaFlow is the wrong choice

The first limitation is scope. KafkaFlow is a Kafka library, not a general messaging abstraction. If the goal is to hide the broker behind one API so that swapping to a different queue later becomes trivial, this is the wrong direction of travel, because its abstractions are named for Kafka concepts such as topics, consumers and producers, and its samples are all Kafka specific.

The second limitation is that it builds on the Confluent client rather than replacing it, which means two dependencies move together. The Confluent client has its own release cadence and its own compatibility requirements, and KafkaFlow's own versioning is separate. A team with a strict policy on dependency count should read the package metadata before committing.

The third is the middleware model itself. The library makes it easy to add behaviour to many topics, which also means it is easy to add behaviour to the wrong topic. Ordering and scope of middleware registration become configuration you must own, and there is no evidence in the repository that KafkaFlow ships tooling to visualise the resulting pipeline.

Finally, the README is thin. It describes the purpose, points to the documentation site and lists contribution channels, but the examples live in separate sample projects and in the hosted docs rather than in the README itself. Budget time for reading the documentation site before estimating adoption.

Release pace, licence and the alternative

The repository is not archived and the last push was on 2026-08-05. Recent releases are KafkaFlow 4.2.0 on 2026-03-11, 4.1.0 on 2025-12-09 and 4.0.1 on 2025-07-30. The spacing suggests roughly quarterly minor releases rather than a fast-moving stream, so an upgrade every few months is a realistic expectation. The licence is MIT, and the README states that KafkaFlow is released under the permissible MIT license, which keeps embedding it inside a proprietary .NET service straightforward with no copyleft obligation on your own code.

The real alternative is the Confluent Kafka client on its own, and the difference in approach is direct. The client gives you a producer, a consumer, a subscription and a delivery report, with no pipeline concept and no opinion about retries or logging. KafkaFlow takes those primitives and imposes a middleware structure so that a policy can be declared once and applied across topics. The same alternative in the other direction is a full enterprise integration platform, which brings schema registries, connectors and routing rules that KafkaFlow does not attempt to cover.

For a team already on the Confluent client, the choice is not between the two but about how much structure you want to own.

Editorial conclusion

Adopt KafkaFlow if your .NET service already runs on the Confluent Kafka client and you want cross-cutting concerns such as logging, retries and tracing moved into a middleware pipeline rather than repeated in each handler. Skip it if your team wants the raw Confluent client API with no added abstraction, or if a NuGet dependency that must be kept in step with the client version is unwelcome. Before choosing it, read the quickstart for creating a first application and confirm that the middleware order documented there matches the retry and logging behaviour your service already depends on.

Frequently asked questions

Official sources

1. Farfetch/kafkaflow on GitHub
2. License: MIT
3. Project website
4. README
5. Releases