Kombu: A Python Messaging Library With Pluggable Transports

The problem Kombu solves: one messaging API, many brokers

Broker client libraries are not interchangeable. A program written against a Redis pub/sub client does not run against RabbitMQ without a rewrite, and a program written against an AMQP library does not run against Amazon SQS. Kombu's stated aim is to make messaging in Python easier by offering an idiomatic high-level interface for AMQP, and it extends that interface to brokers that do not speak AMQP at all.

The audience is application authors who want to support several message server solutions without forking their code per broker. The README puts pluggable transports first in its feature list, ahead of serialization and exception handling. That ordering reflects what the library is really selling: a stable surface above a moving set of backends.

The secondary audience is anyone already running Celery. Kombu is the messaging layer underneath Celery, so a team debugging task routing or connection behaviour is often reading Kombu code whether or not they chose it.

Native and virtual transports, and why the distinction matters

Kombu splits its backends into two kinds. Native transports speak the broker's own protocol: amqp through py-amqp or qpid-python, and qpid. Virtual transports implement Kombu's own semantics on top of a non-AMQP system, and the README lists Redis, Amazon SQS, ZooKeeper, SoftLayer MQ, MongoDB, PGMQ and Pyro in that group. There is also an in-memory transport, which the README describes as being for unit testing.

The transport comparison table is the most useful page in the README, because it is where the abstraction stops being uniform. Direct and topic routing are marked Yes for every transport. Fanout is Yes for amqp, redis, mongodb and pgmq, but No for zookeeper, in-memory, SLMQ and Pyro, and for SQS it is Yes only through AWS SNS with the supports_fanout transport option, disabled by default. Priority is Yes for amqp, redis, mongodb and zookeeper, and No for qpid, pgmq, SQS, in-memory, SLMQ and Pyro. TTL is Yes only for amqp and mongodb.

Two footnotes qualify the table further. For virtual transports that mark topic routing as declaration-only, exchanges and queues must be declared by all clients that need them, because declarations are kept in memory rather than on the broker. For AMQP, both message priority and message or queue TTL depend on the broker implementation, so the Yes in the table is a statement about what Kombu can express, not a guarantee about what your RabbitMQ will do.

Read together, the table says something the feature list does not: the abstraction is real for routing, and thin for delivery guarantees and message lifetime. If your design depends on per-message TTL, you are choosing between amqp and mongodb, and that choice is being made by Kombu rather than by you.

Installing Kombu and publishing a first message

The README points at PyPI for downloads and at kombu.readthedocs.io for documentation. The package name is kombu. Install it with pip:

pip install kombu

That gets you the library itself. Individual transports need their own client libraries, which is why the repository carries a requirements directory with extras; the README does not spell out the extra names, so check the requirements files before assuming a transport works out of the box.

The README's quick overview is the shortest path to a working example. It declares a direct exchange and a queue bound to it with the routing key video, defines a callback that prints the body and acknowledges the message, then opens a connection and publishes a JSON-serialized payload.

from kombu import Connection, Exchange, Queue

media_exchange = Exchange('media', 'direct', durable=True)
video_queue = Queue('video', exchange=media_exchange, routing_key='video')

def process_media(body, message):
    print(body)
    message.ack()

with Connection('amqp://guest:guest@localhost//') as conn:
    producer = conn.Producer(serializer='json')
    producer.publish({'name': '/tmp/lolcat1.avi', 'size': 1301013},
                      exchange=media_exchange, routing_key='video',
                      declare=[video_queue])

The connection URL in the example is amqp://guest:guest@localhost//, the default RabbitMQ credentials on a local broker. The declare argument is what makes the snippet self-contained: passing the queue there means the exchange and queue are created as part of the publish, so a fresh broker does not need pre-provisioning. The callback signature takes the decoded body and a message object, and message.ack() is what tells the broker the message was handled. The README's snippet is truncated at that point, so the consuming half is not shown there. The examples directory in the repository has hello_consumer.py, hello_publisher.py, simple_send.py, simple_receive.py and a memory_transport.py, which is where to look for the receive side and for a transport that needs no broker at all.

What Kombu does not paper over

The first limitation is the one already described: features that live in the broker rather than in the routing layer are not portable. TTL is available on amqp and mongodb only. Priority is unavailable on qpid, pgmq, SQS, in-memory, SLMQ and Pyro. Fanout is unavailable on ZooKeeper, the in-memory transport, SLMQ and Pyro. Code that relies on any of these will not move between transports without changing behaviour, and in some cases it will not run at all.

The second is declaration state. For several virtual transports, topic routing is supported with the caveat that declarations are kept in memory, so every client that needs an exchange or queue must declare it. That is a real operational constraint: it pushes topology setup into each application process instead of into the broker, and a client that forgets to declare will fail against a broker that has never heard of the queue.

The third is cost. The README notes that SQS fanout goes through AWS SNS, that a notification is sent to SNS and a copy is set to every subscribed SQS queue, and that readers should consult the AWS SNS and SQS pricing pages to see how this affects usage costs. That is an unusually direct warning to include in a feature table, and it is worth taking at face value: an abstraction that turns one publish into one SNS notification plus N queue copies has a bill attached.

The fourth is that Kombu is a library, not a broker. It does not add durability, ordering or delivery guarantees that the underlying transport lacks. The README frames its reliability claims around handling connection and channel errors gracefully, which is error handling, not message persistence.

Kombu compared with talking to the broker directly

The straightforward alternative is to use the broker's own client library and skip the abstraction. With RabbitMQ that means py-amqp or pika; with Redis it means redis-py; with SQS it means boto3. The difference in approach is where the configuration lives. Direct clients expose the broker's full feature set and its native error model, and they let you use features the Kombu transport table marks as No. What you give up is portability: moving from Redis to RabbitMQ becomes a rewrite of the messaging layer rather than a change of connection URL.

Within the Celery ecosystem there is a second comparison worth drawing. Celery itself is the task queue, and it uses Kombu as its transport layer. If your goal is to run background tasks, Celery is the higher-level tool and Kombu is the plumbing underneath it; adopting Kombu directly only makes sense if you want the messaging primitives without the task machinery. The repository's own examples directory includes a simple_task_queue example, which suggests the maintainers expect people to build in that direction.

A third option, for readers who want AMQP semantics with a different client, is qpid-python, which Kombu lists as an alternative AMQP transport alongside py-amqp. It is worth noting that the transport table marks qpid as lacking priority and TTL support, so it is not a drop-in substitute at the feature level.

Maintenance, licence and upgrade cost

The repository is not archived, and its last push was on 2026-09-23. Recent releases in the repository metadata are v5.7.0a1 from 2026-09-21, v5.6.2 from 2025-12-29 and v5.6.1 from 2025-11-25. The current version string in the README is 5.7.0a1, which is a pre-release, so anyone installing from PyPI without pinning may or may not land on it depending on how pip resolves pre-releases for the package. Pin the version you have tested.

The licence is BSD-3-Clause. That is a permissive licence, and it is the same family used across the Celery organisation. This is not legal advice; if you redistribute Kombu or bundle it into a product, read the LICENSE file in the repository and take your own advice on attribution obligations.

Upgrade cost is dominated by the transport dependencies rather than by Kombu itself. A version bump can change which client library version is expected for a given transport, and the repository keeps those pins in the requirements directory rather than in the README. Before upgrading, compare the requirements files between the two tags you are moving between. The Makefile also exposes a test-all target that runs tests across supported Python versions, which is the maintainers' own compatibility check and a reasonable thing to mirror in CI.

Editorial conclusion

Kombu is a reasonable choice if you already run Celery, or if you want one Python API for producing and consuming messages across more than one broker. It is the wrong tool when you need a broker feature the transport table marks as unsupported, such as TTL on Redis or fanout on ZooKeeper. Before adopting it, check the transport comparison table against the exact broker and feature you need, and confirm which optional dependency your transport requires.

Frequently asked questions

Official sources

1. celery/kombu on GitHub
2. License: BSD-3-Clause
3. Project website
4. README
5. Releases