# fast-data-dev: a single Docker image for Kafka, Schema Registry and Connect

> fast-data-dev packs Kafka, Confluent Schema Registry and Kafka Connect with Stream Reactor connectors into a single Docker image for local and CI development. The README lists Kafka 3.9.0 as a component, but the Dockerfile shows version 4.3.1 with a note that ZooKeeper was dropped at 4.0.

**lensesio/fast-data-dev** — Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, , 20+ connectors

- Repository: https://github.com/lensesio/fast-data-dev
- Website: https://lenses.io
- Stars: 2,080 · Forks: 339
- Language: Shell
- License: Apache-2.0
- Published: 2026-10-11 · Updated: 2026-10-11 · Language: en
- Canonical page: https://hysenlabs.com/projects/lensesio-fast-data-dev

## What fits in the container: Kafka, Schema Registry, Connect, and pre-installed connectors

fast-data-dev puts everything a Kafka development environment needs into a single container. The Kafka Broker runs in KRaft mode, the Confluent Schema Registry is included, and Kafka Connect comes with Stream Reactor connectors already installed. On top of that, the container includes data generators and a web interface at port 3030.

Stream Reactor connectors cover AWS S3, Azure Datalake, Azure DocumentDB, Azure EventHubs, Azure ServiceBus, Cassandra, ElasticSearch, FTP, Google Cloud Platform PubSub, Google Cloud Platform Cloud Storage, HTTP, JMS and MQTT, among others. Debezium connectors for CDC use cases are also included. Having these pre-installed means you do not need to download and mount connector JARs before testing a pipeline.

The project was first introduced in 2016. Both Kafka and Confluent Schema Registry are built from source rather than taken from a pre-built distribution, which matters when you need to run against a specific Kafka version for compatibility testing.

## Starting the full environment: four port mappings and one command

The simplest startup maps four ports and pulls the image:

```bash
docker run --rm \
    -p 9092:9092 \
    -p 8081:8081 \
    -p 8083:8083 \
    -p 3030:3030 \
    lensesio/fast-data-dev
```

After the image starts, the Kafka Broker is available at localhost:9092, the Schema Registry at http://localhost:8081, Kafka Connect at http://localhost:8083, and the web interface at http://localhost:3030. Each service has its own port, which means you can use a standard Kafka client pointed at localhost:9092 without changing any connection string beyond the host.

For remote access from another machine or a cloud VM, the ADV_HOST environment variable tells the broker what address to advertise in its metadata responses:

```bash
docker run --rm \
    -e ADV_HOST=<YOUR_IP_ADDRESS or DNS name> \
    -p 9092:9092 \
    -p 8081:8081 \
    -p 8083:8083 \
    -p 3030:3030 \
    lensesio/fast-data-dev
```

This setup may require allowing connections from the VM to itself. A cloud VM that does not have its own IP in the firewall allowlist will prevent the broker from advertising correctly, and this is worth checking before assuming the broker is broken.

## Disabling connectors and generators to cut RAM usage in docker-compose

Running all pre-installed connectors uses RAM even if you never use them. Two environment variables control what runs: DISABLE takes a comma-separated list of connector names to turn off, and CONNECTORS limits the set to only the connectors you name. The docker-compose example in the README disables five Debezium connectors explicitly:

```yaml
  kafka:
    image: lensesio/fast-data-dev:3.9.0
    hostname: kafka
    environment:
      ADV_HOST: kafka
      RUNNING_SAMPLEDATA: 1
      RUNTESTS: 0
      KAFKA_LISTENERS: PLAINTEXT://:9092,DOCKERCOMPOSE://:19092,CONTROLLER://:16062
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,DOCKERCOMPOSE://demo-kafka:19092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: >
        DOCKERCOMPOSE:PLAINTEXT,CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,
        SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL
      DISABLE: debezium-mongodb,debezium-mysql,debezium-postgres,debezium-sqlserver,debezium-jdbc
    ports:
      - 9092:9092
      - 8081:8081
      - 8083:8083
      - 3030:3030
```

This docker-compose configuration also shows the dual-listener setup for mixed host and container access: the broker is at localhost:9092 from the host and at kafka:19092 from inside the compose network. Setting RUNTESTS to 0 skips the built-in self-tests that run at startup. Disabling Kafka Connect entirely uses CONNECT_PORT=0.

## The README's component list says 3.9.0 but the Dockerfile builds Kafka 4.3.1

The README has a Components section that lists Kafka 3.9.0, Confluent Schema Registry 7.7.1, Stream Reactor Connectors 9.0.2 and Debezium Connectors 2.7.4. A reader who checks those versions before running the image would take those as the current state.

The Dockerfile tells a different story. The ARG KAFKA_VERSION is set to 4.3.1, and a comment directly above it reads: "KRaft-only, no ZooKeeper since 4.0." Kafka 4.x is a major version change. Removing ZooKeeper is not a patch-level update; the metadata storage model changed entirely. The Schema Registry ARG also shows version 8.3.1-lkd-r0, not the 7.7.1 in the README.

The README anticipates this inconsistency and provides a way to check the truth: `docker run --rm lensesio/fast-data-dev:latest cat /build.info` prints the versions that are actually in the image. That command is the only reliable source for what a given image tag contains. The Components section in the README is a point-in-time snapshot, not a guarantee that every version string there matches the image on Docker Hub.

## KRaft mode and no ZooKeeper: what changed at version 4.0

ZooKeeper was a separate process that Kafka depended on for cluster coordination until Kafka 3.x. Starting from Kafka 4.0, the KRaft consensus protocol handles that role internally, and ZooKeeper is no longer part of the setup. The Dockerfile comment documents this: "KRaft-only, no ZooKeeper since 4.0."

For fast-data-dev users, this means any documentation or scripts that describe a ZooKeeper connection string are no longer applicable to the current image. Configurations pointing at a ZooKeeper host and port will not find a listener. The broker configuration file is at /var/run/broker/server.properties inside the container, and services can be restarted with `supervisorctl restart broker` or `supervisorctl restart schema-registry`.

The KRaft Kafka Broker has JMX available at localhost:9581 and Kafka Connect has JMX at localhost:9584. Those ports are exposed by the services running inside the container, so a JMX monitoring client on the host can connect to them through the standard port mapping.

## TLS setup generates certificates automatically; the keystore password is always fastdata

TLS is an opt-in feature controlled by the ENABLE_SSL=1 environment variable. When enabled, the container generates self-signed certificates and makes them available at http://localhost:3030/certs/. Additional hostnames or IP addresses that need to appear in the certificate are added with SSL_EXTRA_HOSTS:

```bash
docker run --rm -p 9092:9092 -p 8081:8081 -p 3030:3030 -p 8083:8083 \
  -e ENABLE_SSL=1 \
  -e SSL_EXTRA_HOSTS=myhost.example.com,192.168.1.100 \
  lensesio/fast-data-dev
```

The keystore and truststore password is always set to fastdata. This is fixed by the image, not configurable. In a development environment on a private network, the fixed password is a minor convenience. On a shared or cloud environment where the TLS endpoint is accessible to others, the predictable password means certificate protection alone is not sufficient access control.

## Adding your own connectors, custom heap sizes, and the Lenses CE companion option

Custom connector JARs can be added at runtime by mounting them under /connectors. Specifying CONNECTORS to a name that matches the mounted JAR's directory loads only that connector, which the README shows as a connector development pattern:

```bash
docker run --rm --net=host \
  -e CONNECTORS=file \
  -e CONNECT_HEAP=2G \
  -v /path/to/custom/connector.jar:/connectors/connector.jar \
  lensesio/fast-data-dev
```

The CONNECT_HEAP variable sets the Java heap for the Connect JVM. Two gigabytes is the value shown in the example, and larger connector sets or high-throughput pipelines may need more.

For teams that want SQL-based access to Kafka topics, the project's own recommendation is to pair fast-data-dev with Lenses CE (community edition) using a provided docker-compose configuration at lenses.io/community-edition/. Lenses CE adds a SQL Studio for running queries on Kafka topics and SQL Processors for streaming SQL. A standalone Kafka deployment tool with no SQL layer is the alternative if Lenses CE is not needed. The project is Apache-2.0 and the last push was on 2026-09-10.

## Conclusion

Use fast-data-dev when you need a self-contained Kafka environment for local development or CI without configuring each component separately. Avoid it in production: it runs all components in one container by design, which makes resource allocation opaque and recovery from single-component failures non-trivial. Before running, check /build.info with `docker run --rm lensesio/fast-data-dev:latest cat /build.info` to see the actual component versions in the image you pulled, since the README's component list and the Dockerfile may not match the image on Docker Hub.

## FAQ

### What is fast-data-dev?

fast-data-dev is a Docker image that packages Apache Kafka, Confluent Schema Registry, Kafka Connect and Stream Reactor connectors into a single container for development and testing. It starts with one docker run command and exposes a web interface at port 3030.

### How do I start fast-data-dev?

Map four ports (9092, 8081, 8083 and 3030) and run the lensesio/fast-data-dev image. The Kafka Broker is then at localhost:9092, Schema Registry at http://localhost:8081, and the web interface at http://localhost:3030.

### Does fast-data-dev require ZooKeeper?

No. The Dockerfile notes that Kafka 4.0 moved to KRaft-only mode, removing the ZooKeeper dependency. Any setup or documentation that references a ZooKeeper connection does not apply to the current image.

### How do I see which component versions are in the fast-data-dev image?

Run `docker run --rm lensesio/fast-data-dev:latest cat /build.info` to print the versions actually in the image. The Components section in the README may not match the current image on Docker Hub.

### How do I disable specific connectors in fast-data-dev?

Set the DISABLE environment variable to a comma-separated list of connector names, for example DISABLE=debezium-mongodb,debezium-mysql. Use CONNECTORS to enable only specific connectors and skip the rest.

## Sources

- [Issues](https://github.com/lensesio/fast-data-dev/issues)
- [lensesio/fast-data-dev on GitHub](https://github.com/lensesio/fast-data-dev)
- [License: Apache-2.0](https://github.com/lensesio/fast-data-dev/blob/fdd/main/LICENSE)
- [Project website](https://lenses.io)
- [README](https://github.com/lensesio/fast-data-dev/blob/fdd/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/lensesio-fast-data-dev