Karapace: an open source route to Schema Registry and Kafka REST

Two Confluent services, one Python codebase

Karapace is an open source implementation of Kafka REST and Schema Registry, and its own summary is that it provides your Apache Kafka essentials in one tool.

The scope has two halves, and the README describes each separately. The registry side supports the storing of schemas in a central repository, which clients can access to serialize and deserialize messages, with the schemas maintaining their own version histories and being checked for compatibility between their respective versions. The REST side provides a RESTful interface to your Apache Kafka cluster, allowing you to produce and consume messages and perform administrative cluster work, all the while using the language of the web.

What makes this a credible replacement rather than a toy is the claim in the feature list that it is a drop-in replacement on both pre-existing Schema Registry and Kafka Rest Proxy client and server sides. That is a strong claim, and the README qualifies it in the compatibility section: Karapace is compatible with Schema Registry 6.1.1 on API level, the goal is to support a new Schema Registry version when it is released in a reasonable time, and Karapace supports all operations in the API.

The two caveats it names are the ones to keep in mind. There are caveats regarding schema normalization, and the error messages being the same as in Schema Registry cannot always be fully guaranteed. Both of those matter in practice. Error message text is often what a client library matches on, and normalization affects whether two schemas that differ only in formatting are treated as the same.

The project is Apache-2.0 licensed, the homepage is karapace.io, and it is developed under the Aiven-Open organisation. The repository is not archived and the last push was on 2026-10-13, with recent releases 6.2.2 on 2026-08-05, 6.2.3 on 2026-09-15 and 6.2.4 on 2026-10-02.

Two processes on two ports, not one binary

The architecture is visible in the way the project asks you to run it, and it surprises people who expect a single service.

The registry and the REST proxy are separate entry points. The registry starts with python -m karapace and listens on http://localhost:8081, while the REST proxy starts with python -m karapace.kafka_rest_apis and listens on http://localhost:8082. The README is explicit about both, and both are shown as separate commands.

The implementation choices underneath explain the split. The feature list says the architecture is asynchronous and based on aiohttp, the schema registry is based on FastApi, and the Kafka side is built mostly on the aiokafka library, with rdkafka also mentioned. So the registry is a FastAPI application and the REST proxy is an aiohttp application, which is why they are two processes with two Python entry points rather than one ASGI app with two route groups.

The observable behaviour is straightforward. The registry exposes the subjects endpoint, and the README says that opening http://localhost:8081/subjects in a browser should return an array of subjects if any exist or an empty array. The equivalent check with curl is:

  $ curl -X GET http://localhost:8081/subjects

The REST proxy has its own check, listing topics:

  $ curl "http://localhost:8082/topics"

Those two endpoints are the fastest way to confirm a deployment works, and they are also the two that tell you which of the services is misconfigured when something is wrong.

The image comes from ghcr.io, and older tags live in another registry

The Docker path is the one most people will take, and the README is specific about the tags.

There are two images in the aiven-open registry. The develop tag tracks the latest build from the main branch, and the latest tag is the latest release:

  # Fetch the latest build from main branch
  docker pull ghcr.io/aiven-open/karapace:develop

  # Fetch the latest release
  docker pull ghcr.io/aiven-open/karapace:latest

And there is a third location, which is the kind of detail that saves an afternoon. Versions 3.7.1 and earlier are available from the ghcr.io/aiven registry:

  docker pull ghcr.io/aiven/karapace:3.7.1

So if you are pinning an old release, the obvious pull will not find it.

For a full setup with configuration and the Kafka connection, there is a compose example in the container directory:

    docker compose -f ./container/compose.yml up -d

And the documentation notes that for local development with Docker there are convenient make commands, which points at the GNUmakefile at the repository root.

The container layout shows the two-service split again, since the compose file is what wires the registry, the REST proxy, Kafka and the configuration together. Configuration lives in karapace.config.json in the repository, with a separate rest-proxy-karapace.config.json for the proxy, and every configuration key can be overridden with an environment variable prefixed with KARAPACE_, except for keys that already start with the karapace string. The example given is that the bootstrap_uri config value becomes the KARAPACE_BOOTSTRAP_URI environment variable.

OIDC authentication is layered, and the audience check is the one to set

The authentication section is the most detailed part of the documentation, and it is worth reading closely because the defaults are permissive in a way you may not expect.

Authentication is enabled with the sasl_oauthbearer_authentication_enabled flag, alongside the JWKS endpoint, the expected issuer, the expected audience and the subject claim name:

   sasl_oauthbearer_authentication_enabled: bool = True
   sasl_oauthbearer_jwks_endpoint_url = "https://idp.example.com/realms/karapace/protocol/openid-connect/certs",
   sasl_oauthbearer_expected_issuer = "https://idp.example.com/realms/karapace",
   sasl_oauthbearer_expected_audience = "karapace-audience",
   sasl_oauthbearer_sub_claim_name = "sub",

When authentication is enabled, every request must carry a valid Bearer token, and the JWT signature, expiry and the configured subject claim are always verified via JWKS. The issuer and audience checks are optional, following the same pattern as Kafka's sasl.oauthbearer.expected.* settings, so leaving one unset skips that claim check.

The README then states the consequence plainly: leaving the audience unset means any token the provider issues for any client is accepted, so set it whenever the provider can mint a usable aud. That is the single most important sentence in the section, because a token minted for a different service in the same identity provider would otherwise pass verification.

Authorization is opt-in on top of that, with sasl_oauthbearer_authorization_enabled, a roles claim path and a method to roles mapping. Enabling authorization without explicitly enabling authentication will auto-enable authentication and log a deprecation warning, and the recommendation is to set both flags explicitly.

There are also two hardening flags, for clock skew tolerance and for requiring the at+jwt token type header:

   sasl_oauthbearer_leeway_seconds: int = 0           # clock-skew tolerance for exp/nbf/iat (must be >= 0)
   sasl_oauthbearer_require_access_token_typ: bool = False  # require typ "at+jwt" header

The REST proxy has its own SASL configuration, with the mechanism set to OAUTHBEARER or PLAIN, the security protocol, and the token provider or the username and password:

   sasl_mechanism = "OAUTHBEARER",
   sasl_oauth_token_provider = token_provider,
   security_protocol="SASL_SSL",
   ssl_cafile="ca.pem",

    sasl_mechanism = "PLAIN",
    security_protocol = "SASL_PLAINTEXT",
    sasl_plain_username = "your_username",
    sasl_plain_password = "your_password"

The repository also ships an example token provider, examples/msk_iam_token_provider.py, which is the shape to copy if you are authenticating against a cloud identity provider that mints tokens on demand.

A source install needs Go, and the Avro dependency is patched

The source install path is short but its prerequisites are not trivial, and the README lists them without dressing them up.

The install command is pip install ., and the troubleshooting notes say that an updated version of wheel is required, an updated version of go and rust is required, and you should create and activate a virtual environment to manage dependencies.

Go and Rust are needed because the package builds a native extension. The setup.py file at the repository root shows exactly what is compiled, and it is a Go extension named protopacelib built from go/protopace/main.go with a build root of go/protopace.

Rust is presumably needed for one of the bundled dependencies, and the patched Avro dependency is where the interesting part is. The pyproject manifest lists Avro as a direct URL reference to a commit in a fork:

  "avro @ https://github.com/aiven/avro/archive/5a82d57f2a650fd87c819a30e433f1abb2c76ca2.tar.gz#subdirectory=lang/py",

The comment above it explains why, and it is a good reason. Referencing patched dependencies by commit has two advantages, reproducible builds, and the contents of the file change, which invalidates the existing docker images and forces a new image generation.

That second point is a deliberate design choice about supply chain and image freshness, and it is worth understanding before you pin a build. The runtime dependencies are otherwise a fairly standard data stack: aiohttp and FastAPI for the two servers, aiokafka and confluent-kafka for Kafka access, jsonschema, protobuf, pydantic, cryptography, pyjwt for the schema work and token handling, plus the compression libraries lz4, snappy and zstandard, orjson and ujson, and the OpenTelemetry and Prometheus packages for the observability the feature list mentions.

The Python floor is 3.12 with an upper bound of 3.15, and the manifest also carries a runtime.txt at the repository root. The project is classified as Development Status 5, Production/Stable.

Where Karapace is the wrong tool

The first wrong case is an application that just needs to talk to Kafka over HTTP. Karapace is two services with their own configuration, their own authentication model, and their own upgrade cadence, and you are deploying all of it to avoid a client library. For a service that needs to produce and consume a handful of topics, a Kafka client is simpler and has fewer moving parts.

The second wrong case is a cluster whose Schema Registry version has moved ahead of the project. The compatibility statement is specific, 6.1.1 on API level, with a stated goal of supporting a new version in a reasonable time. The release notes show that the project is moving, since 6.2.3 is described as SR Oidc and Rest proxy headers and 6.2.4 as IMPORT mode with schema migration, so features are being added, but a client library pinned to a newer Schema Registry API may not work.

The third wrong case is a team that needs the Confluent-specific behaviour exactly. The README is upfront that schema normalization has caveats and that error messages cannot always be fully guaranteed to match. If your client library matches on error strings, or if your schema registry has historically accepted a particular non-normalized form, that is a compatibility risk the project names rather than hides.

The alternative worth naming is the vendor's own Schema Registry, or a hosted schema service, and the difference in approach is the licensing and the operational model rather than the API. Confluent Schema Registry is the reference implementation that Karapace is measured against, and using it means the compatibility question disappears because there is nothing to be compatible with. Karapace's case is that you get the same API surface under the Apache 2.0 licence, with the option to self host the whole thing, at the price of tracking a community implementation's release line and reading its caveats. For a team that already runs Kafka and wants to drop the commercial registry, that trade is usually worth it. For a team with one application and no infrastructure appetite, it usually is not.

Maintenance state, licensing and the cost of the 6.x line

The repository is not archived, and the last push was on 2026-10-13, with 6.2.4 tagged on 2026-10-02, 6.2.3 on 2026-09-15 and 6.2.2 on 2026-08-05. Three releases in ten weeks is the pattern of a project that ships against a moving target, which in this case is the Schema Registry API.

The licence is Apache 2.0, with the LICENSE file at the repository root, and the project is developed under the Aiven-Open organisation, so the code is available under terms that allow commercial use and modification. What a company gets in exchange is the engineering work of Aiven, and the repository is explicit about the commercial relationship in the Aiven-Open naming rather than hiding it.

The upgrade cost is the thing to plan for. Because the compatibility target moves with the Schema Registry version, and because the release notes carry substantive changes, a team on the 6.x line should expect to read release notes before each bump rather than treat it as a routine dependency update. The 6.2.3 note about SR Oidc and rest proxy headers is a good example, since it touches both servers at once.

The project also carries the full set of engineering scaffolding that a long lived service needs. There is a GNUmakefile, a pre-commit configuration, mypy.ini, pytest.ini, a .coveragerc, an .isort.cfg, a copyright.sh script, a benchmarks directory, a performance-test directory, a runtime directory, a stubs directory, a website directory, a SECURITY.md, a CODE_OF_CONDUCT.md, a CLAUDE.md and an .env.example. The presence of benchmarks and a separate performance-test directory is a good sign for a project whose value proposition is partly about efficiency, since the feature list claims moderate memory consumption and observability with metrics and OpenTelemetry.

The .env.example file is worth a look before you deploy, because it shows the shape of the deployment environment: the runner UID and GID for the container user, the Python version, the Karapace version, whether the compose file is named docker-compose, and the pytest arguments used for coverage with multiple processes and a log file. That last one tells you the test suite is run in parallel with coverage appended, which is how a project of this size keeps its own test cycle manageable.

Editorial conclusion

Karapace fits a team that needs Confluent-compatible Schema Registry and REST proxy behaviour, especially on Kafka clusters where schemas already live in the cluster. It is the wrong choice for a single application that only needs to produce and consume JSON, since running two services to speak HTTP to a broker is not a small operational cost, and it is the wrong choice if you need the newer Schema Registry features the project has not matched yet. Before you adopt it, check the compatibility statement in the README, confirm the 6.x release line matches the Schema Registry version your clients expect, and read the patched Avro dependency note in pyproject.toml before you pin a build.

Frequently asked questions

Official sources

1. Aiven-Open/karapace on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases