pgstream: Postgres logical replication that also carries DDL

The gap pgstream tries to close in Postgres change data capture

Logical replication in Postgres carries row changes. It does not carry the ALTER TABLE statements that reshape the tables those rows live in, so a downstream index drifts out of step with the source schema and the operator is left fixing mappings by hand. pgstream exists to close that gap. The README describes it as an open source CDC command line tool and library that offers Postgres replication support with DDL changes to any provided target.

That framing matters for the audience. The tool is aimed at teams that keep a search index, an analytics feed or an event stream fed by a Postgres source, and that have been harmed by schema drift between the two. The feature list names the targets it ships with, Elasticsearch and OpenSearch, webhooks and PostgreSQL, along with initial and on demand snapshots for teams that want a copy rather than a continuous stream.

The deployment story is deliberately narrow. The feature list says the configuration is modular and only requires Postgres, which is a way of saying you do not need to run a separate broker or coordinator unless you ask for one. Kafka appears as an optional target rather than a requirement.

The library angle matters too. pgstream is usable through its CLI or imported as a Go module, so a team that already has a Go data pipeline can wire the replication logic into existing code rather than running a separate daemon. The Go module path is github.com/xataio/pgstream, and the CLI is built from the cmd directory.

How the schema change tracking works

The mechanism is visible in the initialisation step rather than in the feature list. Passing the --init flag to the run command initialises the pgstream state inside the source Postgres database, and the README lists three things that happen as a result: a pgstream schema is created, tables, functions and triggers are created to keep track of schema changes for DDL replication, and a replication slot is created.

That ordering explains the architecture. The slot is what makes row level logical decoding work. The triggers and functions in the source are what make the tool aware that a DDL statement happened, because the Postgres write ahead log alone records data, not the intent of a schema edit. The tracking tables give the tool a place to record what it has already seen and applied, which is what allows it to keep the target aligned and to resume without replaying everything.

Initialisation is not optional. The README states that initialisation is required for pgstream replication, and offers a separate pgstream init command as an alternative to the --init flag when you want to prepare the source ahead of time. The architecture document referenced from the README covers tracking schema changes in more detail, and the internal package layout visible in the repository, including pkg/stream/preflight and pkg/wal/processor/postgres, suggests a preflight stage that checks the source can support the run before the stream starts.

The practical consequence for an adopter is that pgstream is not a read only observer of your database. It writes into it.

Installing the pgstream CLI and running a first replication

The README gives several installation paths, and the release binary is the shortest. On a Linux machine you can fetch the latest build, make it executable and place it on the path.

curl -L https://github.com/xataio/pgstream/releases/latest/download/pgstream.linux.amd64 -o pgstream
chmod +x pgstream
sudo mv pgstream /usr/local/bin/

Go developers can install the same tool with the Go toolchain, and there is a Homebrew tap for macOS and Linux. Building from source is the fourth route.

go install github.com/xataio/pgstream@latest
brew tap xataio/pgstream
brew install pgstream

If you want a local environment to point at, the repository ships a Docker Compose setup with profiles that selectively start Postgres, Kafka and OpenSearch. The pg2pg profile is enough for Postgres to Postgres replication.

docker-compose -f build/docker/docker-compose.yml --profile pg2pg up

The first real run targets OpenSearch from a local Postgres. You can drive it from an environment file, a YAML file or plain flags, and the flags form is the easiest one to read while you are learning the tool.

pgstream run --source postgres --source-url "postgres://postgres:postgres@localhost:5432?sslmode=disable" --target opensearch --target-url "http://admin:admin@localhost:9200" --init

You should see the tool create the pgstream schema in the source database, install the tracking objects, create the replication slot and then begin streaming rows. The README also shows the same run driven by the shipped example configuration files, pg2os.env and pg2os.yaml, with a --log-level flag that accepts trace for development and info for normal use.

pgstream run -c docs/examples/pg2os.env --init --log-level trace
pgstream run -c docs/examples/pg2os.yaml --init --log-level info

The --init flag is what a first run needs. Leaving it off on a source that was never initialised is the most likely way to get a run that starts and immediately has nothing to work with.

Column transformers and anonymisation in the stream path

The feature list advertises column value transformations with the parenthesis that you can anonymise your data on the go, and the configuration documentation points to a transformation rules file and a list of available transformers.

The integrations are named rather than described, which is worth reading carefully. Transformers can be provided through greenmask, neosync and go-masker, and custom transformers are supported on top of those. In other words, masking a column is not a hardcoded feature of pgstream; it is a plugin mechanism with three external libraries plugged into it by default. A team that already depends on one of them can reuse its rules rather than rewriting them.

The placement of the transformer in the pipeline is what makes this more than a post-processing step. Transformations are applied to row values as they are processed, before the record is handed to the target, so masked values are what lands in the index and not just what a reader sees. For a search index that holds customer records, that ordering is the difference between a working anonymisation policy and a policy that leaks through a replica.

The repository layout backs this up. pkg/wal/processor/transformer appears in the integration test target list in the Makefile, alongside pkg/wal/processor/postgres, which suggests a processor chain where decoding and transformation are separate stages feeding the same downstream.

Kafka as an intermediate hop and the snapshot mode

Two features set pgstream apart from a single hop copier. The first is Kafka support with schema based partitioning, and the README shows a two terminal workflow where one process runs Postgres to Kafka and another runs Kafka to OpenSearch. The commands for the first leg are shipped as example configurations too.

pgstream run -c docs/examples/pg2kafka.env --init --log-level trace
pgstream run -c docs/examples/pg2kafka.yaml --init --log-level info

Splitting the pipeline this way decouples capture from indexing. The Postgres side keeps a steady read regardless of how slow the search cluster is, and the Kafka topic becomes a replayable buffer when the target has an outage. Schema based partitioning is the detail to check, because it determines whether related records land on the same partition and preserve their relative order. The README does not describe the partitioning key selection rules, so that is a point to confirm in the configuration documentation before depending on ordering guarantees.

The second feature is snapshots. The README lists initial and on demand PostgreSQL snapshots for when you do not need continuous replication. This is the path for backfilling a search index from scratch or reproducing a dataset for testing, and it means you do not have to run a live stream just to seed a target. The repository contains pkg/snapshot/generator/postgres/data and pkg/snapshot/store/postgres/integration, which indicates a generator that reads source data and a store abstraction for where the snapshot lands.

Where pgstream is the wrong tool

The first limitation is structural and the README is direct about it: initialisation is required, and initialisation creates a schema, functions, triggers and a replication slot in the source database. On a managed Postgres service with strict schema permissions, or on a production database where a DDL trigger on hot tables is unacceptable, this is a blocker rather than a tuning problem. The tool needs write access to the database it reads.

The second limitation is the target list. Elasticsearch, OpenSearch, webhooks, PostgreSQL and Kafka are what the feature list names, with custom targets possible through the extension mechanism. There is no mention of replication into a warehouse column, a cloud storage bucket or a message queue other than Kafka, so a BigQuery or Snowflake destination is not what this project currently provides.

The third limitation is that the stream is a stream, not a log with retention. A replication slot in Postgres holds WAL until the consumer advances. A consumer that stops and is not restarted promptly can fill the disk of the source database. The README does not describe a slot cleanup or heartbeat behaviour, so the operator needs to know what the tool does about an abandoned slot before running it in an environment where disk pressure matters.

The fourth limitation is the source only. The tool is described as Postgres to anything, with no mention of reading from MySQL or SQLite. A team with a mixed database estate standardising on one CDC tool will find that it only covers part of the estate.

pgstream compared with raw logical replication and with Debezium

Against raw Postgres logical replication, the difference is scope. Native logical replication gives you row changes over a publication and subscription pair, and leaves schema and data change tracking entirely to you. pgstream keeps the same underlying machinery, a replication slot and logical decoding, and adds the DDL tracking objects, a set of ready made targets, transformer hooks and snapshot support. If you only need Postgres to Postgres row movement and you are prepared to handle schema changes yourself, plain logical replication is fewer moving parts.

Against Debezium the difference is where the logic lives. Debezium is a change event framework built around a Kafka Connect runtime, so it typically deploys as a service in a Kafka estate and publishes change events that downstream consumers interpret. pgstream is a Go binary that can run as a single process against a source database, and it can target Kafka directly or use it as the middle of a two leg pipeline. The trade-off runs both ways. A Debezium setup inherits the operational surface of Kafka Connect and benefits from a large connector ecosystem. A pgstream setup needs only Postgres for the basic case, but its target list is shorter and its extension model is a Go interface rather than a connector jar.

For a team already running Kafka Connect, the honest comparison is not feature by feature but deployment by deployment: a Connect connector lives in a cluster you already maintain, while pgstream is a process you supervise yourself.

Release cadence, licence and the cost of upgrades

pgstream is licensed under Apache 2.0, and the repository carries a LICENSE file and a license-header-checker target in the Makefile that checks source headers against license-header.txt. The module path carries no vendor suffix. Apache 2.0 permits use in commercial products and internal tooling, and the licence file is the authoritative text on what that means for a given deployment.

The release history in the repository shows steady tagged work: v1.4.2 on 2026-09-02, v1.5.0 on 2026-09-10 and a nightly build on 2026-10-09, which is also the date of the most recent push. That is a project shipping in the 1.x line with tagged releases roughly weekly during that window, and the 1.5.0 tag suggests a versioning scheme that treats minor bumps as meaningful.

Upgrades carry the usual cost of a tool that owns objects in your database. The pgstream schema, its functions and its triggers are created by initialisation, so a major version could change what those objects look like. The CLI documentation section referenced from the README is the place to check whether a given release expects a re-initialisation or a migration step, and the migrations directory at the repository root suggests that at least some state changes are handled by versioned migration files rather than by hand.

The other upgrade cost is the Go toolchain. The module declares go 1.26.6, so building from source needs a matching Go release, while the prebuilt binaries avoid that dependency entirely.

Editorial conclusion

Adopt pgstream when the target is a search index, a webhook or Kafka and the schema moves faster than your pipeline. Skip it when you need point-in-time recovery, multi-writer Postgres targets or full control over the replication slot. Before switching, run pgstream init against a copy of production and confirm the pgstream schema, the triggers and the slot behave as the architecture document describes on your table shapes.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. Project website
3. README
4. Releases
5. xataio/pgstream on GitHub