microsoft/pg_durable: durable execution as a PostgreSQL extension

The problem pg_durable solves, and who is meant to run it

Background work in a Postgres shop usually grows the same way. A cron job inserts rows into a jobs table, a worker polls it, status columns track progress, and retry counters accumulate. The workflow logic ends up spread across SQL, workers, queues, dashboards, and status tables, as the README puts it. A restart in the middle of a long job means rerunning work that already succeeded.

pg_durable's answer is to move the workflow definition into SQL and let PostgreSQL checkpoint each step. The README names three audiences: backend and data engineers who want workflows next to the data they touch, DBAs and SREs automating runbooks that must survive restarts and be auditable in SQL, and teams building data or AI pipelines that need durable execution per row, document, or batch. The listed workloads are concrete: vector embedding pipelines that chunk, call an embedding API, and upsert into pgvector; ingest pipelines that stage, deduplicate, transform, and publish batches; scheduled maintenance that detects bloat, notifies, waits for approval, then acts; fan-out aggregation; and external API workflows such as enrichment and classification.

That list is also the boundary. Every example is a job that runs in the background and whose state belongs in the database. If your work is a synchronous request handler with a sub-millisecond budget, this is the wrong layer.

How a durable SQL function actually executes

The mechanism has four documented stages. You define a workflow in SQL using composable operators such as ~> and |=>. You start it with df.start() and get back an instance ID. The runtime executes each step durably with checkpointing between steps. You query status and results from PostgreSQL while the workflow runs or after it completes.

The README's diagram shows a durable function fanning out into three parallel queries (count users, count orders, sum revenue) that join into a dashboard step. So the graph is not a linear script: independent branches run in parallel and converge. The operators encode that shape, with |=> introducing a named batch and ~> chaining the next step.

Under the hood the extension is a Rust crate built with pgrx 0.16.1, and Cargo.toml pins duroxide 0.1.30 with duroxide-pg 0.1.34 for the execution engine. SQL activities that connect back to PostgreSQL use sqlx, cron expressions for wait_for_schedule come from the cron crate, and df.http uses reqwest with native-tls selected explicitly. That is a real dependency chain: the durable-execution semantics are not written from scratch here, they are wired into Postgres through pgrx and a Postgres-backed store. Cargo.toml also carries a warning worth repeating: the test-hooks feature compiles timing hooks into the background worker and must never be enabled in a shipped build, because the hooks let the process environment stall worker initialization.

Operational visibility is the payoff. The README points at Postgres tables such as df.instances, using the same auth and backup model as your data. That is a different posture from an external orchestrator, where the run history lives in another system with its own access control.

Installing pg_durable and running a first durable function

There are two documented install paths, and both assume PostgreSQL 17 or 18. Tagged releases publish Debian packages for amd64 named pg-durable-postgresql-<PG major>_<pg_durable version>-1_<arch>.deb, which install the extension library, control file, and SQL upgrade files into the matching PostgreSQL directories. Tagged releases also publish a ready-to-run image at ghcr.io/microsoft/pg_durable, built by installing the released Debian package on top of the official postgres image. Tags are immutable and carry the major version, for example 0.2.2-pg17 and 0.2.2-pg18; the floating pg<major> tags track the highest stable release, and pg17 also updates latest.

The repository's docker-compose.yml is the shortest route to a running instance. It maps host port 5433 to container 5432 to avoid colliding with a local server, and it sets the preload and database settings the extension needs.

services:
  postgres:
    image: pg_durable_postgres:latest
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: pg_durable
    ports:
      - "5433:5432"
    command: >
      postgres
      -c shared_preload_libraries=pg_durable
      -c pg_durable.database=pg_durable

Bring it up and connect on the mapped port. If the extension is not preloaded, the background worker does not start, so the preload line is not optional.

docker compose up -d
psql -h localhost -p 5433 -U postgres -d pg_durable

Once connected, the README's quick example is the first real use. It starts a durable function that selects unprocessed documents into a named batch and then marks them processed by reading from that batch.

SELECT df.start(
    'SELECT id FROM documents WHERE processed = false LIMIT 100' |=> 'batch'
    ~> 'UPDATE documents SET processed = true WHERE id IN (SELECT id FROM $batch.*)'
);

The call returns an instance ID. From there, status and results come from Postgres tables such as df.instances. To build from source instead, the Dockerfile shows the intended toolchain: cargo-pgrx 0.16.1, cargo pgrx init against the PostgreSQL 17 pg_config, then cargo pgrx package. Note that the shipped Dockerfile packages with the http-allow-test-domains feature, which is a test-oriented build rather than a production one.

Where the SQL-shaped model breaks down

The README is unusually direct about the ceiling: the model is intentionally SQL-shaped. If a step needs arbitrary code, a non-HTTP SDK, or rich in-memory control flow, you may need to wrap that logic in a SQL function, expose it behind an HTTP endpoint for df.http(), or use a general-purpose orchestrator for that part of the system. Read that as three separate escape hatches, each with a cost. Wrapping logic in a SQL function pushes complexity back into the database. Exposing it over HTTP adds a service you now operate. Splitting the workflow means you have two orchestration systems and a boundary between them.

The README's own not-for list is the sharper test. Do not reach for pg_durable when the job is already a single INSERT ... SELECT or one ordinary SQL statement, when you need sub-millisecond synchronous request handling, when you cannot install extensions or run a background worker, or when the workflow mostly lives outside Postgres and spans many heterogeneous systems. The last one matters most in practice: the moment your steps are mostly calls into systems that are not HTTP-reachable, the SQL graph becomes a thin wrapper around something else.

There is also a build-level caveat. Cargo.toml declares feature flags for PostgreSQL 13 through 18, but the README badges and the packaging section cover only 17 and 18. Treat the older feature flags as build options, not as a support statement. And the http-allow-all feature exists, which the name suggests opens outbound HTTP broadly; the README does not document the allow-list semantics in the excerpt available, so verify the intended production setting yourself.

pg_durable compared with an external orchestrator

The most useful comparison is against the tools the README lists as the current alternative: Airflow, Temporal, Step Functions, and Argo, or the pg_cron-plus-jobs-table pattern. The difference is architectural, not cosmetic.

An external orchestrator runs its own control plane: its own database or store for run history, its own scheduler, its own worker fleet, and its own auth model. It reaches into Postgres to do work, and the run record lives outside the database. pg_durable inverts that. The runtime is a PostgreSQL extension, there is no Redis and no Temporal, and the checkpoint state and run history sit in the same database as the data, queryable through df.instances. Backups, roles, and connection limits apply to the workflow state the same way they apply to your tables. For a team whose only durable store is Postgres, that removes a class of drift between the two systems.

The trade is portability and reach. An external orchestrator can call anything: any SDK, any protocol, any language, with workers that scale independently of the database. pg_durable's steps are SQL, with HTTP as the documented escape hatch. Its execution capacity is also coupled to the PostgreSQL instance, since the background worker runs there. If your workflows are polyglot and span many heterogeneous systems, the orchestrator's extra control plane buys back flexibility that pg_durable deliberately does not offer.

Release cadence, upgrade cost, and the licence question

The repository is not archived, and the last push was on 2026-09-24. Recent releases are v0.2.8 on 2026-09-11, v0.2.7 on 2026-09-01, and v0.2.6 on 2026-08-24, while Cargo.toml declares version 0.2.9 on main. That is a fast, pre-1.0 cadence: roughly a release every one to three weeks across August and September 2026. The practical consequence is that upgrade cost is real and recurring. Under 1.0, minor version bumps can change behaviour, and the extension ships SQL upgrade files precisely because the on-disk objects change between versions. Plan on reading CHANGELOG.md before each bump rather than upgrading blind.

The packaging model helps here. Debian packages are versioned per PostgreSQL major, and container tags are immutable and carry the major version, so you can pin a specific combination instead of tracking latest. The floating pg<major> tags are convenient for evaluation and a poor choice for anything you depend on.

On licensing, the repository ships LICENSE.txt, the README badge reads PostgreSQL License, and Cargo.toml declares license = "PostgreSQL". The repository metadata reports the licence as NOASSERTION, which means the automated classifier did not match it to a known SPDX identifier. That is a metadata gap, not a statement about the terms. The PostgreSQL License is a permissive, BSD-style licence, but this is not legal advice: if your organisation has rules about which licences it accepts, read LICENSE.txt and NOTICE directly and route them through your own review.

Editorial conclusion

Adopt pg_durable if your workflows already live in Postgres, you can install an extension and set shared_preload_libraries, and your logic maps to SQL steps, branching, loops, or df.http() calls. Do not adopt it if the job is one INSERT ... SELECT, if you need sub-millisecond synchronous handling, if you cannot run a background worker, or if the workflow spans many systems that are not reachable over HTTP. Before committing, verify three things: that your PostgreSQL major version is 17 or 18, that the extension loads after a restart with the preload setting in place, and that a workflow you start with df.start() is still queryable in df.instances after you stop and start the server.

Frequently asked questions

Official sources

1. Issues
2. microsoft/pg_durable on GitHub
3. Project website
4. README
5. Releases