PostgreSQL — Replication Slots, WAL & Recycling

Everything below revolves around one structure: the WAL (Write-Ahead Log). Replication — physical replicas, logical CDC (Datastream, Debezium), backups — is just shipping the WAL stream to consumers and having them replay it. Slots, WAL senders and the four config parameters all exist to answer one question: "which WAL is still needed, and who may delete it?"

There is an interactive, step-by-step animation of both scenarios below: 🖌 Replication Slots — Lifecycle & Recycling (animation)

1. WAL, segments and LSNs

Before Postgres changes any data page it first appends a WAL record describing the change. The WAL is one append-only byte stream per instance (cluster-wide, shared by every database in that instance).

Segments

WAL is stored on disk in pg_wal/ as fixed-size segment files of 16 MB (the default). When one fills, Postgres rolls over to the next.

pg_wal/  (one shared stream, split into 16 MB files)
┌──────────────┬──────────────┬──────────────┬──────────────┐
│  Segment 0   │  Segment 1   │  Segment 2   │  Segment 3    …
│  16 MB       │  16 MB       │  16 MB       │  (being written)
└──────────────┴──────────────┴──────────────┴──────────────┘

Segment file names are 24 hex chars: timeline(8) + high(8) + segment#(8), e.g. 000000010000000000000001.

LSN (Log Sequence Number)

An LSN is a byte address into the WAL stream — "how many bytes of WAL have been written since the beginning." Printed as two 32-bit halves in hexadecimal:

0 / 1C00
▲   ▲
│   └── low 32 bits  (hex)
└────── high 32 bits (hex)      → a 64-bit byte offset, split & printed in hex

Key facts:

• An LSN is a record's position, not a tag attached to it. A WAL record's LSN is simply the byte offset where it begins. The next LSN = this LSN + this record's length.
• Because record sizes vary (a tiny commit vs. a full-page image), real LSNs are irregularly spaced — 0/16D3A20 → 0/16D8F08 → ….
• The hex can look odd but is perfectly regular. Counting by 0x400: 0x1000 → 0x1400 → 0x1800 → 0x1C00 → 0x2000 (after …8, 9 come A, B, C; 0x1C00 + 0x400 = 0x2000 rolls the carry). 1C00 is just 1800 + 1024 in hex.
• LSNs are not 1:1 with segments. A segment is 16 MB = 0x1000000 bytes, so one segment spans ~16 million consecutive LSNs. An LSN points at a byte that lives inside some segment file. Postgres only ever deletes whole segments.

SELECT pg_current_wal_lsn();            -- the write head (next write position)
SELECT pg_walfile_name('0/2800');       -- 0000...0000  (segment byte 0x2800 lives in)
SELECT pg_walfile_name('0/1000000');    -- 0000...0001  (0x1000000 = 16 MB = seg #1)

The write head (pg_current_wal_lsn()) is where the server will write next. It is driven by write activity on the primary, not by any consumer — consumers always trail behind it and can only catch up.

2. The four configuration parameters

• wal_level — minimal (crash recovery only) → replica (default; physical replicas + PITR) → logical (adds metadata for row-level decoding). CDC / Datastream requires logical. On Cloud SQL this is the cloudsql.logical_decoding flag; on Azure Flexible Server wal_level / azure.replication_support.
• max_replication_slots — how many bookmarks may exist. One per consumer.
• max_wal_senders — how many WAL sender processes may run at once. Set ≥ max_replication_slots plus headroom for pg_basebackup/reconnects.
• max_slot_wal_keep_size — a size (bytes/MB/GB), internally divided by the 16 MB segment size → "how many segments a slot may hold behind the head." Default -1 = unlimited (the dangerous default that lets a slot fill the disk).

3. Replication slot

A slot is a named, durable, on-disk bookmark tracking how far a specific consumer has read the WAL — and, crucially, it prevents Postgres from deleting WAL the consumer still needs. It creates back-pressure that releases as the consumer confirms progress.

• Physical slot — for streaming standbys / pg_basebackup / pg_receivewal; raw WAL bytes, no plugin.
• Logical slot — for logical replication / CDC; tied to an output plugin (pgoutput, wal2json) that decodes WAL into row-level changes.

restart_lsn

The slot's core field: the oldest LSN this slot still needs ("resume from here").

• Logically: Postgres guarantees all WAL from restart_lsn → head is retained.
• Physically: a byte-pointer that lands inside one 16 MB segment; Postgres keeps every segment from that one onward.

SELECT slot_name, slot_type, active, active_pid, restart_lsn, wal_status,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained
FROM pg_replication_slots;

Slot lifecycle & identity

• Created explicitly (by the consumer's tooling or manually) — not just by a client connecting. CREATE SUBSCRIPTION, Datastream setup, or pg_create_logical_replication_slot('s','pgoutput').
• Durable & persistent — survives disconnects, crashes, and server restarts. Stored on disk in pg_replication_slots/.
• Consumer gone ≠ slot gone. An orphaned slot with active=false keeps pinning WAL forever → the #1 way to fill the disk. It only disappears when something explicitly drops it (pg_drop_replication_slot, DROP SUBSCRIPTION).
• 1:1 at a time — at most one active streamer per slot; a second connection is rejected. Identity = the unique slot name.
• Reuse = reconnect — the intended behaviour: a consumer reboots and resumes exactly at restart_lsn, zero data lost. This durability is the whole point. Repurposing a slot for a different consumer is usually wrong — drop & recreate.

CREATE (explicit)
   │
   ▼
INACTIVE ◄────────────┐   (pins WAL the entire time it exists)
   │ connect          │ disconnect / crash
   ▼                  │
ACTIVE ───────────────┘
   │  drop / DROP SUBSCRIPTION / tool teardown
   ▼
GONE   (only now does it stop pinning WAL)
   ── or ──
INVALIDATED ("lost")   exceeded max_slot_wal_keep_size → server killed it to save disk

4. WAL sender

A WAL sender (walsender) is a backend process that streams WAL to one connected consumer. One connected replica/CDC client = one walsender.

• Slot = the persistent bookmark (on disk, survives disconnects).
• WAL sender = the live process (exists only while the consumer is connected).

SELECT * FROM pg_stat_replication;   -- one row per live walsender

5. Consumers of the WAL stream

A consumer = any client that connects to a walsender to receive WAL, whether it replays it, archives it, or decodes it:

• Physical (physical slots / replica level): streaming read replicas / hot standbys, pg_basebackup, pg_receivewal, backup/DR tools (Barman, pgBackRest, WAL-G).
• Logical (logical slots / logical level): CDC platforms (Datastream, Debezium, Fivetran, Airbyte), native Postgres SUBSCRIPTION (built-in pub/sub, common for zero-downtime major upgrades), custom apps via pg_recvlogical.

6. One WAL, many markers

There is one shared WAL stream; each slot is an independent marker (restart_lsn) pointing into it.

ONE WAL stream (append-only, grows right):

  ...──[seg N]──[seg N+1]──[seg N+2]──[seg N+3]──► (write head)
          ▲                    ▲            ▲
     slot: backup_wal    slot: replica  slot: datastream
     (lagging, oldest)   (mid)          (caught up)

Consequences:

• WAL cannot be recycled past the oldest marker. One slow/dead consumer holds WAL for the whole instance.
• The recycle floor = MIN(restart_lsn) over all valid slots (plus other constraints below).

7. When does Postgres recycle WAL?

Recycling is always on, checkpoint-driven, and never needs a DBA to "flush." This corrects a common misconception:

• WAL is produced by every write and auto-recycled at checkpoints (triggered by checkpoint_timeout, approaching max_wal_size, or a manual CHECKPOINT), and at restart. It is not continuous and not "the instant a threshold is crossed."
• At each checkpoint Postgres computes the oldest WAL anyone still needs:

oldest-needed-LSN = MIN(
    crash-recovery requirement,     ← always
    wal_keep_size floor,            ← if set
    un-archived WAL,                ← if archive_mode = on
    every slot's restart_lsn        ← the slot constraint
)
→ recycle every segment older than that.

• Without slots (and no failed archiver), WAL is automatically bounded — Postgres keeps only what crash recovery needs between checkpoints (~max_wal_size) and recycles the rest by itself, forever. No human involvement.
• Unbounded growth comes from a pinning constraint that never advances: an inactive/lagging slot, or archive_mode=on with a failing archive_command. Slots don't enable recycling; they hold it back.

A single WAL record may be larger than the space left in a segment, so a record can span a segment boundary. Also, Postgres usually recycles (renames) an old segment file rather than creating a fresh one — a perf detail, functionally still "a new 16 MB segment begins."

8. max_slot_wal_keep_size & the wal_status states

max_slot_wal_keep_size is the per-slot safety valve. The single instance-wide value is applied per slot: no single slot may hold WAL more than this far behind the head. It does not cap total pg_wal/ size and is unrelated to max_wal_size.

At checkpoint, if honouring a slot's restart_lsn would exceed the cap, Postgres removes the WAL anyway and invalidates that slot (wal_status = lost, restart_lsn = NULL). Only the offending slot is sacrificed — healthy slots are untouched, and WAL recycles up to the next-oldest valid slot.

-- early-warning: how many bytes you can still write before this slot is invalidated
SELECT slot_name, wal_status, safe_wal_size FROM pg_replication_slots;

Who decides cleanup:

• DBA — explicit: pg_drop_replication_slot() / DROP SUBSCRIPTION. The normal, intentional end of a slot's life.
• Postgres server — automatic, at checkpoint: invalidation when a slot exceeds max_slot_wal_keep_size. The server chooses the disk over the slot.

The trade max_slot_wal_keep_size buys you: "lose a slot (force a resync)" over "fill the disk and crash the primary." For a production primary, bounding it is almost always right — the resync is recoverable, a full disk is an outage.

TL;DR mental model

LSNs are byte-addresses into one ever-growing WAL stream (printed in hex, hence 1C00). The head is where the server writes next, driven by writes — not consumers. Each slot's restart_lsn is a byte-pointer to the oldest WAL that consumer still needs, living inside some 16 MB segment. Postgres auto-recycles WAL at every checkpoint by deleting segments older than the minimum of all constraints — this runs with or without slots and never needs a DBA. Slots add a retention constraint; max_slot_wal_keep_size caps how far it can drag retention before Postgres invalidates the slot instead of filling the disk.

See also

• 🖌 Replication Slots — Lifecycle & Recycling (animation)
• Datastream: Serverless CDC
• Postgres — Buffer I/O