Change Data Capture

Pradeep P
3 days ago
3 min read

Series: Modern System Design · Layer 4 — Data

Layer 4 · Post 11 of 11

← Previous: Event Sourcing → Next: Containers

Layer 4 — Data · Post 50 of 88

Change Data Capture streams database writes as events so other systems can react without polling or dual-writing.

What you'll learn

How the WAL / binlog becomes a stream (Debezium, DynamoDB Streams, logical replication)
Why CDC beats dual writes for "DB plus Kafka plus search"
Ordering, deletes, schema change, and the exactly-once story you will be asked about

The idea in one minute

Change Data Capture (CDC) watches the database's own log and emits row-level events: this order was inserted, this quantity went from 5 to 4, this user was deleted.

You already paid for durability in ACID (the WAL). CDC reuses that log so other systems see the same commits, in commit order, without a second write in app code.

App --> BEGIN/COMMIT --> Postgres WAL | CDC connector | v Kafka --> search, cache, warehouse, saga consumers

Local ACID stays the source of truth. Eventual copies — replicas, CQRS views, warehouses — catch up from the log.

Why it matters

Dual write (db.save(); kafka.send()) is the bug posts 45–47 exist to avoid. An outbox table is a manual CDC: same transaction, later publisher. Real CDC reads the WAL so application code stays boring.

Interview designs that say "service A notifies B" should mention CDC or outbox, not polling updated_at.

How it works

The engine writes changes to WAL (Postgres), binlog (MySQL), or a change stream (DynamoDB Streams, MongoDB change streams, SQL Server CDC).
A connector (Debezium is the usual Kafka one) tails that log, decodes row images, and publishes to a topic per table (or a filtered subset).
Consumers project (CQRS), index (Elasticsearch), ETL (Snowflake), or trigger workflows. They use the primary key + log offset (or LSN) for idempotency.
Snapshots: first run often dumps current table state, then switches to streaming. Deletes and PK updates are first-class; your consumer must handle tombstones.

You get at-least-once delivery almost always. Exactly-once is "idempotent consumer + offset tracking," not a magic Kafka flag you set and forget.

Schema evolution: add a column, consumers must tolerate it. Debezium can include schema in the message (Avro/Registry). Breaking changes still break projections.

A simple example

Orders live in Postgres. You need Elasticsearch for search and a warehouse for finance. Debezium streams orders and order_items to Kafka. One consumer updates Elastic; another loads Snowflake. Checkout code only COMMITs to Postgres. If Kafka is down, the WAL retains (until you fill the disk — slot lag is a pager, not a footnote). When Kafka returns, CDC continues. No missing "we charged but did not publish."

DynamoDB Streams plus Lambda is the same idea on AWS: cache invalidation, not a second PutItem in the app.

Common mistakes

Unreplicated slot / binlog consumer down, disk fills, primary dies. CDC is a production dependency. Monitor LSN lag.

Treating CDC events as domain events. A qty update is not InventoryReserved. You can derive domain events, but a raw row change is a poor saga API.

No handling for deletes or PK changes. Search still shows the user.

Using CDC as a two-way sync between two writers. You just built a conflict machine. One source of truth.

How this shows up in real systems

Debezium + Kafka on Postgres/MySQL/SQL Server/MongoDB.
Postgres logical replication (even without Kafka) to followers and warehouses.
DynamoDB Streams, Cosmos DB change feed, MongoDB change streams.
Fivetran / Airbyte / Datastream: CDC into analytics.
Outbox pattern: CDC-of-a-table you control when you do not want to stream every row.

Recap

CDC turns the database log into a stream so copies stay eventual, not dual-written.
Layer 4's path: local ACID, then isolation and locks; across systems, 2PC or sagas; CQRS and event sourcing for models; CDC feeds copies without dual writes.
Infrastructure is next: how you package and run all of this.