Log-based CDC: how capture works
Change data capture reads the transaction log a database already writes. This page explains what each log contains, how the first run establishes a baseline, and where the operational risk actually sits.
Docs · Reference · Updated 2026-11-03 · Applies to current Bridge releases
Definition
Log-based change data capture is the technique of reconstructing inserts, updates and deletes by parsing the source database's native transaction log, using read-only access, instead of issuing repeated queries against user tables.
What each log contains
| Engine | Log | Record granularity | What to configure |
|---|---|---|---|
| Oracle | Redo log | Row change with before/after image when supplemental logging is on | Supplemental logging, archive log retention, LogMiner or XStream access |
| MySQL | binlog | Row image in ROW format | binlog_format=ROW, retention period, replication privileges |
| PostgreSQL | WAL | Logical decoding output | wal_level=logical, replication slot, slot retention monitoring |
| SQL Server | Transaction log | CDC capture instance rows | CDC enabled per table, capture job window |
| KingbaseES | WAL | Logical decoding output | Logical decoding enabled, slot monitoring |
| DM / Dameng | Redo | Native log records | Archive mode and log retention |
The important consequence: because the log carries the row image, capture cost is a function of change volume, not table size. A 2-billion-row table with 1,000 daily changes costs about the same to capture as a 2,000-row table with 1,000 daily changes. Polling designs invert that relationship, which is why they degrade as tables grow.
First run: snapshot, then stream
A cold start needs a consistent baseline. Bridge performs an initial snapshot and records the corresponding log position, so changes that commit during the snapshot are replayed afterwards rather than lost. The sequence is:
- Record the current log position for each captured table.
- Take a consistent snapshot of the table set.
- Begin streaming from the recorded position and apply changes, deduplicating against snapshot rows by key.
- Store the applied offset so a restart resumes from the last committed point rather than re-snapshotting.
A restart therefore resumes, it does not re-copy. That property is what makes long-running pipelines operationally cheap.
Where the risk actually sits
| Risk | Symptom | Control |
|---|---|---|
| Log retention too short | Capture falls behind and the needed log segment is purged | Alert on offset age vs retention window |
| Replication slot not consumed | WAL accumulates on the source | Monitor slot lag; alert on retained bytes |
| Supplemental logging off | Updates arrive without the before-image | Verify source configuration during the POC |
| Per-key reordering | Target reflects a stale update | Preserve key ordering across load threads |
| Backpressure ignored | Offset grows under target throttling | Apply backpressure from loader to reader |
CDC questions
Is capture read-only?
Yes. Capture uses a read-only log reader; it does not create triggers, shadow tables or write anything to the source.
What is the impact on the source?
Primarily log retention and log-read I/O. In typical deployments capture lag is 40–60s at P95 with no measurable change to source query latency.
Does it capture deletes?
Yes. The log records deletes as well as inserts and updates, which snapshot-diff approaches cannot do reliably.
What happens if the pipeline stops for a day?
It resumes from the last committed offset, provided the required log is still retained. Retention window is therefore a recovery-time parameter you must size deliberately.