Log-based CDC vs polling: a reproducible benchmark (PostgreSQL → Snowflake)
Most CDC comparisons argue about technique. This one is a method: what to measure, how to measure it, and which results are valid. Run it on your own data before you believe either vendor.
Engineering · 2026-11-03 · Sumeta engineering
The two designs differ in where cost lands. Polling pays in repeated table scans whose cost scales with table size; log-based capture pays in log retention and log-read I/O, whose cost scales with change volume.
Method
| Parameter | Setting | Why it matters |
|---|---|---|
| Source | PostgreSQL with wal_level=logical | Logical decoding is required for log-based capture |
| Target | Snowflake warehouse, fixed size | Removing warehouse elasticity isolates capture cost |
| Dataset | One wide table plus one high-churn narrow table | Separates scan cost from change cost |
| Load | Steady inserts and updates for 6 hours | Avoids burst effects that mask steady-state cost |
| Instrumentation | Source CPU, I/O, query latency; capture lag | Measures impact, not just throughput |
The two designs should be run against the same dataset and the same target, with the same transformation. Changing more than one variable makes the comparison an anecdote.
What to measure
| Metric | Polling | Log-based | Interpretation |
|---|---|---|---|
| Source CPU during sync | Spikes with table scans | Broadly flat; log-read I/O only | The main operational difference |
| Source query latency | Visible during scans | No measurable change in typical deployments | What your application team feels |
| Capture lag (P95) | Bounded by interval | 40–60s typical | Freshness floor |
| Bytes read from source | Scales with table size | Scales with change volume | Why the curves cross |
| Recovery after outage | Re-scan or diff | Resume from offset | Cost of a bad night |
The result that matters is not "which is faster" in the abstract. It is where the two curves cross for your table sizes and change rates. A small, high-churn table favours log-based capture on impact; a large, low-churn table favours whichever design minimizes scans — and that is usually log-based too, which is the counter-intuitive part.
How to reproduce it
- Create a source table with a realistic row width and a primary key that reflects your access pattern.
- Enable logical decoding and confirm a replication slot can be created.
- Generate a steady change workload; record rows changed per second.
- Run polling at the interval you actually use, and log-based capture against the same target.
- Sample source CPU, I/O and query latency at one-second resolution for both runs.
- Force a 30-minute outage mid-run and measure recovery time and duplicate rows for each design.
Step six is the one most benchmarks omit, and the one that decides operational cost. A design that recovers by resuming an offset behaves very differently from one that must re-scan.
What invalidates a benchmark
- Different targets. Warehouse size changes load latency more than capture design does.
- Different transforms. A transformation tier hides the capture difference.
- Burst-only load. Peaks hide steady-state scan cost.
- No outage test. Recovery behaviour is where the designs diverge most.
Run the benchmark on your data
Download the build and repeat the six steps against a copy of your own source.