The right tool for the right job: CDC with initial load in Tabsdata 2.2.0

Every CDC pipeline I have seen starts the same way. A bulk load copies what is in the database today, and a change stream picks up “more or less” where the bulk load stopped. Then somebody writes the code that glues the two together. Then somebody else finds the duplicates. And when there is more than one table (there is always more than one table) the real problem shows up. An invoice is a row in invoice_header and a few rows in invoice_item, written in one database transaction. If your pipeline lets a report see the header without its items, the report is wrong, and nobody will tell you.

Tabsdata 2.2.0 adds a full load to the CDC sources. start_from already told a stream where in the change log to begin; start_from="full" makes it begin with the tables themselves. One stream publisher snapshots the tables you track, switches to the change log at the exact position of that snapshot, and from there on publishes changes: all tables, one commit at a time. No glue code. No second system.

The publisher

We need four tables out of the billing database: customer, address, invoice_header, invoice_item.

@stream_publisher(
    source=MySQLCdcStreamSrc(
        tables=[
            "billing.customer",
            "billing.address",
            "billing.invoice_header",
            "billing.invoice_item",
        ],
        start_from="full",
    ),
    batch_max_secs=60,
    output_tables=["customer", "address", "invoice_header", "invoice_item"],
)
def billing_cdc(
        customer: TableFramesSpec,
        address: TableFramesSpec,
        invoice_header: TableFramesSpec,
        invoice_item: TableFramesSpec,
) -> tuple[TableFrameSpec, TableFrameSpec, TableFrameSpec, TableFrameSpec]:
    return concat(customer), concat(address), concat(invoice_header), concat(invoice_item)

That is the whole thing. Host and credentials live in the collection’s connection, not in the function. Registering the function starts the stream: a consistent snapshot of the four tables, then CDC from the exact position of that snapshot. No gaps, no duplicates, always consistent.

What start_from="full" does

Snapshot first. The connector pins one position in the binlog (the GTID set) and opens a consistent-snapshot, repeatable-read transaction there. It takes a global read lock for an instant: the time it takes to read the position, not the time it takes to scan the tables. The four tables are read inside that one transaction, so the snapshot is one instant of the database, not four.

Changes that land while the tables are being read are not in the snapshot, which is one instant. They come next, from the binlog, starting at the snapshot’s position. Nothing lost, nothing read twice, and nothing for you to configure or reconcile.

The load is one commit. All four tables staged as one batch, published as one Tabsdata commit. If anything fails half way, nothing is published. Each table opens with an l (load) control row followed by its rows as inserts, so downstream functions know the table is being reset to a full state.

Then CDC, from exactly there. The stream reads the binlog from the GTID set the snapshot is consistent at. Batches close only between source transactions, and each batch is published as one commit across all the output tables. A header and its items written in one MySQL transaction arrive in Tabsdata in one commit.

Every change row carries the operation (@td.cdc.meta.op: i, u, d, plus the l and t control rows) and the source transaction id. Need to resync from scratch? tdk fn update --reset-stage and the stream starts over with a new full load.

Same setting, same behavior, for PostgreSQL (PostgresCdcStreamSrc, reading the WAL through a logical replication slot) and Db2 (Db2CdcStreamSrc).

What you need on the MySQL side: the binlog on, in row format with full row images (log_bin=ON, binlog_format=ROW, binlog_row_image=FULL), GTID mode on, InnoDB tables, and a user with replication privileges, SELECT on the tracked tables and, where the server grants it, RELOAD for the lock. Managed services such as Amazon RDS and Aurora refuse the global read lock; Tabsdata handles that too, with the same transactional consistency.

What you don’t need: Kafka, Debezium, an ingestion runtime, a staging area, reconciliation code.

Where Databricks and Snowflake stand

Databricks and Snowflake are excellent warehouses, and Tabsdata makes getting data into them (ingested, prepared, trustworthy) simple. CDC that is consistent across tables is a gap in both today, and I want to be straight about it. This is their own tooling, for the same MySQL billing database, as of October 2026:

Both managed connectors handle the snapshot-to-CDC handoff. Both apply each table on its own. Consistency across tables is left to you, and getting it means more systems (Debezium, Kafka, Kafka Connect) and a good deal of custom code: buffer the transactions, match them against the transaction metadata, manage the offsets, commit the related tables together. One exception worth noting: Snowflake’s data mirroring for Snowflake Postgres (in Preview) does apply source transactions across tables together, if your source is Snowflake’s own hosted Postgres.

One publisher, one setting

If your warehouse runs on CDC data, and if you build bronze, silver and gold on it, consistency across tables is what decides whether those layers can be trusted. With Tabsdata it is one declarative publisher and one setting. Full load and change stream in a single flow, every commit covering every table, the medallion layers running inside the same transaction boundaries, all of it observable from one place. And when the gold tables need to be in Databricks or Snowflake, a subscriber takes them there.

Tabsdata 2.2.0 is out. The MySQL CDC, PostgreSQL CDC and Db2 CDC guides have the setup details.