Skip to content

[SS-97] Tests and benchmarks for MySQL snapshot#37698

Merged
peterdukelarsen merged 1 commit into
MaterializeInc:mainfrom
peterdukelarsen:pl/mysql-snapshot-tests
Jul 17, 2026
Merged

[SS-97] Tests and benchmarks for MySQL snapshot#37698
peterdukelarsen merged 1 commit into
MaterializeInc:mainfrom
peterdukelarsen:pl/mysql-snapshot-tests

Conversation

@peterdukelarsen

@peterdukelarsen peterdukelarsen commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Motivation

Setting up tests and benchmarks for #37642 ahead of time where possible.

Description

  • 8-worker benchmark for a single 1M row table (which should get faster if we parallelize)
  • 8-worker benchmark for many tables to prevent regressions
  • Varied test-drive tests to help ensure no regressions in correctness

Verification

The first benchmark was used to show the speedup, the second caught a regression in the original version of the PR, and the testdrive tests should help with any obvious/simple correctness issues.

@peterdukelarsen
peterdukelarsen force-pushed the pl/mysql-snapshot-tests branch 5 times, most recently from b2be70f to 79512bd Compare July 16, 2026 21:58
@peterdukelarsen
peterdukelarsen marked this pull request as ready for review July 17, 2026 15:17
Add mysql-cdc multi-worker snapshot-correctness testdrive cases and the
MySqlInitialLoadMultiWorker* feature-benchmark scenarios ahead of the parallel
PK-range snapshot implementation. They run and pass against the current
single-worker-per-table snapshot (the testdrive cases assert data correctness
regardless of partitioning; the benchmarks establish a pre-feature baseline),
so the implementation PR stays focused on snapshot.rs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@peterdukelarsen
peterdukelarsen force-pushed the pl/mysql-snapshot-tests branch from 79512bd to 1d729a7 Compare July 17, 2026 15:27
@peterdukelarsen
peterdukelarsen requested a review from a team July 17, 2026 15:39

@patrickwwbutler patrickwwbutler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, though maybe we want @def- to have a look at some of the finer details of the benchmarking. I think the parallel workload failure is unrelated, given you didn't add anything there

@def- def- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests lgtm, thanks for adding them!

@peterdukelarsen
peterdukelarsen merged commit ba99c4b into MaterializeInc:main Jul 17, 2026
257 of 262 checks passed
@peterdukelarsen
peterdukelarsen deleted the pl/mysql-snapshot-tests branch July 17, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants