Skip to content

fix: preserve persistence health across page splits - #61

Merged
pathscale merged 14 commits into
masterfrom
fix/terminal-panic-health
Aug 8, 2026
Merged

fix: preserve persistence health across page splits#61
pathscale merged 14 commits into
masterfrom
fix/terminal-panic-health

Conversation

@pathscale

Copy link
Copy Markdown
Owner

Summary

  • preserve historical unsized-page identities across same-batch splits and removals
  • convert the missing table-of-contents page panic into an actionable persistence error
  • catch persistence worker panics and expose terminal health immediately
  • prepare WorkTable 1.0.0-beta.9

Incident reproduced

AgencyZero 3.20 hit page should be available in table of contents while repeatedly upserting variable-sized settings values. The KV persistence worker died, but the process continued accepting writes until quit attempted a drain roughly twenty minutes later.

The added repeated_string_upsert regression uses the same key/value shape, 256 keys, and 1,000 varying-size upserts. It deterministically reproduced the original panic before the fix and now passes.

A separate lifecycle test deliberately panics a persistence worker and verifies that wait_for_failure reports the terminal error immediately.

Root cause

A split can change a physical page maximum, then a later removal in the same persistence batch can lower it again. A subsequent insert may still reference the pre-split/pre-removal maximum. The table-of-contents lookup had already discarded that historical identity, so the worker panicked. The fix keeps batch-scoped aliases and resolves updates through the current buffered page key.

Verification

  • cargo test: 174 library tests, 522 integration tests passed; 4 ignored
  • doctests: 2 passed
  • cargo clippy --all-targets -- -D warnings
  • deterministic 1,000-upsert regression passes
  • persistence panic lifecycle test passes

Review focus

Please scrutinize alias lifetime and collision behavior, same-batch split/remove/insert ordering, terminal lifecycle transitions, panic payload conversion, and whether any generated table API can miss a worker failure.

@pathscale
pathscale merged commit dc8f398 into master Aug 8, 2026
5 checks passed
@pathscale
pathscale deleted the fix/terminal-panic-health branch August 8, 2026 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant