fix: concurrency correctness and release hardening

[pathscale]

Summary

Single rolling review PR for WorkTable's concurrency-correctness and beta hardening work. Supersedes #44 and #45 without claiming the separate unsized-update performance issue is fixed.

Closes #37.

Review findings resolved

• Makes immutable row publication mandatory for generated safe APIs, with strict point-read visibility and read-side reclamation.
• Uses the published dependency chain: WorkTablesIndex =0.0.5 directly and through data_bucket =0.5.2; removes the ineffective downstream [patch.crates-io] override.
• Caps the upsert backoff exponent before shifting and advances its counter with saturating_add, preventing debug/test overflow.
• Defers vacuum source-page reset and reuse until the read grace period completes. Empty-link fragments are removed before the whole page can become reusable.
• Prevents reclamation from advertising both an entire empty page and an overlapping retired row link to competing allocators.
• Adds a real multi-page vacuum ABA regression. It fails on the pre-fix implementation by returning an unrelated row at an old physical link and passes with this fix.
• Serializes synchronous insert publication with generated same-key update, delete, upsert, in-place update, and vacuum mutation through 64 FIFO ticket stripes. Multi-row queries acquire only one stripe while processing each already-row-locked key, preserving their global primary-key lock order.
• Strengthens raw insert/delete/upsert churn coverage: every result is classified, then quiesced forward index, reverse index, published data, and public select state are audited together.
• Uses release/acquire synchronization between row-lock holders and waiters.
• Bounds the formerly infinite ignored vacuum soak and joins/aborts its tasks so an interrupted runner cannot leave an unbounded orphan.
• Fixes formatting in update_in_place_unsized.rs.

Dependency status

• WorkTablesIndex 0.0.5 is published and contains the exact multimap-removal fix.
• data_bucket 0.5.2 is published and depends on WorkTablesIndex =0.0.5.
• A clean downstream resolution now uses registry releases only; no workspace-root patch is required.

Performance boundary (explicit)

page_access is currently one RwLock<()> per table. Archived-page insert, update, delete, hydration, reset/reuse, and vacuum mutations that take its exclusive side therefore form a table-wide writer barrier, including for disjoint rows/pages. README and protocol documentation now say this explicitly.

The same-key correctness fix adds one primary-key hash and an uncontended FIFO-ticket acquire/release to mutation operations. Its throughput and p99 impact has not yet been measured.

This PR does not claim HFT contention validation is complete. A fresh throughput and tail-latency campaign for the mandatory publication and mutation-gate paths remains a release/deployment gate.

The ignored same_length_update_stays_in_place fixture also remains an open performance issue: equal-length String updates currently reinsert because the attempted in-place archived-string write is unsafe/corrupting. The fixture documents the blocker; this PR does not claim to resolve it.

Commits (15 on top of master)

• e39364c fix: make row publication concurrency-safe by default
• 6515ba6 fix: avoid stale multimap removal lookup
• 3c248fd fix: stabilize concurrent mutation tests
• 7eab7c1 fix: bound upsert retry backoff to prevent same-key churn livelock
• 8d4e9f2 test: regression for unsized-update in-place (currently failing)
• 4f3f790 test: document unsized-update reinsert bug (ignored regression + root cause)
• 34fcad4 build: bump WorkTablesIndex pin to reviewed rev (multimap-gated)
• 111f61e test: bound vacuum soak to prevent orphaned harnesses
• 87f3dfb fix: cap upsert retry shift before backoff
• b569492 build: consume published index dependency chain
• fdc780b fix: defer vacuum page reuse through read grace
• feba51d test: audit index invariants after mutation churn
• 274da81 docs: disclose table-wide publication writer barrier
• a21cb07 fix: avoid overlapping page and link reclamation
• e730460 fix: serialize synchronous insert with row mutations

Local verification

• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets -- -D warnings
• cargo clippy --workspace --all-targets --all-features -- -D warnings
• cargo test --workspace --all-targets — passed
• cargo test --workspace --all-targets --features versioned-row-publication — passed
• cargo test --workspace --all-targets --all-features — WorkTable: 146 unit + 342 integration passed, 5 integration tests ignored
• Raw insert/delete/upsert churn: 90/90 standalone runs across default, versioned-publication, and all-features, plus passes inside each complete feature suite
• Multi-row primary-key lock-order regression and concurrent secondary-unique rollback coverage pass

CI is the final authority for the pushed branch.

[pathscale]

Added commit 7eab7c1: fix upsert same-key churn livelock (#37).

The generated upsert retry loop used a hot yield_now spin; under sustained unlocked insert/delete churn on one key it could livelock and starve upserter tasks past their 60s timeout — raw_insert_delete_churn_never_panics_or_stalls was failing ~28% of integration runs. Replaced with bounded exponential backoff in both in-memory and persist codegen.

Validation on this branch:

• Failing test: 30/30 clean after fix (was ~28% fail).
• Full integration matrix: 50/50 clean (was 36/50 before fix).
• fmt clean, strict clippy clean, no regression (341 pass / 0 fail / 3 ignored).

Note for reviewers: verified against the branch that several catalog items are already resolved here — #15 (tuple-key persisted reload test passes) and #36 (Running/Closing/Failed/Closed lifecycle + close() implemented). The stale docs/1.0-issue-catalog.md was audited against old master.

[pathscale]

Added commit 1113e9a: move paper-bench → wt-benchmarks as the release-validation harness.

These benchmarks are retained past CIDR-2027 for ongoing v1+ release validation, so they belong on the v1 branch. The move also finishes latent breakages the original crate carried (it never compiled against current worktable): it opts out of the parent workspace, so it needs its own WorkTablesIndex git-rev patch (published 0.0.4 lacks NodeLike::delete_at) and must declare the crates the worktable! macro expands to (rkyv, eyre, derive_more, futures, uuid).

Verified: all three binaries (ablation, baselines, contention) build; contention runs and emits CSV; parent workspace unaffected (crate opts out via its own [workspace]).

[pathscale]

Correction: commit 1113e9a (benchmark move) has been removed from this branch and force-pushed away. The benchmarks live in the standalone ~/code/wt-benchmarks repo, not inside WorkTable. This PR contains only the v1-blocker fixes (through 7eab7c1).

[pathscale]

Stacked two more commits onto this PR (single WT review PR):

• 8d4e9f2 + 4f3f790 — unsized-update overwrite perf bug: root-cause analysis + regression test (committed #[ignore]d; codegen unchanged from master, no behavior change). WorkTable leads insert/point_read but is ~9× behind on overwrite because every update to a String-bearing table reinserts (custom-update gen_size_check inits need_to_reinsert = true). The naive fix corrupts rows, so it needs a storage-path change — tracked with a proof test.

PR now contains 6 commits: #40 row-publication, #23 integration, concurrent-mutation stabilization, #37 upsert livelock, + the overwrite-bug regression.