Preserve origin record versions and transaction-log keys during replication - #812
Draft
kriszyp wants to merge 18 commits into
Draft
Preserve origin record versions and transaction-log keys during replication#812kriszyp wants to merge 18 commits into
kriszyp wants to merge 18 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request separates the handling of the origin's log key and the record's version within replication frames, preventing issues where the two clocks were conflated (such as during backdated cache fills). It also introduces validation for incoming frame log keys to avoid poisoning resume cursors with corrupt values, backed by new integration and unit tests. The review feedback suggests adding a safety check to ensure replicationSharedStatus is defined before accessing it to prevent potential runtime errors, and validating lastSequenceIdReceived with Number.isFinite() to prevent NaN propagation into the transaction's local time.
…on receive A replication frame carries two clocks and the receiver now adopts them separately. The leading float64 is the origin's log key — the batch key its entries occupy in the origin's own transaction log — and the forwarded audit bytes carry the record's own version, which a cache fill sets from the source's lastModified rather than from its commit. The receiver used to skip that header and put the entry's version on the apply event as the transaction timestamp, so a peer filed the origin's write in its copy of the origin's log at the source's clock. Leaf state looked right; a third node resuming through that peer read cursors in a domain the relay's log no longer used, and the receive-path resume cursor and the received-version watermark were maxima over record versions rather than log positions. The apply event now carries `timestamp` from the frame's log key and `version` from the entry, the sender frames transactions by the same log key, and both receive-path cursors move with it. The frame key is validated before it can reach setTimestamp, a durable cursor or a status field: a value that is not a representable date — including the sentinel a body too short to hold its own header yields — closes the connection and leaves every cursor where it was. Refs #790 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WMMWWeFsqFuKduJHNMP6ZY
kriszyp
force-pushed
the
fix/replication-dual-clock-origin-log-key
branch
from
September 4, 2026 17:52
71809c8 to
21d2adf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replication frames carry two clocks with different jobs:
versionis the origin record's LWW version.txnLogKeyis the transaction's position in the origin node's log.Previously the frame/log-position value was named
localTimein parts of the implementation, inherited from the deprecated LMDB model. This PR makestxnLogKeythe canonical internal name.localTimeremains only on pre-existing compatibility payloads such as listener/history andend_txn; it is no longer the conceptual name of the clock.What changes
txnLogKeyin the existing frame header and carries the audit body'sversionseparately.txnLogKeybefore any durable state moves.versionand commits it under the origintxnLogKey, then advances resume state and replication watermarks in the log-key domain.txnLogKeyrather than the record version.copyStartTime; the receiver does not persist its per-row frame key as a replay cursor. Audit read errors still fail the copy rather than masquerading as retention.txnLogKeyrather than the record version.This depends on Harper #2497, "Expose record version and txnLogKey as separate audit clocks", which supplies the core audit/read/apply contract and replay-safe log-key identity. The two PRs ship as one artifact.
Why preserve the origin log key
A source fill can have a backdated record version but a current transaction-log position. If a receiver keys that entry at its own receive time, every hop invents a new cursor identity. If it keys the entry at the record version, resume state moves into the LWW clock domain. Preserving the origin
txnLogKeygives every relay the same durable write identity while preservingversionfor record ordering.Verification
versionandtxnLogKeyon every hop, then survives an unclean T restart and resume through R unchanged.npm run buildemits successfully but exits on the repository's existing analyticsNumeric/bigint,Promise.withResolvers, WebSocket_socket, and error-property type failures; no new error is attributable to this change.Scope and follow-ups
Deprecated LMDB keeps its existing transaction-version apply and grouping behavior; the internal audit reader still calls its log position
txnLogKey. A pre-existing relayed-resume duplicate-entry issue remains outside this naming and clock-preservation change.Refs #790
Review-Coverage: authored=codex; ran=cursor-composer,gemini; declined=claude,cursor-grok,domain; rounds=19 @ 0c5976d
Human-Review-Need: 4 @ 0c5976d