SC-12: partial and append file transfers - #15
Merged
Conversation
Verified against real rsync source (generator.c, sender.c): plain --append blindly trusts the existing prefix with zero real checksums; --append-verify runs the identical normal per-block signature algorithm, needing no new algorithm, only an eligibility gate. Both share two rules: non-existent destination transfers normally, destination not shorter than source is skipped entirely. Prerequisite correctness fix, not scope creep: Receiver previously wrote regular files directly to their final destination with no temp file at all, meaning a process kill mid-write could corrupt a good destination file regardless of any flag - this predates SC-12 entirely. Implementing --partial correctly required making temp-file-then-atomic- rename the unconditional default write path first; --partial/ --partial-dir only control what happens to that temp file if a transfer aborts before the rename. --partial/--partial-dir: file-granularity, explicitly disclosed as not true mid-file resumption given grsync's atomic-per-file wire frames (same disclosure pattern as SC-10's progress reporting). Real content-level resumption across runs via using a leftover partial-dir file as the delta comparison basis. --append: wire-level blind trust via a directly-constructed CopyOp, reusing sync.ApplyDelta unchanged. --append-verify: normal pipeline, gated by eligibility. Self-review: --append's documented corruption risk faithfully reproduced, not worsened, locked in by a dedicated test. Found and fixed sync.ApplyDelta unconditionally rejecting BlockSize <= 0 even when no CopyOp needed it - would have broken append mode against a brand-new empty destination file. Real wire-byte-count proof partial-dir resumption is genuinely cheaper. Real interrupted-multi-file-sync test, corruption-preserved-vs-corrected proof, dry-run and hard-link exclusion tests, real SSH test. Clean on native Windows and cross-compiled Linux.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Verified against real rsync source (generator.c, sender.c): plain --append blindly trusts the existing prefix with zero real checksums; --append-verify runs the identical normal per-block signature algorithm, needing no new algorithm, only an eligibility gate. Both share two rules: non-existent destination transfers normally, destination not shorter than source is skipped entirely.
Prerequisite correctness fix, not scope creep: Receiver previously wrote regular files directly to their final destination with no temp file at all, meaning a process kill mid-write could corrupt a good destination file regardless of any flag - this predates SC-12 entirely. Implementing --partial correctly required making temp-file-then-atomic- rename the unconditional default write path first; --partial/ --partial-dir only control what happens to that temp file if a transfer aborts before the rename.
--partial/--partial-dir: file-granularity, explicitly disclosed as not true mid-file resumption given grsync's atomic-per-file wire frames (same disclosure pattern as SC-10's progress reporting). Real content-level resumption across runs via using a leftover partial-dir file as the delta comparison basis.
--append: wire-level blind trust via a directly-constructed CopyOp, reusing sync.ApplyDelta unchanged. --append-verify: normal pipeline, gated by eligibility.
Self-review: --append's documented corruption risk faithfully reproduced, not worsened, locked in by a dedicated test. Found and fixed sync.ApplyDelta unconditionally rejecting BlockSize <= 0 even when no CopyOp needed it - would have broken append mode against a brand-new empty destination file.
Real wire-byte-count proof partial-dir resumption is genuinely cheaper. Real interrupted-multi-file-sync test, corruption-preserved-vs-corrected proof, dry-run and hard-link exclusion tests, real SSH test.
Clean on native Windows and cross-compiled Linux.