SC-9: compression support (--compress / -z using zlib) - #13
Merged
Conversation
Literal DataOp bytes compressed via zlib, batched per-file into one buffer per FrameDelta rather than per-op, to avoid zlib's ~8-byte overhead disproportionately hurting small literal runs - falls back to uncompressed if compression doesn't actually shrink the data. CopyOps and signatures are never compressed. --compress-level: verified range 1-9, default 6, 0=off (overrides -z), -1=default, out-of-range clamped, and the '--compress-level alone implies --compress' rule - all matching token.c/rsync.1 exactly. --skip-compress: real rsync's own 96-entry default suffix list, copied verbatim from rsync.1.md. Notable finding: upstream's current docs admit --skip-compress 'has no effect' in their own implementation (no algorithm supports mid-stream level changes) - grsync's per-file design makes a genuine skip decision instead, disclosed honestly as a real behavioral difference from upstream. Compression turned out to be sender-only: Sender always runs locally for every CLI-wired path, so Receiver needed no new options and no SSH-argv or daemon wire-protocol changes. Manual smoke test: a 129,562-byte compressible file dropped to 1,008 bytes received with -z. Verified compatible with dry-run, hard-links, and stats (correctly reporting bytes received, not sent, since Stats runs receiver-side). Clean on native Windows and cross-compiled Linux.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Literal DataOp bytes compressed via zlib, batched per-file into one buffer per FrameDelta rather than per-op, to avoid zlib's ~8-byte overhead disproportionately hurting small literal runs - falls back to uncompressed if compression doesn't actually shrink the data. CopyOps and signatures are never compressed.
--compress-level: verified range 1-9, default 6, 0=off (overrides -z), -1=default, out-of-range clamped, and the '--compress-level alone implies --compress' rule - all matching token.c/rsync.1 exactly.
--skip-compress: real rsync's own 96-entry default suffix list, copied verbatim from rsync.1.md. Notable finding: upstream's current docs admit --skip-compress 'has no effect' in their own implementation (no algorithm supports mid-stream level changes) - grsync's per-file design makes a genuine skip decision instead, disclosed honestly as a real behavioral difference from upstream.
Compression turned out to be sender-only: Sender always runs locally for every CLI-wired path, so Receiver needed no new options and no SSH-argv or daemon wire-protocol changes.
Manual smoke test: a 129,562-byte compressible file dropped to 1,008 bytes received with -z. Verified compatible with dry-run, hard-links, and stats (correctly reporting bytes received, not sent, since Stats runs receiver-side).
Clean on native Windows and cross-compiled Linux.