feat(copy): use rsync by default with scp fallback#423
Open
theFong wants to merge 9 commits into
Open
Conversation
# Conflicts: # pkg/cmd/copy/copy.go # pkg/cmd/copy/copy_test.go
Skip the rsync attempt entirely when rsync is not installed locally, and when the instance itself lacks rsync, say so in the fallback message. Both scp paths now tell the user to install rsync for faster transfers.
stephahart
previously approved these changes
Jul 14, 2026
Without a trailing slash rsync nests the source directory inside the destination (dest/dir/...), while scp makes the destination the copy when it does not exist. Normalize directory-upload sources to trailing-slash form so the destination always mirrors the source.
Directory sources are normalized to contents-copy form (trailing slash for rsync, /. for scp) so the destination always becomes a mirror of the source, whether or not it already exists, and both transfer tools produce identical layouts. Downloads learn the remote path type via a single 'ssh test -d' probe; uploads stat the local source directly. This also makes repeated directory copies idempotent instead of nesting dir inside dir on the second run.
patelspratik
approved these changes
Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuation of #297 by @immanuel-peter (his commits are preserved on this branch), rebased onto current main since the fork branch had conflicts.
Summary
brev copynow attempts rsync first for file/directory transfers and falls back to scp automaticallycopyExternalNode) go through the same rsync-first pathWhy rsync first
rsync supports incremental/delta transfer, so repeated and large transfers send fewer bytes and finish faster than scp. scp remains as an automatic fallback so environments without working rsync behave as before.
Verification
go test ./pkg/cmd/copygo build ./...Closes #297
Performance: rsync vs scp
Measured (real Brev instance)
Benchmarked from macOS (openrsync, protocol 29) to a running Brev instance (
n2d-standard-2, Ubuntu 22.04, GNU rsync 3.2.7) over a real WAN link (~1 MB/s upload at test time), using the exact commands this PR ships:scp [-r]vsrsync -z -e ssh [-r]. All transfers md5-verified after the run.Notes on the numbers:
-zand delta), which matters since macOS Sequoia 15.4+ ships openrsync instead of GNU rsync (details).Public references
Known tradeoff
-zcompression is a win on slow links with compressible data (code, logs, CSVs) but is CPU-bound and can bottleneck fast links when the payload is already incompressible (model checkpoints, archives, images) — rsync itself keeps a--skip-compresssuffix list for this reason (man page, example report). In our test it was neutral even on incompressible data over a ~1 MB/s WAN, which matches the typical laptop→cloud path forbrev copy. If users report slow checkpoint uploads on fast links, dropping-zor adding--skip-compressdefaults is a easy follow-up.End-to-end validation
Validated with a binary built from this branch (
brev copy, not raw rsync/scp) against a live Brev instance (macOS openrsync client → Ubuntu 22.04 / GNU rsync 3.2.7):PATH: printsrsync not found on this machine, using scp. Install rsync for faster transfers.and completes via scp (md5-verified)Validation caught one regression, fixed in d4fc346: without a trailing slash, rsync nests an uploaded directory inside the destination (
dest/dir/...) where scp madedestitself the copy. Directory-upload sources are now normalized to trailing-slash form so the destination always mirrors the source.Unified directory semantics (71d7760)
rsync and scp historically disagree about directory copies (nest vs. copy, depending on trailing slashes and whether the destination exists). This PR unifies both engines on one rule: the destination always mirrors the source.
brev copy ./dir instance:dest→dest/a.txt— same result whetherdestexists or not, whether rsync or the scp fallback ran, and for downloads too. Repeated copies are idempotent (no moredest/dir/dirnesting on re-runs).Implementation: directory sources are normalized to contents-copy form — trailing
/for rsync,/.for scp (verified equivalent on OpenSSH 10 SFTP-mode scp). Uploads stat the local source; downloads determine the remote path type with a singlessh <alias> test -d <path>probe (~1 extra round trip per download). All six layout combinations validated end-to-end against a live instance.