perf(concourse): shell out git clone instead of libgit2's RepoBuilder

[bodymindarts]

Summary

The [cepler-perf] timing logs from v0.7.21 (shipped in #25) showed every wall-clock second of the in step lives in one place:

[cepler-perf] RepoBuilder::clone (fetch + pack index): 64.35s   ← 99.6% of the time
[cepler-perf] index sync (Mixed reset from HEAD):       0.00s
[cepler-perf] read config from HEAD tree:               0.00s
[cepler-perf] selective checkout (config + state dir):  0.11s
[cepler-perf] ws.check (construct_env_state walk):      1.25s
[cepler-perf] ws.prepare (...):                         1.60s

Build #109 — the same build whose log shows a parallel concourse git-resource cloning the same repo in ~6s via the git CLI:

[21:04:29] Cloning into '/tmp/build/get'...
[21:04:31] Receiving objects: 100% (64754/64754), 15.20 MiB | 19.49 MiB/s, done.
[21:04:35] Resolving deltas: 100% (39641/39641), done.

So our 64s is libgit2 doing pack indexing slowly, not a fundamental cost of the operation.

This isn't an anomaly — it's a documented libgit2 limitation:

• Cloning large repo in libgit2 slower than git clone · #4674
• Diff via libgit2 on bare repo almost 10 times slower than git · #3920
• Very slow cloning of "file://" repository format · #2836
• Local Repository.Clone performance is terrible · libgit2sharp#403
• RFC: libgit2 is slow/inefficient; switch to git command? (Guix devel) — same conclusion we reached.

Per those threads the bottleneck is mmap/munmap inefficiency in git_mwindow_free_all_locked, not anything we can patch from cepler.

Fix

Replace RepoBuilder::clone with a git clone --no-checkout subprocess. The rest of cepler keeps using libgit2 via Repository::open — only the bulk transfer is shelled out.

The container image already ships git + openssh via Alpine (see images/concourse/Dockerfile). Standalone CLI users almost certainly have it too.

SSH key handling

The source's private_key is now passed to git via GIT_SSH_COMMAND instead of libssh2's in-memory key path:

• write_ssh_key writes it to /tmp/cepler-ssh-key-<pid>-<ns> with 0600 perms (ssh refuses anything looser).
• A KeyFile RAII guard removes the tempfile on drop, so a panic between key-write and git clone exit can't leak the key.
• GIT_SSH_COMMAND adds -o IdentitiesOnly=yes -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null — matches the prior CertificatePassthrough-style trust-on-first-use posture (concourse pipelines authenticate via the key, not via host-key trust).
• Empty private_key short-circuits to None, so local-bare-repo unit tests keep working without going through ssh setup.

Test plan

• cargo nextest run — 7/7 pass:
  • The two existing clone tests (clone_skips_working_tree_checkout_but_populates_index, checkout_paths_materialises_only_requested_files) now exercise the git subprocess end-to-end against a local bare repo. They prove git clone --no-checkout produces the same WT-empty / index-populated state libgit2's GIT_CHECKOUT_NONE + Mixed reset did — including write_tree-from-index reproducing HEAD's tree, which is the load-bearing invariant commit_state_file on out depends on.
  • Two new tests: write_ssh_key_round_trips_content_and_perms_and_drop_removes_file (the file exists with 0600 while KeyFile is alive; gone after drop) and write_ssh_key_returns_none_for_empty_input (empty-key short-circuit).
• cargo clippy --all-targets -- -D warnings — clean
• nix flake check — passes
• Roll the new image into volcano-qa and confirm [cepler-perf] git clone subprocess drops from ~64s to ~5-8s, mirroring what the parallel git-resource sees today

Expected impact

in step total drops from ~65-70s to ~5-8s on volcano-qa-lana-bank. The Mixed reset stays as defensive insurance (git clone --no-checkout already populates the index, so it's a no-op there) — kept for invariant locality at ~0s cost.

Trade-offs

• ✅ ~10x speedup on the dominant cost
• ✅ Multi-threaded delta resolution for free (git CLI uses all cores; libgit2 doesn't)
• ✅ git CLI is already in the image
• ⚠️ Adds a git CLI dependency on PATH. Surfaced clearly in the error message if it's missing.
• ⚠️ Subprocess spawning surface — but on a code path that was already shelling out to ssh under libssh2 anyway.

🤖 Generated with Claude Code