Install map archives transactionally

[FrogAi]

Summary

• Stage and validate each complete offline map group before touching the live files.
• Require clean in-group paths, supported entry types, unique entries, complete file writes, and clean tar/gzip completion.
• Replace the live group through backup and rollback renames, and advance progress only after the replacement commits.

Motivation

The existing extractor writes archive members directly into the live map directory. A malformed header, truncated body, gzip checksum failure, file write failure, or process error can therefore leave a mixture of old and partially extracted files. Those failures are logged, but the archive can still be removed and counted as downloaded.

The current file open mode also does not truncate existing files, so replacing a tile with a shorter payload can retain bytes from the old file. Because each downloaded archive contains one complete offline/<latitude>/<longitude> group, installing it as one validated group is the correct ownership boundary.

Behavior

Scenario	Base	Head
Invalid gzip or malformed/truncated tar	May panic or leave partial live files; progress can still advance	Rejects the archive before commit; live group and progress remain unchanged
Corrupt gzip trailer	Extracted content can be accepted	Drains and validates gzip completion before commit
Shorter replacement file	Old trailing bytes can remain	Writes a new staged file into a fresh staging tree, so no stale bytes can survive
Wrong root, traversal, duplicate path, or unsupported entry type	Can escape the expected group or be ignored inconsistently	Rejects the entire archive
Write, sync, or close failure	Can leave live files modified	Aborts the staged installation without changing live files
Live-group backup rename fails	No transactional boundary	Leaves the existing live group untouched
Staged-group install rename fails	No rollback	Restores the previous group; if restoration also fails, preserves the backup and reports its path
Successful archive	Overlays files and can retain obsolete files	Replaces the complete group and increments progress once

Validation

• Audit evidence: source tracing on 68813e05 confirmed direct live writes, non-truncating file opens, undifferentiated tar termination, and progress updates after extraction errors.
• Exact-head review: two independent reviews traced the failure paths and accepted c92bda11db7ba1936d58611182b5ecf0ec3efdba with no material blockers. The reviewed matrix covers invalid gzip, malformed/truncated tar, corrupt trailers, path/type/duplicate rejection, write/sync/close failures, both rename failures, rollback failure preservation, exact shorter replacement, and successful replacement.
• Build: FrogAi Build #52 completed make build successfully for exact head c64d04e. The workflow establishes compilation; the failure matrix is static source-path validation rather than executed fault injection or a tile replay.
• Adversarial audit and follow-up commit. An independent multi-agent audit of c92bda1 installed twelve real production archives (1.7 KB to 55 MB, four continents) and confirmed byte-identical output trees against a verbatim baseline port, reproduced all four fixed baseline defects, and confirmed the CI claim. It raised three items, all addressed in c64d04e:
  • Orphaned staging directories are now reclaimed. Cleanup lived only in a defer, and the repo installs no signal handlers, so any kill or power loss stranded a .mapd-install-* tree (~40 MB each) that nothing ever swept — the existing sweep only covers <base>/tmp. downloadBounds now removes stale .mapd-install-* directories before starting. Verified: three interrupted installs left three orphans that survived both the tmp sweep and a later successful install on the previous head, and are reclaimed now.
  • The archiveInstallOps injection seam is removed (~30 of the original lines). It had a single caller always passing the default, no tests, and could not reach the failure paths it existed for — os.Open and gzip.NewReader bypassed it entirely. Call sites now use the os functions directly.
  • Directory permissions no longer come from the archive. os.FileMode(header.Mode).Perm() is replaced with a fixed 0o755, matching base. Verified: an archive whose group entry carries mode 0555 installed a read-only offline/38 on the previous head and installs 0755 now. The mode is umask-masked regardless, so honoring the archive had no upside.
  • Also dropped: the dead tar.TypeRegA case (Go normalizes it before Next() returns) and a name == "" check already covered by clean == ".".

Compatibility and scope

• Merge order with #107: the two conflict in settings/download.go, because Own the map download progress lifecycle #107 inserts a cancellation check against the block this PR replaces. The verified resolution keeps both, with the cancel check first and installArchive after it, leaving the os.Remove and continue intact — that ordering matters so a cancel still discards an already-downloaded archive as Own the map download progress lifecycle #107 does on base. The resolved merge, and Install map archives transactionally #108+Own the map download progress lifecycle #107+Ignore invalid download paths instead of panicking #109 together, build and test clean. #109 and #110 merge without conflict.
• After merging with Own the map download progress lifecycle #107, note that this PR's continue on install failure skips Own the map download progress lifecycle #107's bottom-of-loop progress publish, so the progress bar stalls through a run of failing archives. Worth a follow-up, not addressed here.
• Download URLs, inputs, output schemas, progress totals, and the offline Cap'n Proto format are unchanged.
• A successful archive now authoritatively replaces its complete group, so obsolete files in that group are removed.
• The transaction boundary is one archive/group, not an entire multi-region download.
• Backup and install use two renames, which creates a brief interval where the live group path is absent. The change handles ordinary runtime failures but does not claim crash or power-loss durability.
• Archive size limits, authenticity, transport retries, and semantic validation of the extracted Cap'n Proto roots remain outside this change.

[FrogAi]

Closing this version because the 202-line transactional subsystem is disproportionate to the demonstrated failure, and its startup orphan sweep removes every .mapd-install-* directory, including a different live worker's staging directory while duplicate download ownership remains possible. Archive installation should be reassessed only after download lifecycle ownership is fixed, with a smaller scope.