Own the map download progress lifecycle

[FrogAi]

Developer summary

Map download ownership is currently claimed only after the main loop consumes a progress update. Two commands handled before that poll can start two workers sharing cancellation and file paths. Progress snapshots also share mutable nested pointers with the worker, and a full one-slot queue can drop the terminal update behind an older Active=true value.

This claims the run synchronously, gives each run its own cancel channel, publishes deep-copied latest-value snapshots, and retains terminal state. It does not change HTTP behavior, archive extraction, or the documented between-file cancellation boundary.

Verification

• Current main deterministically leaves a queued active snapshot when cancellation occurs before the first file; this branch exposes the inactive/canceled terminal snapshot.
• The focused lifecycle and immutability matrix passes 100 runs and 10 race-enabled runs.
• Repository-wide test, race, vet, and build gates pass on Linux/Go 1.25.1.

Compatibility

The existing commands, channels, exported download APIs, output schema, URLs, retry behavior, extraction behavior, and documentation are unchanged. Intermediate progress remains coalesced rather than becoming an event queue.

---

Engineering record and audit trail

Root cause

downloadActive currently changes only in GetDownloadProgress. Handle(download) launches a goroutine while that flag is false, so two commands arriving before the next progress poll can both start. Those workers share the global cancel channel, temporary paths, output paths, and progress destination.

The global buffered cancel channel is also not tied to a run. A cancel sent while idle, or too late for the final file's pre-file check, can remain queued and cancel a later run.

Finally, sending DownloadProgress copies only the slice and map headers. The map values are pointers to objects the worker continues mutating. A consumer can therefore serialize a snapshot concurrently with those mutations. Because progress sends are nonblocking on a capacity-one channel, a queued active update can also cause the terminal update to be dropped permanently.

Implementation

• Set downloadActive synchronously before launching the worker.
• Allocate a new buffered cancel channel for each accepted run.
• Ignore cancellation while no run is active and release the run channel when terminal progress is consumed.
• Deep-copy LocationsToDownload, LocationDetails, and each detail value before publication.
• Keep the capacity-one/latest-value model: if a snapshot is already queued, replace it with the newer immutable snapshot.
• Route both intermediate and terminal publications through the same helper.

Behavior

Scenario	Current main	This branch
Two commands before a progress poll	Both can start	First command claims the run; second is ignored while active
Cancel while idle	Token can poison a later run	No-op
Late cancel near completion	Token can cross into the next run	Token remains owned by the completed run's channel
Terminal update with full progress queue	May be dropped behind Active=true	Replaces the stale queued value
Consumer retains an earlier snapshot	Nested values can mutate underneath it	Snapshot remains immutable
Nil or unbuffered caller-supplied progress channel	Nonblocking/best effort	Unchanged

Red/green evidence

The public-API oracle preloads cancellation and calls Download with the production one-slot queue behavior. Current main leaves the first active snapshot queued and drops terminal cancellation. The candidate replaces it with Active=false, Canceled=true without making progress sends blocking.

A controlled HTTP transport then exercises the real settings handler and worker lifecycle. It verifies synchronous ownership, one request for two back-to-back commands, run-scoped cancellation, terminal consumption, and idle-cancel cleanup. A separate retained-snapshot case mutates the worker's source state after publication and verifies the previously published value does not change.

Check	Result
Current-main terminal oracle	Expected failure
Candidate focused matrix, -count=100	Pass
Candidate focused race matrix, -count=10	Pass
go test ./...	Pass
go test -race ./...	Pass
go vet ./...	Pass
go build ./...	Pass
git diff --check	Pass

The original audit also reproduced the nested progress race while extended output serialized the shared map/detail pointers.

Scope limits

This is process-local ownership for the daemon's existing settings path. Direct external callers of Download remain responsible for coordinating their own workers and channels. Terminal retention is guaranteed for mapd's owned capacity-one/single-producer queue; arbitrary nil or unbuffered channels remain best effort.

This does not add request deadlines, in-flight HTTP cancellation, retries, resume support, checksums, multi-process locking, transactional extraction, archive validation, or new schemas. Cancellation still takes effect at the documented boundary after the current file finishes.