GC never collects when idle: heap freezes at last-allocation state — cc TUI pins 425-680 MB indefinitely; pacing+defrag defaults leave ~200 MB on the table

[proggeramlug]

Measured 2026-09-03 on the verified cc build (macOS, main 8d27f7b964), three-arm PTY experiment (idle → 400 KB paste-churn → 4-min settle), footprint sampled every 10 s. Raw data /tmp/gc_arms.csv on the dev box.

arm	idle sawtooth trough	post-churn (4 min)
default	449 MB	425 MB, flat — zero collections
MAJOR_PACING_GROWTH=1 FLOOR_MB=8	~365 MB	564 MB, flat
+ OLD_DEFRAG=1	~365 MB	458 MB, flat

Finding 1 — no idle-time collection (the structural gap)

Collections trigger on allocation. An idle TUI barely allocates ⇒ after a burst, the heap freezes at whatever the last mid-burst cycle left — the post-churn column is last-collection timing luck (564 > 425 under more aggressive pacing proves it), and it persists indefinitely. A live user session sat pinned at 680 MB through minutes of idle for exactly this reason. Node idles the same bundle at ~150–200 MB because V8 runs an idle memory-reducer; perry has no equivalent. Fix shape: an idle hook (the event loop knows it is parking — the #9540 body_check site is where "about to idle" is known) that after N quiet seconds runs a major + page return, budgeted so a keystroke never lands mid-cycle (incremental or bounded pause).

Finding 2 — default pacing + defrag leave ~200 MB combined

Aggressive pacing lowers the idle trough 449→365 (~85 MB). OLD_DEFRAG=1 reclaimed ~105 MB of post-burst residue. Both knobs work today; the defaults are tuned for throughput benchmarks, not resident TUIs. Proposal: idle-reducer runs with the aggressive profile (collect-to-live + defrag) regardless of the throughput defaults, since by definition nobody is waiting.

Finding 3 — bounded by a separate census

Fresh-idle live set is ~330 MB GC (vs node's whole process 150–200) — that remainder is representation/side-tables territory, being censused separately; this issue is only the two collector-behaviour gaps above.

Verification bar

The three-arm PTY harness is the regression test shape: post-churn line must descend within N idle seconds instead of staying flat, without keystroke-latency regression (assert input-echo timing under the reducer). The 2.1 GB transient peak during heavy ops should also shrink under pacing review — assert peak in the harness too.

[proggeramlug]

PR #9618 implements Finding 1 and, unexpectedly, a second gap that would have hidden it. Both are runtime-only.

1. Idle-time reclaim. js_wait_for_event now hands the collector the moment before it parks — no notify, no queued microtask, no timer due. That is the runtime's real idle point, and it has to be: "no allocation for N seconds" can never fire on this workload. Measured on the verified cc build at rest, the TUI runs a copying minor every ~6 s that copies ~10 MB of surviving render tree at 673 permille survival, and [gc-incremental] reports cycle_starts=0 — the budgeted collector never runs in cc at all. What cc does do is park thousands of times a minute.

When at least one collection has completed since the reducer's last full and the loop has been quiet 3 s, the hook opens a full budgeted cycle (new idle_reclaim trigger, block-pool drain armed) through the existing incremental stepper and drives it in 4 ms slices, checking for a pending wake before each one, so a keystroke waits at most one slice. Unproductive fulls back off exponentially, which is what keeps a retaining idle heap from paying a whole-heap mark per minor; a burst re-arms it immediately. A 500 ms-per-wall-second work cap bounds a cycle that somehow could not finish. Kill switch PERRY_GC_IDLE_RECLAIM=0, counters on the [gc-idle-reclaim] diag line.

2. The page-return half was independently broken, and it hid the first fix. The full cycle's reclaim tail called glibc malloc_trim / Darwin malloc_zone_pressure_relief, but the global allocator is mimalloc, which neither reaches. The first working version of the reducer therefore showed heapUsed 47 MB → 18 MB with RSS rising. The GC trace said exactly what happened: 18 dead 1 MiB blocks pooled, drained and deallocated — into mimalloc's cache, waiting for a delayed purge that only the next allocation or free would trigger, which is precisely what an idle program never does. The tail now also calls mi_collect(true), and the park hook purges once more after the cycle completes, because the cycle's own bookkeeping is only freed after its tail ran. This also repairs the existing critical-pressure path (js_gc_memory_pressure(2)), which had the same hole.

Measured on a fixture that builds 300k objects, drops them, then idles behind a timer:

arm	heapUsed	Linux RSS	macOS footprint
default	47.2 → 18.1 MB at t≈3.5 s	113 → 83 MB	63 → 48 MB
PERRY_GC_IDLE_RECLAIM=0	47.2 MB flat	flat	flat

The reducer's full: 34 slices, 79 ms of total step time, reclaimed_old=21.9 MB freed=29.2 MB productive=true.

Tests: 9 unit tests covering every gate in both directions, the yield-on-wake path, the work cap, finishing a cycle the pacer left open, and the kill switch OFF; plus the compiled-program regression above as a two-arm test, so the OFF arm exercises the knob's off state in CI as the kill-policy requires. perry-runtime is 3021 passed / 0 failed with RUST_TEST_THREADS=1.

I am running your three-arm PTY harness against a cc binary built from this branch now and will post the idle-trough and post-churn numbers, plus keystroke-echo latency under the reducer, when it finishes. Finding 3's census is untouched here; Finding 2's pacing knobs are no longer what decides the idle trough, since the reducer collects to live at idle, and PERRY_GC_OLD_DEFRAG stays opt-in pending #7917's stress corpus.

[proggeramlug]

The census this issue's Finding 3 defers to is now done (landed with the allocator hand-back fix). It corrects Findings 2 and 3, and leaves Finding 1 standing as the real gap.

Finding 3 was wrong by an order of magnitude, and it was my number. Fresh-idle GC-live is 22.9 MB, not ~330 MB. The 644 MB "GC arena" reading behind that estimate came from macOS VM tag 240, and perry retags mimalloc with that tag — so it covers every Rust allocation in the process, not the arena. There is no representation problem of that size to chase: per-object overhead is 10.4 MB inside a 22.9 MB live set.

Finding 2's mechanism was not pacing. The GC's allocator hand-back step called libc::malloc_trim on glibc and malloc_zone_pressure_relief on macOS while mimalloc is the #[global_allocator], so a collection freed objects the OS never got back. Telemetry: three explicit full collections each ran malloc_trim(0) in 16-23 us and moved RSS by 0 MB. Same binary, same forced collection, one env var apart:

arm	idle RSS	after major GC
baseline	854 MB	868 MB
MIMALLOC_PURGE_DELAY=0	835 MB	517 MB

That is why the aggressive-pacing arm could read higher post-churn than the default one: the knobs were changing when collections ran, not whether their output reached the OS. mi_collect(true) now runs at the same Reclaim point on major cycles, and the default configuration drops the idle TUI from 826 MB to 459 MB on a major collection. Side tables also came down about 38 MB (hashbrown never shrinking after pruning, and the function name/source registries copying the program image into the heap at module init).

Finding 1 is untouched and is now the whole issue. There is still no idle-time collection. What changed is the payoff: before, an idle reducer would have run a collection that returned nothing to the OS, so it would have looked like it barely worked. Now a major collection hands back ~360 MB, and an idle hook is the thing that decides whether an idle process ever gets to run one.

Two collector-side over-retention bugs found while validating the instrument are filed separately and both make an idle reducer less effective than it should be: #9628 (an explicit gc() resurrects the whole recent-block window) and #9629 (a full mark-sweep marks the remembered set as roots, retaining young objects through dead old-gen pages).

Suggested retitle when someone picks this up: the pinned-MB figures and the pacing framing in the title no longer describe what is happening.

For the regression harness proposed here, PERRY_GC_CENSUS=<path> plus SIGUSR2 gives a per-phase breakdown (live/late-marked/dead by space, type and class, plus every side table) without needing to infer anything from RSS.

[proggeramlug]

The reducer and its pricing fix are both on main (PR #9618 via merge train #9627; PR #9643 content is on main — freed_bytes >= bar in gc/idle_reclaim.rs plus changelog.d/9589-idle-reclaim-productivity.md). Closing this; the remaining old-gen work is #9644, which has its own PR (#9671).

Finding 1 — fixed, and then fixed again. The park-site reducer landed, but measuring it on the compiled TUI showed it switching itself off on the workload it was built for: two fulls scored unproductive at reclaimed_old=2304 and 0 while freeing 2.37 MB and 0.51 MB, and the activity requirement doubled twice inside a minute of idle. The bug was the meter. old_gen_in_use_bytes is the sum of the live blocks' bump offsets, and a non-moving sweep cannot lower it — dead objects go to the old-gen free list and the block keeps its offset. Fulls are now priced by what the sweep freed. The unit tests had passed because their litter fills blocks that end up entirely dead, a shape no real workload has.

Finding 2 — mechanism was mimalloc, and the residue is #9644. Corrected earlier in this thread. What the pacing knobs were really moving was when collections ran, not whether their output reached the OS. The rest of it — the 50 MB of swept-dead free list on a 93.6 MB old gen — needs compaction, which a budgeted (non-moving) cycle cannot do. That is #9644, now measured: the fixture goes 41.9 -> 13.6 MB old gen with the free list to zero, in a 132 ms pause.

Finding 3 — retracted. Fresh-idle GC-live is 22.9 MB, not ~330 MB; that estimate was mine and came from reading macOS VM tag 240, which perry retags mimalloc with.

Measured on cc, for the record (verified build, PERRY_GC_DIAG=1, real session, 279 collections behind it): the reducer reaches the park hook, opens a full, steps it 86 slices with zero yields, and purges. It fires only in genuine idle stretches — both attempts landed after a paste, not during one. Keystroke-latency numbers from my earlier harness runs were an artifact of the harness (cc never reached a prompt in a bare PTY; every arm timed out identically at 6 s) and should not be read as latency data.

Two collector-side over-retention bugs found on the way stay open and both make an idle reducer less effective than it should be: #9628 and #9629.

[proggeramlug]

Follow-up measurement: the idle reclaim fires once at startup and then never again — this issue is not finished.

From a validated heap census on the current build (which contains this fix), 5-minute idle soak, phys_footprint sampled at t=30/60/120/180/300 s:

• perry: 279.0 / 279.0 / 279.0 / 279.0 / 279.1 MB — bit-for-bit flat, tag-240 dirty pinned at 250.3 MB.
• node, same bundle/HOME: V8 heap 93.6 → 58.8 MB, footprint 239 → 120 MB over the same window.

The log shows gc-idle-reclaim … freed_bytes=13352896 once during startup, then silence. So the hook exists and works, but is not re-armed for a genuinely idle session.

Also recorded, so nobody re-runs them: eight knob arms move the idle footprint by nothing (all inside the ±3.5 MB run-to-run spread; A-arm repeats: 280.4 / 275.6 / 282.7 / 275.7): MIMALLOC_PURGE_DELAY=0 276.0; +MIMALLOC_ARENA_PURGE_MULT=1 +PURGE_DECOMMITS=1 276.0; PERRY_GC_MAJOR_PACING_GROWTH=1 FLOOR_MB=8 282.8; PERRY_GC_SCAVENGE_NURSERY_MB=8 282.9, =16 278.1. Positive control: PERRY_GC_MALLOC_PURGE=0 (disabling the post-cycle trim) gave 276.2 — the trim currently buys nothing either. The ~85 MB pacing / ~105 MB defrag savings measured earlier do not reproduce on this binary.

Two corrections to the record that matter for whoever picks this up: Memory Tag 240 is mimalloc, not "the GC arena" — the arena is only 98 MB of the 250 MB dirty; and RSS is misleading on this binary (~1.9 GB, of which ~1.4 GB is MADV_FREE reclaimable) — trust phys_footprint.

Note the biggest idle component is not collectable at all: ~84 MB of leaked compiled regex programs (#9678). An idle reclaim cannot return memory the GC never owned, so #9678 should land first or the reclaim will look ineffective for the wrong reason.