Skip to content

GC: 50 MB of an idle TUI's old gen is swept-dead but never returned — the idle reducer's cycle cannot compact by construction #9644

Description

@proggeramlug

Half the old gen of an idle claude-code TUI is dead, swept, and unreachable to the OS, because nothing compacts it and the reducer's cycle cannot.

Measured

Compiled cc TUI (cli_2.1.112.js, macOS), a session with 279 collections behind it, idle after a paste, PERRY_GC_DIAG=1:

[gc-idle-reclaim] start attempt=1 external_collections=279 old_in_use=93608664
[gc] blocks: general=42 (1 live), non_general=143 (130 live, survivors+longlived+old), freed_bytes=2321504
[gc-old-free] reusable_bytes=49986912
[gc-idle-reclaim] done old_in_use=93608664->93606360 reclaimed_old=2304 freed=2370304

50.0 MB of the 93.6 MB old gen is on the free list — already swept, already dead, reusable for future old-gen allocation but never returned. 130 live blocks before the cycle, 130 after. A second reducer full a few seconds later moved the free list to 50.1 MB and released nothing.

This is the residue #9589's Finding 2 was pointing at (it attributed it to pacing defaults; the census in #9637 corrected the mechanism for the allocator half, and this is the arena half).

Why the idle reducer cannot fix it

Old-page defrag is the only mechanism that empties partially-occupied old blocks, and it rides on a moving minor:

  • gc/policy.rs: evacuation_policy_allowed = !low_pause_non_moving, and low_pause_non_moving = progress_kind.is_budgeted().
  • The idle reducer's cycle is budgeted by construction (it is stepped in 4 ms wake-checked slices), so evacuation is off in it by definition.
  • GcCycleState::new_full takes no old-page selection at all; only new_minor_fallback does. A full mark-sweep never compacts, budgeted or not.

So no amount of gate-tuning in gc/idle_reclaim.rs reaches this memory. What would: a moving minor with select_old_page_defrag_pages armed, started from the park hook when the free-list residue crosses a threshold.

What that needs decided first

  1. gc: PERRY_GC_OLD_DEFRAG=0 (the rollback path for default-on old-page relocation) is exercised by no test and no CI arm #7917. Defrag went opt-in (PERRY_GC_OLD_DEFRAG=1) pending "the reproducer and a dependency-scale stress corpus are clean", and that corpus still does not exist — none of the 19 benchmark programs can even produce a candidate page (selection wants dead_bytes >= live_bytes). An idle cc TUI does produce them, at 50 MB, which makes it the missing corpus as much as it makes it the beneficiary.
  2. Pause. Idle is not a promise: the user can end it with a keystroke at any moment. A moving minor is not sliceable the way the budgeted stepper is, so this needs a measured pause budget and a policy for what happens when a wake lands mid-evacuation.

Filed off #9589's reducer work (PR #9643 fixes the pricing bug that made the reducer switch itself off before it ever got near this).

https://claude.ai/code/session_011qE5TRRJFzqxN44K34AnG2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions