fix(gc): retire uncarried shape descriptors

changelog.d/9724-shape-descriptor-slab.md

@@ -0,0 +1,102 @@
+### Performance
+
+- **Shape descriptors: an id-indexed slab and two compact reverse indices
+  instead of a boxed hash map and two `Vec`-valued ones; owned growth history
+  is retired** (#9706). `PERRY_GC_CENSUS` on the compiled
+  claude-code TUI at idle put the agent-local shape table at ~330 bytes per
+  live descriptor. The bytes were the STORAGE, not the facts: a 56-byte
+  `Box<ShapeDescriptor>` in a 64-byte allocator bin, a 16-byte
+  `PtrHashMap<u32, Box<_>>` entry sitting at ~25% load after
+  `shrink_to(2 * len)`, a 57-byte bucket in the exact-facts reverse map plus a
+  16-byte `Vec<u32>` buffer per entry, and a 33-byte bucket in the keys-address
+  reverse map — four allocations and three hash tables saying the same thing.
+
+  `crates/perry-runtime/src/object/shapes_store.rs` (new):
+
+  - **`ShapeSlab`** — the by-id store. A ShapeId is `SHAPE_ID_BASE + n` from a
+    process-global monotonic counter, so `n` indexes a chunked slab directly:
+    no hash, no per-record allocation, and a record address that never moves
+    for the record's lifetime, which is the property the collector relies on
+    when it enumerates the record's `keys` word as a rewritable slot (#8112)
+    and retains that address across budgeted resumptions. Chunks (32
+    records, behind a two-level page directory) are allocated lazily — a
+    worker's ids interleave with the main thread's, and the claude-code TUI
+    mints a million ids at startup for 44 k survivors — and an all-dead chunk
+    or page is released at the same cadence as the reverse-index shrink
+    (`shrink_shape_tables`, once per major collection).
+    The direct-mapped lookup-way cache that fronted the hash map is gone: a
+    slab probe IS "shift, index, deref", and it needs no invalidation epoch.
+  - **`ShapeRecord`** — the packed 32-byte `#[repr(C)]` table record (`keys`
+    first, so the record address is the keys slot). `ShapeDescriptor` stays
+    the by-value copy the rest of the runtime consumes, lifted from the
+    record; the copy's `record` field is the slab address.
+  - **`IdList`** — a 16-byte id list (three inline, a spilled `Vec` beyond)
+    that is the value of both remaining reverse indices: `by_facts`, keyed by
+    a 64-bit FNV fold of the six identity facts (every hit re-validates the
+    record, so a collision costs a second record read, never a wrong answer),
+    and `families`, keyed by keys address. The `ShapeFacts`-keyed map,
+    `indexed_keys` (the address a record was last indexed under) and the
+    per-scan `PROBE_MEMO` map are gone: the family index is the memo, and
+    `scan_shape_table_rekey_mut` now probes each keys address ONCE per
+    family — with the marking visit when any member is an old/cache
+    carrier, which is exactly the #8112 rooting duty.
+
+  A family is small by construction. A SHARED keys array is immutable
+  (mutation forks a private clone), so its descriptors differ only in the
+  birth bound, a semantic generation, the class kind, or a tombstone count.
+  An OWNED array grows in place, and until now every same-address append left
+  the predecessor descriptor alive until the array itself died
+  (`retain_key_count_versions`): a dictionary built by ten thousand appends
+  kept ten thousand prefix descriptors. **`publish_object_shape_from` now
+  retires that history behind the version its single owner carries**
+  (`retire_owned_shape_siblings`), after the successor is stamped and armed
+  (#9200's order), keeping any version an optimization cache permanently owns
+  (`cache_carrier`). Sound because `GC_FLAG_SHAPE_SHARED` is sticky: an
+  unflagged array has had exactly one owner for its whole life, a stale IC
+  token already misses on the stamp compare, and `shape_descriptor_by_id` of
+  a retired id is `None`. The one owner whose history IS reinstalled — an
+  Array-subclass receiver, whose tail-transition cache learns the
+  (predecessor, successor) pair right after the publish and stamps the
+  predecessor back on `pop` — keeps the old behaviour, gated on the receiver
+  class the learner itself is scoped to.
+
+  `PERRY_GC_CENSUS` rows: `shapes.ids_by_facts` / `shapes.ids_by_keys` are
+  replaced by `shapes.by_facts` / `shapes.families`; `shapes.descriptors` now reports the slab's
+  real bytes; new `shapes.ids_minted(process)`,
+  `shapes.descriptors.carried(live objects)` / `.uncarried` (with the
+  `cache_carrier` / `old_carrier` split) and `shapes.families.multi` /
+  `.largest` rows say how the population relates to the live heap.
+
+  `scripts/shape_descriptor_census.py` pins the new invariants (slab chunks
+  individually boxed and never reallocated; `keys` first in the record;
+  owned-history retirement scoped to the family, keeping cache carriers, and
+  ordered after the stamp) with sabotage self-tests for each.
+
+Measured on the compiled claude-code TUI (`cli_2.1.112.js`,
+  `PERRY_GC_CENSUS` via `SIGUSR2` at idle, the same compiled objects relinked
+  against the two runtimes, third census = after `shrink_shape_tables`):
+
+  | | before | after |
+  |---|---|---|
+  | live descriptors | 68,661 | 43,724 |
+  | `shapes.descriptors` | 9.40 MB | 3.82 MB |
+  | facts reverse index | 8.39 MB | 1.64 MB |
+  | keys-address reverse index | 2.67 MB | 0.89 MB |
+  | shape tables total | 22.22 MB (29.86 at the first census) | 8.11 MB (8.45) |
+  | RSS at census | 441 MB | 419 MB |
+
+  Of the remaining 43.7 k descriptors 8,664 are carried by a live object;
+  the rest are per-object semantic generations and shared-array prefix
+  versions whose keys array is still alive, which nothing prunes yet.
+
+  Benchmarks (min of 5): `bench_dynamic_property_keys` 38/12 → 37/12 ms,
+  `bench_populated_delete` 62 → 58 ms, `bench_shared_shape_delete` 45 →
+  40 ms; a 150,000-key dictionary built by appends 11.7 s → 0.23 s (on
+  `main`, `retain_key_count_versions` rebuilt the whole same-address id
+  list on every append), 2,000 × 200-key dictionaries 491 → 246 ms, and
+  200 k `{a,b,c}` literals with 20 hot read passes 103 → 47 ms.
+
+  Validation: `cargo test -p perry-runtime` 3113 passed (dev and release,
+  single-threaded); `scripts/run_lint_gates.sh` 64/64; a 233-test
+  shape/object/class/GC/JSON gap subset gives identical verdicts on both
+  arms (224 PASS, 2 pre-existing PARITY_FAIL).

changelog.d/9733-uncarried-shape-descriptors.md

@@ -0,0 +1,7 @@
+**Full collections now retire shape descriptors that no live object or
+restamping cache owns.** A full trace records every shaped receiver, and a
+synchronous sweep prunes only records absent from that complete census.
+Minor and budgeted cycles remain conservative. Generated-module ids and
+shape/transition caches retain exact ownership, while unstable transition
+entries validate their target before stamping. In the claude-code census,
+uncarried descriptors without an owner fell from 34,501 to zero.

crates/perry-runtime/src/fast_hash.rs

@@ -197,13 +197,13 @@ impl Hasher for FastKeyHasherImpl {
     // Integer writes fold one word per call instead of falling into `Hasher`'s
     // default `write_uN` -> `write(&n.to_ne_bytes())` byte loop.
     //
-    // This is what the shape table's `ids_by_facts` key pays for: `ShapeFacts`
-    // is six integer fields (two `u64`, three `u32`, one enum discriminant, an
-    // `isize`), so the derived `Hash` fed ~36 bytes -- ~36 serial multiplies --
-    // through the byte loop for a key that six folds mix just as well. That
-    // lookup runs on every shape publish (`shape_descriptor_ensure_with_holes`
-    // probes `ids_by_facts` before minting an id), i.e. on every object
-    // property add/delete that transitions a shape.
+    // This was written for the shape table's `ShapeFacts` key (six integer
+    // fields: two `u64`, three `u32`, one enum discriminant, an `isize`), whose
+    // derived `Hash` fed ~36 bytes -- ~36 serial multiplies -- through the byte
+    // loop for a key that six folds mix just as well. #9706 replaced that map
+    // with a pre-folded `u64` key (`object/shapes_store.rs::facts_key`), but
+    // the same shape of key remains on this hasher: `RegisteredTypedShapeKey`
+    // (`gc/layout/typed_shape.rs`) and the `(usize, String)` descriptor keys.
     //
     // `write_u8` is deliberately included even though it is exactly equivalent
     // to the byte path for a single byte: routing it here keeps every integer

crates/perry-runtime/src/gc/census.rs

@@ -346,6 +346,10 @@ struct Census {
     clo_captures: u64,
     // objects
     obj_meta: u64,
+    /// ShapeId stamp of every live shaped object (duplicates included; sorted
+    /// and deduplicated once the walk is over). Feeds the shape table's
+    /// "carried by a live object" rows (#9706).
+    live_shape_ids: Vec<u32>,
 }
 
 const SPACE_NAMES: [&str; 6] = [
@@ -432,7 +436,11 @@ impl Census {
             self.obj_meta += 1;
         }
         let live = match crate::object::shapes::object_shape_descriptor(obj) {
-            Some(d) => (d.live_inline_slot_count as usize).min(slot_capacity),
+            Some(d) => {
+                self.live_shape_ids
+                    .push(crate::object::shapes::object_shape_stamp(obj));
+                (d.live_inline_slot_count as usize).min(slot_capacity)
+            }
             None => {
                 entry.unshaped += 1;
                 0
@@ -764,7 +772,13 @@ fn take_census(label: &str, pass1: Option<Vec<usize>>) {
         })
         .collect();
 
-    let side: Vec<serde_json::Value> = side_tables()
+    c.live_shape_ids.sort_unstable();
+    c.live_shape_ids.dedup();
+    let mut side_rows = side_tables();
+    side_rows.extend(crate::object::shapes::shape_table_liveness_census(
+        &c.live_shape_ids,
+    ));
+    let side: Vec<serde_json::Value> = side_rows
         .into_iter()
         .map(|(n, e, b)| serde_json::json!({"table": n, "entries": e, "bytes": b}))
         .collect();

crates/perry-runtime/src/gc/cycle.rs

@@ -1710,7 +1710,10 @@ impl GcCycleState {
                 // per-object finalizers. Minor traces never mark the old
                 // generation, so deadness there is only trusted for
                 // untenured nursery headers.
-                .with_dead_collection_finalize(full_trace),
+                .with_dead_collection_finalize(
+                    full_trace,
+                    full_trace && !self.progress_kind.is_budgeted(),
+                ),
             );
         }
         let done = self

crates/perry-runtime/src/gc/dead_owner.rs

@@ -203,17 +203,27 @@ fn owner_type_matches(header: &GcHeader, expected_obj_type: Option<u8>) -> bool
 /// Post-trace fan-out (full mark-sweep + fallback minor). Runs at sweep
 /// entry, before any header is finalized or freed, so deadness probes read
 /// intact headers.
-pub(super) fn prune_dead_owner_side_tables_post_trace(full_trace: bool) {
+pub(super) fn prune_dead_owner_side_tables_post_trace(
+    full_trace: bool,
+    synchronous_full_trace: bool,
+) {
+    debug_assert!(!synchronous_full_trace || full_trace);
     if full_trace {
+        // Rebuild every restamping table's ownership before consulting the
+        // complete receiver census. An evicted cache entry releases its id in
+        // this same post-trace window.
+        crate::object::shape_carriers::recompute_after_full_trace();
+        if synchronous_full_trace {
+            crate::object::shapes::prune_uncarried_shape_descriptors_after_full_trace();
+        }
         // #8112: a full trace enumerated every live object, so the old-carrier
         // notes it accumulated are exactly the shapes old objects still carry.
         // Adopting them here is what lets the gate SHED a shape — minors only
         // ever add notes, so without this the table's root set would grow
         // monotonically and no keys array would ever be reclaimed again.
+        // #9726: this also clears the all-generation carried note after the
+        // synchronous prune consumed it. Budgeted traces only clear the note.
         crate::object::shapes::rotate_old_carrier_epoch_after_full_trace();
-        // The same rule for the array-tail transition caches: their carrier
-        // bits are exact only when rebuilt from live occupancy.
-        crate::object::array_tail_transition::recompute_cache_carriers_after_full_trace();
     }
     let probe = PostTraceProbe::new(full_trace);
     fan_out(

crates/perry-runtime/src/gc/layout_slot_visit.rs

@@ -51,6 +51,12 @@ pub(super) unsafe fn visit_gc_layout_slot_descriptors(
     // needed: without it the verifier aborts on a `slot_page_ever_dirty=false`
     // old→young edge through this word.
     let shape_keys_edge = if (*header).obj_type == GC_TYPE_OBJECT {
+        // #9726: unlike the minor-rooting gate below, full-trace descriptor
+        // liveness is generation-blind. Every reachable shaped receiver must
+        // note the exact id it carries before synchronous-full pruning.
+        if full_trace_active() {
+            crate::object::shapes::note_full_trace_carrier(child_slots.object_shape);
+        }
         // A receiver the minor will not enumerate for itself arms the table's
         // ephemeron gate. The test is "not in the nursery", not "in old-gen":
         // a `gc_malloc`'d large object and an immortal bootstrap resident are

crates/perry-runtime/src/gc/oldgen.rs

@@ -1200,12 +1200,19 @@ impl IncrementalSweepState {
     /// 2026-07-09 audit: buffers and typed arrays joined the same pattern —
     /// their registry/side-table entries are pruned when the owner is
     /// genuinely dead (full traces only; they are all tenured old residents).
-    pub(super) fn with_dead_collection_finalize(mut self, full_trace: bool) -> Self {
+    pub(super) fn with_dead_collection_finalize(
+        mut self,
+        full_trace: bool,
+        synchronous_full_trace: bool,
+    ) -> Self {
         // 2026-07-09 GC audit wave 2: death-prune the object-address-keyed
         // side tables in the same marks-fresh window. Cheap (one flag-check
         // walk over tables the root scanners already walk every cycle), so
         // it runs eagerly here rather than budget-chunked.
-        super::dead_owner::prune_dead_owner_side_tables_post_trace(full_trace);
+        super::dead_owner::prune_dead_owner_side_tables_post_trace(
+            full_trace,
+            synchronous_full_trace,
+        );
         self.dead_maps = crate::map::collect_dead_registered_maps_post_trace(full_trace);
         self.dead_sets = crate::set::collect_dead_registered_sets_post_trace(full_trace);
         self.dead_buffers = crate::buffer::collect_dead_registered_buffers_post_trace(full_trace);

crates/perry-runtime/src/gc/tests/dead_owner_side_tables.rs

@@ -1578,7 +1578,9 @@ fn test_live_transition_cache_entry_survives_full_gc() {
 
     let next_keys = crate::arena::arena_alloc_gc_old(64, 8, GC_TYPE_ARRAY) as usize;
     let live_key = crate::arena::arena_alloc_gc(64, 8, GC_TYPE_STRING) as usize;
-    let prev_shape_id = crate::object::shapes::shape_id_for_keys_ensure(std::ptr::null(), 0);
+    // This predecessor can recur from a generated module global; without such
+    // an owner, retiring it and the now-unusable cache entry is intentional.
+    let prev_shape_id = crate::object::shapes::js_object_shape_id_for_keys(0, 0);
     js_shadow_slot_set(0, ptr_bits(next_keys));
     js_shadow_slot_set(1, ptr_bits(live_key));