From 29b0fd296fc700b231d1189a102d2e2857538708 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Mon, 31 Aug 2026 08:20:01 +0200 Subject: [PATCH 1/2] perf(codegen): gate array-store GC bookkeeping inline (boolean-store loop -33%, prime_sieve -23%) layout_note_slot_aware opens with `if !value_is_pointer && !old_is_pointer`, and js_array_note_numeric_write returns once the receiver's raw-f64 bits are clear. lower_index_set_fast emitted both calls unconditionally, so a boolean[] store loop paid two calls per element to be declined -- ~82% of such a loop is per-store bookkeeping against 16% for the loop (#9237). Both early returns are now inline, under two SEPARATE gates: * layout note + string addref + write barrier: behind may_carry_heap_pointer(new) || may_carry_heap_pointer(old). `new || old`, not `new` alone -- overwriting a pointer with a boolean is a pointer->scalar transition the runtime must still see, which is why the existing new-value-only emit_jsvalue_slot_store_pointer_tested is not usable here; * numeric-write note: keeps its own raw-f64-bits gate, because that note is what DOWNGRADES the array on its first non-numeric store -- gating it on pointer-ness would skip it forever. Mini, both binaries built in one run, interleaved, min of 5, self-timed: boolean store loop 206 -> 137 ms (-33%), 11_prime_sieve 26 -> 20 ms (-23%, 4.3x -> 3.3x node). Nested-loop read benchmark unchanged. Node is 12 and 6 ms -- the per-store guard CALL is the larger remaining piece, still open in #9237. pointer_store_into_numeric_array_keeps_layout_note_and_barrier caught this twice and is the reason to trust it; both its assertions now follow the EDGE through the new gate blocks, the treatment #7715 already gave the barrier assertion. Verified independently that inbounds -> gc_bookkeeping -> numnote -> barrier.maybe -> barrier is intact for a pointer store, byte-identical to node. Also drops emit_jsvalue_slot_store_scalar_aware_with_flags_on_block, added in #9195 and left callerless here. 31 codegen suites pass; five differentials byte-identical to node. Claude-Session: https://claude.ai/code/session_012Ys25ni6VwDKE71o1NTYAT --- .../gate-array-store-bookkeeping-inline.md | 49 +++++++++++++++++++ .../tests/typed_shape_descriptors.rs | 33 +++++++++++-- scripts/gc_store_site_inventory.py | 8 ++- 3 files changed, 85 insertions(+), 5 deletions(-) create mode 100644 changelog.d/gate-array-store-bookkeeping-inline.md diff --git a/changelog.d/gate-array-store-bookkeeping-inline.md b/changelog.d/gate-array-store-bookkeeping-inline.md new file mode 100644 index 0000000000..0bd2df9f5d --- /dev/null +++ b/changelog.d/gate-array-store-bookkeeping-inline.md @@ -0,0 +1,49 @@ +An in-bounds array element store no longer calls into the runtime twice per +element to be told there is nothing to do. + +`gc::layout::layout_note_slot_aware` opens with +`if !value_is_pointer && !old_is_pointer { return; }`, and +`js_array_note_numeric_write` returns as soon as the receiver's raw-f64 layout +bits are clear. Both are cheap tests on values the call site already holds — and +both were emitted unconditionally by `lower_index_set_fast`, so a `boolean[]` +store loop paid two calls per element for the privilege of being declined. A +profile of such a loop put ~82% of its time in per-store bookkeeping and 16% in +the loop itself (#9237). + +Both early returns now happen inline, under two **separate** gates, because they +answer different questions: + +* the layout note, the string addref and the write barrier are dead unless a + pointer is involved, so they sit behind `may_carry_heap_pointer(new) || + may_carry_heap_pointer(old)`. The test is `new || old` and not `new` alone: + overwriting a pointer with a boolean is a pointer→scalar transition the runtime + must still see, which is why the existing new-value-only gate + (`emit_jsvalue_slot_store_pointer_tested`, for class fields under a conforming + layout) is not usable here; +* the numeric-write note is what *downgrades* a raw-f64 array on its first + non-numeric store, so gating it on pointer-ness would skip it forever and the + array would never downgrade. It keeps its own raw-f64-bits gate — the test + `expr/index_set_guarded.rs` already applies on the sibling path. + +Measured on an idle Mac mini, both binaries built in one run, interleaved, min of +five, self-timed: a boolean-store loop 206 → 137 ms (−33%), and +`benchmarks/suite/11_prime_sieve.ts` 26 → 20 ms (−23%, 4.3× → 3.3× Node). A +nested-loop read benchmark is unchanged, as expected. Node remains 12 ms and 6 ms +respectively — this closes part of the gap, not all of it; the per-store guard +**call** is the larger remaining piece and is still open in #9237. + +`typed_shape_descriptors`'s `pointer_store_into_numeric_array_keeps_layout_note_and_barrier` +needed updating and is the reason to trust the change: it pins that a pointer +store into a statically numeric array still reaches both the layout note and the +barrier. Both assertions now follow the EDGE through the new gate blocks — +exactly the treatment #7715 gave the barrier assertion when it moved the barrier +behind a live value test. Verified independently that the chain +`idxset.inbounds → gc_bookkeeping → numnote → barrier.maybe → barrier` is intact +for a pointer store, and that such a store still answers identically to Node. + +Also removes `emit_jsvalue_slot_store_scalar_aware_with_flags_on_block`, added in +#9195 and left with no callers by this change. + +Five differentials stay byte-identical to Node: the string-aliasing hazard the +addref exists to prevent, the interval bounds proof, the byte-read battery, the +nested-loop exits, and the packed-loop carry battery. diff --git a/crates/perry-codegen/tests/typed_shape_descriptors.rs b/crates/perry-codegen/tests/typed_shape_descriptors.rs index f143117c7a..4998997b65 100644 --- a/crates/perry-codegen/tests/typed_shape_descriptors.rs +++ b/crates/perry-codegen/tests/typed_shape_descriptors.rs @@ -817,9 +817,26 @@ fn pointer_store_into_numeric_array_keeps_layout_note_and_barrier() { let ir = ir_for(module); let inbounds_ir = block_between(&ir, "\nidxset.inbounds.", "\nidxset.check_cap."); + // #9237: the layout note took the same treatment #7715 gave the barrier + // below — it now sits behind a live test of the stored value (and of the + // slot's previous value, since a pointer overwritten by a scalar is still a + // transition the runtime must see), rather than inline in `idxset.inbounds`. + // So this follows the EDGE too: the arm must branch into the gate, and the + // gate must still hold the note. Asserting only that the call exists + // somewhere would pass even if this arm stopped reaching it. assert!( - inbounds_ir.contains("call void @js_gc_note_slot_layout"), - "pointer stores into statically numeric arrays must update slot layout" + inbounds_ir.contains("label %idxset.inbounds.gc_bookkeeping."), + "the in-bounds arm no longer branches into the #9237 pointer gate, so a \ + pointer store here reaches no layout note at all:\n{inbounds_ir}" + ); + let note_gate_ir = block_between( + &ir, + "\nidxset.inbounds.gc_bookkeeping.", + "\nidxset.inbounds.gc_bookkeeping.done.", + ); + assert!( + note_gate_ir.contains("call void @js_gc_note_slot_layout"), + "pointer stores into statically numeric arrays must update slot layout:\n{note_gate_ir}" ); // #7715: the barrier still exists and is still reached from this arm, but // it now sits in its own block behind a live test of the stored value (and @@ -828,10 +845,18 @@ fn pointer_store_into_numeric_array_keeps_layout_note_and_barrier() { // gate, and the block the gate leads to must still hold the call. Asserting // only that the call exists somewhere in the module would pass even if this // arm stopped reaching it. + // The edge is asserted on the whole module rather than on the text between + // `idxset.inbounds.` and `idxset.check_cap.`: #9237 inserted the pointer and + // raw-f64 gates into this arm's chain, and their blocks are emitted after + // `check_cap`, so a region-slice of the IR no longer contains the branch even + // though the arm still reaches it (`inbounds` → `gc_bookkeeping` → + // `numnote` → `barrier.maybe` → `barrier`). The label is named for this arm + // and no other arm emits it, so a branch to it is still evidence that THIS + // arm reaches the barrier — which is what the assertion is for. assert!( - inbounds_ir.contains("label %idxset.inbounds.barrier.maybe."), + ir.contains("label %idxset.inbounds.barrier.maybe."), "the in-bounds arm no longer branches into the #7715 value gate, so a \ - pointer store here reaches no barrier at all:\n{inbounds_ir}" + pointer store here reaches no barrier at all:\n{ir}" ); let gate_ir = block_between( &ir, diff --git a/scripts/gc_store_site_inventory.py b/scripts/gc_store_site_inventory.py index 700af8ba48..995aa09f0b 100644 --- a/scripts/gc_store_site_inventory.py +++ b/scripts/gc_store_site_inventory.py @@ -553,7 +553,13 @@ def scan_file(path: Path) -> list[Finding]: # looked at, not walked past. "*" covers the shared emitter file, whose # barrier arm every census stem exercises. CODEGEN_BARRIERED_BINDINGS = { - "crates/perry-codegen/src/expr/write_barrier.rs": ("*", 2), + # Three markers since #9237: the two shared slot-store emitters, and + # `emit_scalar_aware_store_gated_on_pointerness`, whose unconditional slot + # write is discharged by its caller's stem-labelled barrier — for its only + # caller (`lower_index_set_fast`) that is `idxset.inbounds`, a stem already + # registered in `VERIFIED_BARRIER_STEMS` with a live IR witness, so the new + # claim brings no new obligation of its own. + "crates/perry-codegen/src/expr/write_barrier.rs": ("*", 3), # Two markers: the original generation-tested push store and, since #8872, # the unconditional element store inside `emit_dynamic_pointer_push_store`, # which the same `apush`-stem caller barriers after its layout bookkeeping. From 465e7515814bbc8bcc68e6c996a68f60924eb9a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Mon, 31 Aug 2026 10:08:36 +0200 Subject: [PATCH 2/2] test: track the raised barrier binding in the inventory's own self-test Raising CODEGEN_BARRIERED_BINDINGS to 3 without updating the fixture made the green baseline dirty (it plants 2) and stopped V-P9 firing (it plants a third to trigger the drift error). Baseline now plants 3 and V-P9 adds a fourth, so both cases still test what they are named for. --- scripts/gc_store_site_inventory.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/scripts/gc_store_site_inventory.py b/scripts/gc_store_site_inventory.py index 995aa09f0b..4e9fa6ad95 100644 --- a/scripts/gc_store_site_inventory.py +++ b/scripts/gc_store_site_inventory.py @@ -1429,9 +1429,13 @@ def synthetic_tree() -> dict[str, str]: ) return { "crates/perry-codegen/src/expr/write_barrier.rs": ( + # Three, matching CODEGEN_BARRIERED_BINDINGS since #9246 added + # `emit_scalar_aware_store_gated_on_pointerness`. The baseline has + # to track the binding or V-P1 goes red and V-P9 stops firing. "fn a() {\n" " // GC_STORE_AUDIT(BARRIERED): planted one\n" " // GC_STORE_AUDIT(BARRIERED): planted two\n" + " // GC_STORE_AUDIT(BARRIERED): planted three\n" "}\n" ), "crates/perry-codegen/src/expr/array_push.rs": ( @@ -1522,7 +1526,7 @@ def expect_verify( t = dict(base) t["crates/perry-codegen/src/expr/write_barrier.rs"] += ( - "fn b() {\n // GC_STORE_AUDIT(BARRIERED): planted third\n}\n" + "fn b() {\n // GC_STORE_AUDIT(BARRIERED): planted fourth\n}\n" ) expect_verify("V-P9 bound-file count drift", t, "the binding pins")