test(gap): #9552 fixture that fails unfixed; changelog fragment for #9565

changelog.d/9565-cross-thread-promise-pin.md

@@ -0,0 +1,53 @@
+### Fixed
+
+- **A promise handed to native code is now rooted until it settles; `fetch`
+  (and ~110 other stdlib hand-offs) could be freed mid-flight (#9552).**
+  `claude -p <120,000-char argument>` compiled with perry died with
+  SIGSEGV in the microtask pump (`pump_protected`, `si_addr=0`) where node
+  exits 1; nondeterministic, ~50% of runs.
+
+  A promise minted by `js_promise_new_cross_thread` leaves the runtime as a
+  bare `usize` inside a worker future, which no root scanner visits, and
+  nothing on the JS side points **at** a pending promise whose only consumer
+  is an `await` — `P.on_fulfilled` and `P.next` are edges out of it. The
+  constructor's contract made the pin the caller's job; `spawn` and
+  `Atomics.waitAsync` took it, the stdlib's `fetch`/db/ws sites never did.
+  An old-generation reclaim at an allocation point ran its malloc sweep while
+  `js_fetch_with_options`'s promise was in flight and freed it (never
+  pinned, no token, still pending); mimalloc gave the 80-byte slot to a
+  `RegExp`; the stdlib pump resolved the stale address and the pump then
+  read `REGEXP_MAGIC` as the promise's `next`. The from-space quarantine
+  reports it as an unrelated fault because the object was never in the
+  arena. Diagnosed with a symbolized build and an env-gated trace of every
+  promise allocation, malloc-sweep free, pin and token event.
+
+  The constructor now owns the invariant: `js_promise_new_cross_thread`
+  pins the (malloc-resident, non-moving) promise itself — one flag bit,
+  through `pin_object_non_young` so the copying minor's young-pin latch is
+  never armed — and `js_promise_resolve` / `js_promise_reject` release it
+  with one byte test on a field in the padding after `state` (no other
+  field moves; an arena promise pays a predictable-branch load and nothing
+  else). `remove_token_from_registry` releases it too, so a native-async
+  token dropped without settling cannot leak its promise. The caller-side
+  pins in `spawn` and `waitAsync` are gone.
+
+  Every place a raw promise address re-enters the runtime from native code
+  — the stdlib pump, the native-async token pump, the `perry/thread`
+  result drain — now classifies it (`native_promise_from_raw`) and aborts
+  naming the site and the slot's current occupant, once per I/O completion,
+  never per `await`. `scripts/check_cross_thread_promise_provenance.py` is
+  a new `lint` step: it fails on an **arena** promise reaching a native
+  settlement sink or a spawned closure (shadowing- and alias-aware,
+  self-tested with three planted and three clean shapes). Unit coverage in
+  `promise/cross_thread_pin_tests.rs` (pinned at creation, released by both
+  settlements, survives `js_gc_collect()` while only an XOR-hidden integer
+  holds it, token-drop release, address classification); gap test
+  `test_gap_9552_cross_thread_promise_survives_gc.ts` fetches from a local
+  server that answers late while collections are forced in between.
+
+  Validation on the report's binary (`cli_2.1.112.js`, `--enable-wasm-runtime`,
+  same compiler, only the runtime archives swapped): 4/5 crashes → 0/12. The
+  gap test hangs 4/4 on the unfixed tree (an env-gated trace shows two of its
+  three fetch promises freed by malloc sweeps, then `js_promise_resolve` on a
+  reused slot) and prints node's output 5/5 fixed; `perry-runtime`'s suite is
+  3005 passed / 0 failed single-threaded.

test-files/test_gap_9552_cross_thread_promise_survives_gc.ts

@@ -1,44 +1,74 @@
 // #9552: a promise minted for a cross-thread settlement — every stdlib
 // `fetch` response, among ~110 stdlib call sites — is referenced only by the
-// worker's raw address while the request is in flight: the awaiting
-// continuation hangs OFF the promise (`P.on_fulfilled`), nothing on the JS
-// side points AT it. A full collection landing in that window freed it, and
-// the completion then resolved whatever the allocator had reused the slot for
-// (the report: a RegExp header read as a promise's `next`, SIGSEGV in the
-// microtask pump).
+// worker's raw address while the request is in flight: the consumer's
+// reaction hangs OFF the promise (`P.on_fulfilled`), nothing on the JS side
+// points AT it. A malloc sweep landing in that window freed it; the slot was
+// reused (in the report, by a RegExp header); and the completion then either
+// saw a "settled" state byte and dropped the response — the request never
+// resolves — or the microtask pump read the occupant as a promise (SIGSEGV).
 //
-// The constructor now pins the promise until it settles. This runs a request
-// against a local server that answers late, forces collections while it is in
-// flight, and expects the response to arrive. Node only exposes `gc` under
-// --expose-gc, so the collection is conditional and the expected output is
-// identical on both.
+// The constructor now pins the promise until it settles. Three consumer
+// shapes each start a request against a local server that answers late,
+// from a frame that has returned before any collection runs (so no native
+// stack slot still names the promise); the churn then trips the
+// malloc-count sweep with Symbols and reuses freed 80-byte slots with RegExp
+// headers (the promise's size class). Unfixed, this hangs: at least one of
+// the three never resolves. Node only exposes `gc` under --expose-gc, so
+// that call is conditional and the expected output is identical on both.
 import http from "node:http";
 
 declare const gc: undefined | (() => void);
 
 const server = http.createServer((_req, res) => {
   setTimeout(() => {
     res.end("ok");
-  }, 60);
+  }, 1500);
 });
 
-async function get(url: string): Promise<string> {
-  const response = await fetch(url);
-  return await response.text();
+class Client {
+  async request(url: string): Promise<string> {
+    const response = await fetch(url);
+    return await response.text();
+  }
 }
 
-server.listen(0, "127.0.0.1", async () => {
+server.listen(0, "127.0.0.1", () => {
   const address = server.address();
   const port = typeof address === "object" && address ? address.port : 0;
-  const inflight = get(`http://127.0.0.1:${port}/`);
+  const url = `http://127.0.0.1:${port}/`;
+  const results: Array<Promise<string>> = [];
 
-  let junk: Array<{ k: number; s: string }> = [];
-  for (let round = 0; round < 8; round++) {
-    junk = Array.from({ length: 4000 }, (_, k) => ({ k, s: "x".repeat(40) }));
-    if (typeof gc === "function") gc();
-    await new Promise((resolve) => setTimeout(resolve, 5));
-  }
+  setTimeout(() => {
+    // (a) no await at all: the reaction hangs off the fetch promise, nothing points at it
+    results.push(fetch(url).then((r) => r.text()));
+    // (b) async arrow
+    const viaArrow = async () => {
+      const r = await fetch(url);
+      return await r.text();
+    };
+    results.push(viaArrow());
+    // (c) async class method
+    results.push(new Client().request(url));
+  }, 0);
 
-  console.log(await inflight, junk.length);
-  server.close();
+  let round = 0;
+  const churn = () => {
+    for (let i = 0; i < 40000; i++) {
+      Symbol(`s${i & 255}`);
+    }
+    if (typeof gc === "function") gc();
+    for (let i = 0; i < 4000; i++) {
+      new RegExp(`r${i & 255}`);
+    }
+    round += 1;
+    if (round < 6) {
+      setTimeout(churn, 5);
+      return;
+    }
+    Promise.all(results).then((bodies) => {
+      console.log(bodies.join(","), round);
+      server.close();
+    });
+  };
+  setTimeout(churn, 20);
 });