The pattern
Three independent regressions broke claude --help — the primary real-app parity workload — on main within one week. Each shipped with its author's fixtures and unit suites fully green:
| break |
mechanism |
how it shipped green |
| #9169 window (fixed by #9247) |
method dispatch / property lookup regression |
perf-only on fixtures; only the bundle showed it |
| #9305 (fixed by #9323) |
latent setjmp slot-coloring; the window's commits merely introduced the throw that detonated it |
needs a JS throw inside a microtask — no small corpus had one |
| #9341 (reverted in #9345) |
#9326's indexed-write change; trigger is a bundle-scale esbuild shape |
88-fixture corpus + 64-check differential all green; the obvious __export mini-fixture passes — only the real bundle fails |
The common property: the failing shape exists only at cli.js bundle scale. Unit fixtures cannot be made to cover it by adding more unit fixtures — two of the three triggers resisted deliberate minimization by the people who found them.
The cost side
Each break voids every cc measurement taken against main until attribution completes (a ~40-minute compile per bisect step), and manufactures false blame: whoever next measures their branch on top of broken main sees their own binary fail. This week that cost several sessions repeated multi-hour attribution work — far more compute than the gate below would spend.
Proposal: label-gated, like run-extended-tests
A workflow job cc-parity that:
- Triggers on the existing opt-in pattern — a
run-cc-parity label (mirroring run-extended-tests), applied to any PR touching crates/perry-runtime/, crates/perry-codegen/, or crates/perry-hir/. (A paths-based auto-trigger is the stronger version, but the label form matches the repo's existing CI-budget conventions and can ship first.)
- Builds the toolchain, compiles a pinned
cli_2.1.112.js (the bundle is already the de-facto acceptance corpus; pin its sha256 so the gate is deterministic), runs --help and --version under HOME=<scratch>, and asserts: rc=0 and byte-identical stdout vs the pinned node oracle output (checked-in golden bytes — no node needed in CI, no network, no API key; the bundle exits before any credentialed call on these paths).
- Budget: one ~25-40 min job, only on labelled PRs. Against the backlog (~29 jobs/merge), this replaces the multi-session bisects each break currently costs.
The golden-bytes detail matters: --help output is 9,175 stable bytes; storing them makes the job hermetic and makes a diff in CI immediately actionable.
Evidence this would have caught all three
Not filing a workflow PR directly because gating policy and CI budget are the maintainer's call; happy to write the workflow if the shape above is acceptable.
The pattern
Three independent regressions broke
claude --help— the primary real-app parity workload — onmainwithin one week. Each shipped with its author's fixtures and unit suites fully green:__exportmini-fixture passes — only the real bundle failsThe common property: the failing shape exists only at cli.js bundle scale. Unit fixtures cannot be made to cover it by adding more unit fixtures — two of the three triggers resisted deliberate minimization by the people who found them.
The cost side
Each break voids every cc measurement taken against main until attribution completes (a ~40-minute compile per bisect step), and manufactures false blame: whoever next measures their branch on top of broken main sees their own binary fail. This week that cost several sessions repeated multi-hour attribution work — far more compute than the gate below would spend.
Proposal: label-gated, like
run-extended-testsA workflow job
cc-paritythat:run-cc-paritylabel (mirroringrun-extended-tests), applied to any PR touchingcrates/perry-runtime/,crates/perry-codegen/, orcrates/perry-hir/. (A paths-based auto-trigger is the stronger version, but the label form matches the repo's existing CI-budget conventions and can ship first.)cli_2.1.112.js(the bundle is already the de-facto acceptance corpus; pin its sha256 so the gate is deterministic), runs--helpand--versionunderHOME=<scratch>, and asserts: rc=0 and byte-identical stdout vs the pinned node oracle output (checked-in golden bytes — no node needed in CI, no network, no API key; the bundle exits before any credentialed call on these paths).The golden-bytes detail matters:
--helpoutput is 9,175 stable bytes; storing them makes the job hermetic and makes a diff in CI immediately actionable.Evidence this would have caught all three
--helpoutput/perf regression on the bundle — caught by the byte/rc check or its timing sibling.cc --helpsegfaults on main (NULL deref in run_microtasks) — clean at 6c880be77b, broken at 42d0f45685 and still broken at tip #9305:--helpSIGSEGV 3/3 deterministic — caught by rc.--helprc=1 with the TypeError on stdout — caught by both.Not filing a workflow PR directly because gating policy and CI budget are the maintainer's call; happy to write the workflow if the shape above is acceptable.