Skip to content

fix(corpus-ratchet): remove the t27c test step; it was measured on the wrong tree - #2293

Merged
gHashTag merged 1 commit into
masterfrom
fix/ci/t27c-gate-landed-red
Aug 20, 2026
Merged

fix(corpus-ratchet): remove the t27c test step; it was measured on the wrong tree#2293
gHashTag merged 1 commit into
masterfrom
fix/ci/t27c-gate-landed-red

Conversation

@gHashTag

@gHashTag gHashTag commented Aug 20, 2026

Copy link
Copy Markdown
Owner

Refs #2292
Refs #2288
Refs #2289

Removes the cargo test -p t27c --release step added 34 minutes earlier in ce6ea628b (PR #2291). I landed that step, it went red on its first run on master, and this takes it back out.

Why it was wrong

ce6ea628b was justified by "1221 passed, 0 failed, 0 ignored". That measurement was real. It was taken in the wrong working tree — the host checkout sits on feat/wave-547/host-heapsort, while the gate runs on master, and the two are nowhere near each other:

feat/wave-547/host-heapsort origin/master
bootstrap/src/compiler.rs 22,142 lines 36,970 lines
#[test] in bootstrap/src 1,781 1,622
cargo test -p t27c main bin 895 passed, 0 failed 1602 passed, 13 failed, 2 ignored

Run that went red: https://github.com/gHashTag/t27/actions/runs/32330080992

A gate that lands red gets disabled rather than obeyed, so it comes out rather than being silenced, baselined in place, or "fixed" by editing tests. The 13 failing test names are recorded in #2292 along with the plan to re-land this as a ratchet.

The failure routing did work

Worth keeping on the record, because it is the one part that behaved as designed:

failure   Unit tests (cargo test -p t27c)
success   Explain a unit-test failure      <- correct explainer fired
skipped   Run the corpus ratchet
skipped   Explain a failure                <- ratchet explainer correctly did NOT fire

So the steps.ratchet.outcome scoping is kept. As a bare failure() it answers a failing Build t27c with "add an entry to docs/reports/suite_expectations.json", which is wrong independently of any test step. That is the only change from ce6ea628b that survives.

Diff against the pre-gate state

Against eeb779e4c (master before #2291), corpus-ratchet.yml differs only by the scoping fix and a comment block recording why the step must not be re-added without a measurement taken on master. The red step and its explainer are gone.

Honesty limits

  • 13 is a lower bound, not the failing count. cargo test stops at the first failing target, so the 21 test binaries in bootstrap/tests/ never ran on master. Their state is unmeasured. A ratchet baseline needs --no-fail-fast first.
  • This made things worse for 34 minutes, not merely no better. corpus-ratchet was already failing at Run the corpus ratchet on eeb779e4c, 57a53005a, 2255e4c32, bffd38982 and 400850702. A failing step in front of it short-circuited the job, so that pre-existing failure stopped being reported at all.
  • CORRECTED (see docs(now): retract "master has no branch protection" -- wrong endpoint #2294): corpus-ratchet is not a required check. This bullet originally read "master has no branch protection", citing GET /repos/gHashTag/t27/branches/master/protection404 Branch not protected. That endpoint sees only legacy branch protection and 404s for a repo protected by rulesets; the claim was wrong. Master IS protected by the active ruleset t27-master-protection — PRs required, deletion and non-fast-forward blocked, four required contexts: check-now-freshness, validate, check, check-linked-issue. corpus-ratchet is not among them (nor coverage, nor withdrawn-live), which is why all three sit red while PRs merge. ci(corpus-ratchet): run t27c's 1221 tests, which had never run on master #2291 still auto-merged at 03:57:13Z with its own corpus-ratchet run starting at 03:57:16Zthree seconds before the gate it was adding began to run — because that gate was never required. This sharpens the finding: the step was added to a workflow outside the required set, so even green it could never have blocked a merge.
  • This PR's own corpus-ratchet run will fail, at Run the corpus ratchet, for the pre-existing reason above. That is the state it is being returned to, not a new regression.
  • No test was edited, in either direction. bootstrap/src/math_compare.rs is compiled by nothing; its plateau test contradicts its own goldens #2290 (math_compare.rs compiled by nothing, plateau test 2.10x tighter than its own goldens) stands untouched.

…e wrong tree

ce6ea62 added `cargo test -p t27c --release` on the strength of "1221
passed, 0 failed". That measurement was real and was taken in a working
tree checked out at feat/wave-547/host-heapsort, not master. The two
trees are not close: bootstrap/src/compiler.rs is 22,142 lines there
against 36,970 on master. On master the same command is 1602 passed,
13 failed, 2 ignored, and the step went red on its first run.

A gate that lands red gets disabled rather than obeyed, so it comes out
rather than being papered over. 13 is a lower bound: cargo test stops at
the first failing target, so the 21 binaries in bootstrap/tests/ never
ran and are still unmeasured. It should return as a ratchet built from a
`--no-fail-fast` run on master. The failing names and that plan are in
#2292; #2288 and #2289 are reopened.

Adding a failing step in front of the ratchet also made things worse for
34 minutes: corpus-ratchet was already failing at `Run the corpus
ratchet`, and the short-circuit stopped that pre-existing failure from
being reported at all.

Kept: `Explain a failure` stays scoped to steps.ratchet.outcome. As a
bare failure() it answers a failing `Build t27c` with "add an entry to
suite_expectations.json", which is wrong independently of any test step.

Refs #2292
Refs #2288
Refs #2289
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@gHashTag
gHashTag merged commit 9fec1a9 into master Aug 20, 2026
20 of 23 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-20 04:04:52 UTC

Summary

Status Count
Total Open PRs 18
PRs with Failing Checks 4
PRs with All Checks Green 14
READY 7
FAILING 4
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=cbbfac87dff3 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants