cargo test -p t27c landed red on master: the 1221-pass measurement was taken on the wrong tree

[gHashTag]

Reopens the substance of #2288 and #2289, which were closed by ce6ea628b on a measurement that was taken on the wrong tree.

What happened

ce6ea628b added a cargo test -p t27c --release step to corpus-ratchet.yml, justified by "1221 passed, 0 failed, 0 ignored". That measurement was real, and it was taken in the wrong working tree.

The host checkout at /Users/playom/t27 sits on feat/wave-547/host-heapsort, not master. The two trees are not close:

	feat/wave-547/host-heapsort	origin/master
bootstrap/src/compiler.rs	22,142 lines	36,970 lines
#[test] in bootstrap/src	1,781	1,622
cargo test -p t27c main bin	895 passed, 0 failed	1602 passed, 13 failed, 2 ignored

So the step went red on its very first run on master: https://github.com/gHashTag/t27/actions/runs/32330080992

The step is removed again in the PR that references this issue. It was live on master for about 34 minutes.

The 13 failing tests on master

Measured on ce6ea628b, ubuntu-latest, cargo test -p t27c --release:

compiler::tests_compiler_rejects::lowers_only_first_of_two_modules_characterization
compiler::tests_phase40_coverage::test_for_range_loop_unroll
compiler::tests_phase40_coverage::test_parse_for_range
compiler::tests_phase40_coverage::test_parse_for_range_c
compiler::tests_phase40_coverage::test_parse_for_range_nested
compiler::tests_phase40_coverage::test_parse_for_range_rust
compiler::tests_phase40_coverage::test_parse_for_range_verilog
compiler::tests_phase40_coverage::test_parse_for_range_with_expr
compiler::tests_w458::local_array_named_after_a_verilog_keyword_is_escaped
compiler::tests_w458::tests_w459::array_param_bound_from_test_block
compiler::tests_w458::tests_w459::test_block_emits_real_function_call
lex_conform::tests::lexer_matches_its_conformance_table
parse_conform::tests::parser_matches_its_conformance_table

This list is the main binary only and is therefore a lower bound. cargo test stops at the first failing target, so the 21 test binaries in bootstrap/tests/ never ran on master. Their pass/fail state is still unmeasured. A ratchet baseline must be built with --no-fail-fast.

What should happen next

Re-land it as a ratchet, not a gate:

1. Run cargo test -p t27c --release --no-fail-fast on a clean master checkout to get the complete failing set (the 13 above plus whatever the other 21 binaries contribute).
2. Record those names in a baseline file next to docs/reports/suite_expectations.json.
3. Fail only when a test outside the baseline fails, or a baselined test starts passing without the baseline being updated in the same commit.
4. Print the failing count openly in the step output, so the number is visible rather than hidden inside an exit code.

The underlying defect from #2288 is unchanged and still real: eight workflows on master run cargo build --release -p t27c, cargo build does not compile #[cfg(test)] modules, and formal-yosys.yml — which does run the tests — has never landed on master. 1,622 #[test] attributes on master have never run in CI, and at least 13 of them do not pass.

Two things this episode also exposed

corpus-ratchet is not a required check, so this step could never have gated anything.

Corrected 2026-08-20 — see #2294. An earlier version of this issue said master had no branch protection, citing GET /repos/gHashTag/t27/branches/master/protection → 404 Branch not protected. That endpoint reports only legacy branch protection and 404s for a repository protected by rulesets. That claim was wrong.

Master is protected, by the active ruleset t27-master-protection: PRs required, deletion and non-fast-forward blocked, and four required status contexts —

check-now-freshness
validate
check
check-linked-issue

corpus-ratchet is not one of them, and neither are coverage or withdrawn-live — which is exactly why all three sit red on master while PRs keep merging. PR #2291 auto-merged at 03:57:13Z while its own corpus-ratchet run started at 03:57:16Z: three seconds before the gate it was adding began to run.

This sharpens the finding rather than softening it: the step was added to a workflow outside the required set, so even had it been green it could never have blocked a merge. It was wrong on its numbers and wrong in its placement. corpus-ratchet.yml's W632 (BLOCKING) header stays aspirational until the context is added to the ruleset — which must follow the ratchet, not precede it, since requiring a currently-failing check would block every PR in the repo.

corpus-ratchet was already failing on master before any of this — Run the corpus ratchet has failed on eeb779e4c, 57a53005a, 2255e4c32, bffd38982, and 400850702. Adding a step before it made things actively worse for 34 minutes: the unit-test failure short-circuited the job, so the pre-existing ratchet failure stopped being reported at all.

Still open, unaffected by the revert

• bootstrap/src/math_compare.rs is compiled by nothing; its plateau test contradicts its own goldens #2290 — bootstrap/src/math_compare.rs is compiled by nothing; its plateau test contradicts its own goldens.
• bootstrap/Cargo.toml's [profile.release] is silently discarded because the workspace root has no [profile] section.