fix(tri): an unreadable history is not a zero, and the guard now reaches the command - #2367
Merged
Merged
Conversation
…hes the command Repairs three defects #2360 merged into master. Its ranking idea is right and its other measurements reproduce exactly, so nothing is reverted. 1. --strict failed open on unknown history. gh() bails on any non-zero exit, so a rate limit or a 502 became "", num("") returned -1, and -1 lost both > 0 tests it met: the hole dropped out of the strict count AND the verdict ladder printed "has never run anywhere at all" about a workflow nobody had measured. Counts are Option<i64> now and None propagates to the exit code. A gate that has never run anywhere (0 runs, 0 failures) is no longer exempt either. The one exemption that was earned survives: a gate that HAS run and has never once concluded failure. The same swallow one level up is closed too: an unreadable default-branch run count used to mean "has a baseline", which deleted the finding. A 404 from the contents API stays a real answer -- 12 of this repo's 54 registered active workflows have no file on master -- so Fetch::Absent is kept apart from Fetch::Failed. 2. The guard constrained std::slice::sort_by, not tri ci baseline. It copied the production sort and the loud predicate into its own body. Inverting the PRODUCTION sort survived (3 passed; 0 failed); so did replacing `if strict && loud > 0` with `if false`. rank, severity, verdict, alarming, strict_verdict, count and is_not_found are lifted out of the command body and the tests call them. Both mutants now die. 3. emit-bitexact was never one of #2309's four holes. It has had a master baseline since 2026-08-20T01:05:13Z, 41 hours before #2360 opened, and #2309 says so in its own body. The claim is corrected in the module doc, the test doc-comment and its fixture; the NOW entry is a dated record, so a correction is appended rather than the claim edited away. Latent, not live: cibase.rs is the only reference to `ci baseline` in the tree, so no job changes colour today. Closes #2361
Contributor
PR DashboardGenerated at: 2026-08-21 20:33:04 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
gHashTag
enabled auto-merge (squash)
August 21, 2026 20:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repairs the three defects #2360 merged into
masteratcb446ea128c7b60cee6e100a49300d227455993d. Nothing is reverted. #2360's ranking idea is right and its other measurements reproduce exactly against the live API:check-now-freshness1446/2,seal-staleness-warn169/0,loop-tools-gate12/0,catalog-count-gate37/0.Everything below was re-verified against
origin/masteratdfdc7ed642970365e91cd67f17ebfa2eb772eb4b, not taken from the review.1.
--strictfailed open on unknown history, and said so out loudOn master,
cli/tri/src/cibase.rs:gh()bails on any non-zero exit, so a rate limit, a 502 or an expired token becomes"", andnum("") == -1. That-1then loses both> 0tests it meets:let loud = holes.iter().filter(|h| h.fails > 0).count();(line 220) — the hole drops out, so--strictexits 0 where the pre-fix(tri): rank baseline holes by failure history, not by trigger table (Closes #2359) #2360 version exited non-zero. One transient API failure downgrades every hole in the sweep at once.fails > 0, thenruns > 0, and printshas never run anywhere at all— a false statement of fact about a workflow whose history could not be read.An unreadable history is not a zero. Counts are
Option<i64>now;Nonemeans "the API did not say" and propagates all the way to the exit code. The verdict for it ishistory could not be read — UNMEASURED, which is not the same as clean.runs == 0, fails == 0— a gate never run anywhere, the strongest form of "no green state has ever been observed" — was also exempt from--strictand no longer is. The one exemption that was actually earned survives: a gate that HAS run, often, and has never once concludedfailure.The same swallow existed one level up and is closed too. An unreadable default-branch run count used to mean "this workflow has a baseline", which deleted the finding outright. Unreadable workflows are now printed under
workflows the sweep could not measureand counted toward--strict. Deliberate exception, measured: a 404 from the contents API is an answer — 12 of this repository's 54 registered active workflows have no file onmasterat all, and a workflow that is not on the default branch gates nothing there.Fetch::Absentkeeps that apart fromFetch::Failed.The doc comment was left untouched by #2360 and had been false since it merged:
/// Non-zero exit if any gate has no baseline. For use in CI.Replaced with what
--strictactually does.2. The guard tested
std::slice::sort_by, nottri ci baselinea_gate_that_has_never_failed_ranks_below_one_that_hascopied production line 194 into its own line 293 and production line 220 into its own line 302, then asserted against its own local vector.BITING — two mutants planted in master's shipped file, both survived
Mutant A — invert the production sort at line 194, leave the test's copy at line 293 alone:
Mutant B —
if strict && loud > 0becomesif false, making--stricta permanent no-op:ALIVE — the assertion is live, it is just aimed at its own copy
A surviving mutant is a lead, not a verdict, so it was composed with a genuine break: invert the test's own copy at line 293, production untouched.
So Mutant A is not an equivalent mutant. The assertion responds to a sort inversion — just not the production one.
TRUE — and the decisive check
struct Holeplus the guard's verbatim body, lifted into a standalone file containing no production code at all, compiles and passes:A guard that cannot reach the code it claims to guard is not a guard.
The repair, and the same mutants dying
rank,severity,verdict,alarming,strict_verdict,countandis_not_foundare lifted out of the command body, and the tests call them.Mutant A′ (production
rankinverted) now kills two tests:Mutant B′ (
strict_verdictforced toNone, the exact analogue ofif false):The same production-free extraction now yields 38 compile errors naming 10 distinct production items (
rank,severity,verdict,alarming,strict_verdict,count,is_not_found,is_pr_gated,decode_b64, and theSeveritytype) and no binary.Consumption proved separately
Unit tests calling a function do not prove the command calls it. Repaired build against the live API:
Same build with production
rankinverted andstrict_verdictforced toNone:baseline()really does call both.3. #2360's central motivating fact was false, and it is in master's source
The PR body's table puts
emit-bitexact-gateat the head of "the four holes" with 144 runs, 84 failures, no baseline. The "no baseline" half was not true when it was written:That is
total_count: 1, aworkflow_dispatchonmaster41 hours before #2360 opened (2026-08-20T01:05:13Zvs2026-08-21T18:05:14Z). #2309 says so in its own body:And #2360's own pasted output listed 2 holes and did not include it, contradicting its table three lines above. The repaired command reproduces that live: the two holes today are
check-now-freshnessandseal-staleness-warn.#2309's real four were
catalog-count-gate,check-now-freshness,loop-tools-gate,seal-staleness-warn, measuring 37/0, 1446/2, 12/0 and 169/0. Exactly one has ever concludedfailure, and it did so twice. The ranking argument still holds — it just rests on 2 failures versus 0, not 84 versus 0.The run and failure counts themselves are real; live now they are 147/88.
The falsehood was in three places, all corrected:
cibase.rsmodule doc,docs/now/2026-08-22-tri-ci-baseline-ranks-by-failure-history-not-by-trigger-tabl.md,Hole { file: "emit-bitexact-gate.yml", … }fixture asserted to rank first.The NOW entry is a dated record, so the correction is appended under its own heading and the original two bullets are left standing verbatim, with a line saying nothing above was edited.
docs/NOW.md:1517also mentionsemit-bitexactwith "0 master runs ever", but that entry is dated 2026-08-20 and was true when written, so it is left alone.Severity: latent, not live
The only reference to
ci baselineanywhere in the tree iscibase.rsitself — no workflow invokes it, so no job changes colour today. This is not a live CI failure and should not be read as one. It matters because the flag says "For use in CI" and would have been wired up in the fail-open state.Verification
cargo test -p tri— 173 passed; 0 failed.cibasealone: 7 passed, up from 3.Cargo.lockis unchanged.Closes #2361