tri does not build from a clean checkout, and no workflow noticed (Refs #2176) - #2177
tri does not build from a clean checkout, and no workflow noticed (Refs #2176)#2177gHashTag wants to merge 15 commits into
Conversation
…Refs #2176) cargo build -p tri failed on main: dlc10 embeds a bitstream with include_bytes! that was committed to a feature branch and never to main. Nothing caught it because none of this repository's 35 workflows built the crate, while an autonomous loop kept committing into cli/tri/src/. tri rtl check runs the five structural checks t27.ai offers, locally. It names the yosys version beside the cell count, because that number is version-dependent: 0.33 reports 45 cells where 0.65 reports 49 on the same design, while wires and flip-flops agree exactly.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
) Eighteen active workflows across three repositories have never produced a green run, consuming 8182 runs between them. The largest are 1705, 1541 and 1224 runs with zero successes. This was a hand-run loop of gh api calls three times before it became a command. It reports and disables nothing: choosing between fix, workflow_dispatch-only and delete belongs to whoever owns the workflow. Verified against t27: 4 workflows, 1565 runs, matching the manual count.
|
Added Eighteen active workflows across trinity, trinity-fpga and t27 have never succeeded once — 8182 runs, zero green. Measured and filed as gHashTag/trinity#810. It reports and disables nothing. A |
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
) Written because of a specific failure of mine, not a hypothetical. The publisher for t27.ai failed six consecutive times, the site served hours-old content, and site-live-gate.yml caught it correctly and went red five times on my own commits. I read it zero times and found the outage by accident when a page I had just published returned 404. Detection was never the problem. Reading it was. Measured on trinity: 27 workflows red on main, several since February. A saturated streak prints as '30+' rather than '30', because the count is bounded by the page size — the same silent truncation this command exists to surface, which appeared in the command itself first.
|
Added Written after a failure of mine rather than a hypothetical one: the t27.ai publisher failed six consecutive times tonight, the site served hours-old content, and Two deliberate choices:
|
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Two vacuous checks turned up within an hour of each other. A workflow step read an 8-digit date out of `yosys -V`, which contains no date, so the guard skipped and the step passed without comparing anything. A verifier had a lookup table on both sides of the identity it tested, so flipping an entry cancelled with itself and left every claim green. Neither was found by reading. Both were found by changing a constant and noticing nothing went red. This automates that: perturb one numeric literal at a time, re-run the checker, report the ones it did not notice. Refuses to run on a file with uncommitted changes -- it rewrites the file and restores it, and an interrupted run must be recoverable with git. Dogfooded on the verifier that prompted it: 28 survivors, all but one in prose. Masking comments and strings cut that to 17, and the one real finding underneath was a quantiser compared only against itself, which would have made the whole inertness claim pass vacuously. Columns are reported because two identical literals on one line are otherwise indistinguishable; I read such a report, hand-checked the wrong one, and wrongly concluded the tool was lying.
|
Added a fourth command: Two vacuous checks turned up within an hour of each other today — a workflow step that read an 8-digit date out of tri mutate run --file conformance/phi_is_phi.py --cmd "python3 conformance/phi_is_phi.py"Dogfooded on the verifier that prompted it:
It refuses to run on a file with uncommitted changes: it rewrites the file and restores it, so an interrupted run has to be recoverable with Report rows carry a column. Two identical literals on one line were otherwise indistinguishable — I read such a report, hand-checked the wrong |
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
A measurement taken against a file this command left perturbed is not a measurement. That is not hypothetical: a perturbed constant survived a hand-run mutation on this machine, was read back as if it were the real value, and produced a written-up finding that did not exist -- an 'unreachable top binade' that the clean file does not have. Each mutant now re-reads the file after restoring it and fails loudly with the git command to recover, rather than continuing quietly.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
The clean-tree requirement was safe and was the wrong trade. Every mutation run on work-in-progress needed a throwaway commit first, and a throwaway commit is how a 'wip-for-mutation' subject reached a repository whose format gate rejects it -- twice in two iterations, the second time an hour after I wrote the lesson down. A sibling backup gives the same recovery guarantee without asking the caller to commit anything. Git cleanliness is still reported, because git checkout is the nicer recovery path when it is available, and the backup is removed on success. Verified on a deliberately dirty file: the run completes, the file comes back byte-for-byte, and no backup is left behind.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Verifying the source came back byte-for-byte was not enough. Most mutations here preserve the file's LENGTH -- 5 becomes 6, 16 becomes 17 -- and Python decides a .pyc is current by comparing the source's (mtime, size). Restore inside the same filesystem second and both match, so the interpreter serves bytecode compiled from the mutant. Measured, not theorised: a benchmark's format table read back as e5m11 and then e6m10 on consecutive runs while the file on disk said e5m10 both times. Clearing __pycache__ made every assertion pass. Second contamination of this kind in two iterations -- first the file itself, now its cache. The restore has to reach derived artefacts or the next measurement in that session is against a mutant nobody can see.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Written after merging a pull request whose language audit was red. The failure was in the list and I read a summary line I had written myself instead of the list. The gate was correct; I was not. The judgement that matters is not 'is anything failing' -- in these repositories something always is -- but 'is anything failing HERE that is not already failing everywhere else'. So it classifies every failure against the default branch and the last few merged PRs, and ends with one unambiguous verdict line. Pending checks produce WAIT rather than a verdict: a verdict computed from a partial list reads exactly like a complete one. Checked against the two real cases: #588 (safe) and #832, the one I merged red -- it names `checks` as the single failure unique to that PR and says DO NOT MERGE.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Written after getting the same task wrong twice in one session, both times producing a plausible number instead of an error. First: the stat parser matched `Number of cells:`, which yosys 0.33 prints and 0.65 does not. It found nothing and reported zero, and zero looks exactly like a small design. Second: the yosys output was captured into a shell variable and printed with `echo`. zsh's echo interprets backslash escapes and yosys writes identifiers with a leading backslash, so `\tern_node` became a tab and a `\c` truncated the stream. Every count came back zero again, from data that was correct on disk. Now the counts are parsed by cell NAME (which did not change between versions), both streams are read, the yosys banner is printed beside the numbers, and a run that never reaches a stat block is an error carrying yosys's own message rather than a row of zeros. Reproduces the hand-measured layer numbers exactly: 4299/0/254/220 and 3906/3/123/204. It also states what it does not measure. `ltp` is not a frequency substitute: it counts topological hops in netlists whose structure differs between designs, so it does not compare across them -- measured today, where it reported 213 hops for a one-adder scale path and 20 for a 32x16 multiplier.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
I merged a pull request while this command's own output said WAIT -- ten checks still running -- because the merge was in the same batch as the readiness check, so the verdict gated nothing. The polling loop that fed it had its own bug: it counted rows matching 'pending' and exited on zero. Zero rows also means the checks have not STARTED, which is exactly the state it hit. So: --wait blocks until the list is both non-empty and quiet, and an empty list is treated as 'not started' for four rounds before being believed. The verdict can no longer be computed against a list that has not appeared yet. Nothing broke on main this time. That was luck, not process.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
The combined form [[.check_runs[]|select(.status!="completed")]|length, [...]|length]|@TSV failed with 'expected an object but got: array' the first time --wait ran, on the very PR it was added to gate. A wait loop that errors out is worse than no wait loop: it turns a gate into an interruption. Two plain queries instead. Verified against gHashTag/trinity-fpga#592.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Written after two folds in one afternoon, both of which reported a number rather than an error. A layer whose operand memory had no write port: yosys propagated the never-written memory as constant and pruned the design. 12 logic cells for a 64-tap multiplier layer, and area flat across fan-in. The same layer with one memory location per lane: reading N per cycle is N read ports, so muxes rather than block RAM. 23052 cells on a 7680-cell part, zero RAM inferred. Neither is an error and both were caught only because a human found the number implausible. This checks the shape: sweep the parameter and say plainly when the area does not respond to it, or responds far too steeply. Both real cases are unit tests, with their real numbers. It stays quiet on proportional growth, because a detector that fires on the ordinary case stops being read.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
The first time --wait met a TLS handshake timeout it propagated the error, the caller's merge ran anyway, and the gate protected nothing -- the third time in this project a verdict has failed to gate. Now a failed poll is retried up to five times before giving up, and a failed FINAL read makes the verdict WAIT rather than an invented clean list. Unknown is not zero.
Inserting a new variant used `Red {` as the anchor, which put the new
command BETWEEN Red's doc comment and Red itself. Clap then read that
comment as the new command's description and left Red with none:
mutate What is failing on the default branch right now, and since
when. Find the constants in a checker that nothing checks
red (blank)
Four commands were added against that same anchor this session, so the
help text has been wrong since the first of them. Nothing behaved
differently -- it just described itself incorrectly, which for a set of
commands whose whole point is being readable is not a small thing.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
The verdict cannot gate anything if the caller puts `gh pr merge` in the same batch as this command. It prints WAIT, the merge runs anyway, and nobody reads the line. That happened four times in one session -- including on the pull request that added the waiting mode, and again on the one that made the wait survive network failures. Each time the fix addressed the tool and the next failure came from the same place: the human batching past it. So the action moves inside the verdict. With --merge, the command merges on 'safe to merge' and refuses on WAIT or DO NOT MERGE, telling the caller to re-run with --wait. There is no longer a gap to batch through.
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Three findings, one change.
1.
cargo build -p trifails on maindlc10embeds that bitstream withinclude_bytes!. It is committed onfeat/rename-tef-to-tnfand has never been on main, so the CLI cannot be builtfrom a clean checkout. Copying the file in by hand and rebuilding gives
rc=0with one unrelated warning — it is the only thing wrong on main. The asset is
added here.
2. Nothing builds this crate
This repository has 35 workflows and none of them references
cli/tri.That is why the above survived: there was no signal to be red. Meanwhile an
autonomous loop has been committing into
cli/tri/src/continuously.It also explains a second, separate breakage on
feat/rename-tef-to-tnf: tencannot find functionerrors infpga.rsforpvt_context_inside_envelope,synthetic_pvt_contextandcclk_period_ns. Those were defined in WaveLoops 443, 441 and 437 and later removed while their call sites stayed —
recoverable from history, not invented. Out of scope here and not fixed in this
PR.
cli-tri.ymlbuilds and tests the crate on every change undercli/, thenasserts the binary emits five verdict lines — because a binary that runs and
reports nothing would pass
buildandtestboth.3.
tri rtl checkThe five structural checks t27.ai offers, run locally in one command, so a
design can be checked without pushing and a CI number can be reproduced on
demand.
CI reported 45 cells, 57 wires, 17 flip-flops on the same design with Yosys
0.33. Wires and flip-flops agree exactly; the cell count differs by 9% — and
the report now names the tool beside the number, so the disagreement explains
itself instead of looking like a defect.
Four unit tests cover the flop counter in both yosys field orders and with
digit-bearing cell names like
$_DFFE_PN0P_, which an earlier[A-Z_]*pattern could not match. The verdict count is asserted at five, because a check
that silently does not run produces a shorter report that reads exactly like a
complete one.