Skip to content

tri does not build from a clean checkout, and no workflow noticed (Refs #2176) - #2177

Closed
gHashTag wants to merge 15 commits into
mainfrom
feat/tri-rtl-check
Closed

tri does not build from a clean checkout, and no workflow noticed (Refs #2176)#2177
gHashTag wants to merge 15 commits into
mainfrom
feat/tri-rtl-check

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

Three findings, one change.

1. cargo build -p tri fails on main

error: couldn't read `cli/dlc10/src/../../../fpga/tools/bscan_spi_xc7a100t_fgg676.bit`: No such file or directory
   --> cli/dlc10/src/lib.rs:129:39

dlc10 embeds that bitstream with include_bytes!. It is committed on
feat/rename-tef-to-tnf and has never been on main, so the CLI cannot be built
from a clean checkout. Copying the file in by hand and rebuilding gives rc=0
with one unrelated warning — it is the only thing wrong on main. The asset is
added here.

2. Nothing builds this crate

This repository has 35 workflows and none of them references cli/tri.
That is why the above survived: there was no signal to be red. Meanwhile an
autonomous loop has been committing into cli/tri/src/ continuously.

It also explains a second, separate breakage on feat/rename-tef-to-tnf: ten
cannot find function errors in fpga.rs for pvt_context_inside_envelope,
synthetic_pvt_context and cclk_period_ns. Those were defined in Wave
Loops 443, 441 and 437 and later removed while their call sites stayed —
recoverable from history, not invented. Out of scope here and not fixed in this
PR.

cli-tri.yml builds and tests the crate on every change under cli/, then
asserts the binary emits five verdict lines — because a binary that runs and
reports nothing would pass build and test both.

3. tri rtl check

The five structural checks t27.ai offers, run locally in one command, so a
design can be checked without pushing and a CI number can be reproduced on
demand.

$ tri rtl check ttsky26c/mini
RTL structural check — tt_um_qbrain_mini
  PASS sources resolve — every file info.yaml declares is present (1 of them)
  PASS elaborates — every module it instantiates is defined in the files given
  PASS no latch inferred — no latch inferred by this flow
  PASS synthesises — 49 cells, 57 wires, flattened — Yosys 0.65
  PASS has clocked logic — 17 flip-flops, so a clock frequency is a meaningful question

CI reported 45 cells, 57 wires, 17 flip-flops on the same design with Yosys
0.33. Wires and flip-flops agree exactly; the cell count differs by 9% — and
the report now names the tool beside the number, so the disagreement explains
itself instead of looking like a defect.

Four unit tests cover the flop counter in both yosys field orders and with
digit-bearing cell names like $_DFFE_PN0P_, which an earlier [A-Z_]*
pattern could not match. The verdict count is asserted at five, because a check
that silently does not run produces a shorter report that reads exactly like a
complete one.

…Refs #2176)

cargo build -p tri failed on main: dlc10 embeds a bitstream with
include_bytes! that was committed to a feature branch and never to main.
Nothing caught it because none of this repository's 35 workflows built
the crate, while an autonomous loop kept committing into cli/tri/src/.

tri rtl check runs the five structural checks t27.ai offers, locally. It
names the yosys version beside the cell count, because that number is
version-dependent: 0.33 reports 45 cells where 0.65 reports 49 on the
same design, while wires and flip-flops agree exactly.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

)

Eighteen active workflows across three repositories have never produced a
green run, consuming 8182 runs between them. The largest are 1705, 1541 and
1224 runs with zero successes.

This was a hand-run loop of gh api calls three times before it became a
command. It reports and disables nothing: choosing between fix,
workflow_dispatch-only and delete belongs to whoever owns the workflow.

Verified against t27: 4 workflows, 1565 runs, matching the manual count.
@gHashTag

Copy link
Copy Markdown
Owner Author

Added tri gates dead on the same branch, since it needs the same crate to build.

Eighteen active workflows across trinity, trinity-fpga and t27 have never succeeded once — 8182 runs, zero green. Measured and filed as gHashTag/trinity#810.

$ tri gates dead --repo gHashTag/t27
4 workflow(s) have never succeeded, across 1565 run(s).

    1224  gHashTag/t27   .github/workflows/auto-merge-ready-prs.yml
     166  gHashTag/t27   Coq kernel
     125  gHashTag/t27   OpenSSF Scorecard
      50  gHashTag/t27   Coq Proofs Validation

It reports and disables nothing. A --min-runs floor (default 50) keeps a new or rarely-triggered workflow from being called dead, because "0 successes over 2 runs" is not evidence — and an alarm that fires on non-evidence becomes the thing this command exists to find.

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

)

Written because of a specific failure of mine, not a hypothetical. The
publisher for t27.ai failed six consecutive times, the site served
hours-old content, and site-live-gate.yml caught it correctly and went red
five times on my own commits. I read it zero times and found the outage by
accident when a page I had just published returned 404.

Detection was never the problem. Reading it was.

Measured on trinity: 27 workflows red on main, several since February.
A saturated streak prints as '30+' rather than '30', because the count is
bounded by the page size — the same silent truncation this command exists
to surface, which appeared in the command itself first.
@gHashTag

Copy link
Copy Markdown
Owner Author

Added tri red now on this branch.

Written after a failure of mine rather than a hypothetical one: the t27.ai publisher failed six consecutive times tonight, the site served hours-old content, and site-live-gate.yml caught it correctly and went red five times on my own commits. I read it zero times, and found the outage by accident when a page I had just published returned 404. The detection was never the problem.

$ tri red now --repo gHashTag/trinity
27 workflow(s) red on the default branch:

    30+ in a row  since 2026-02-28T13:42  AGENT MU - Build & Deploy
    30+ in a row  since 2026-08-15T04:05  S³AI Brain CI — "Functional MRI" Gate
    30+ in a row  since 2026-04-03T20:56  docs-check
    ...

Two deliberate choices:

  • skipped is stepped over, not counted either way. It is not a verdict about the code — it means the workflow's condition was not met. Counting it as success ends a real streak early; counting it as failure invents one. Both were live mistakes here this week.
  • A saturated streak prints 30+, not 30. The count is bounded by the page size, so a full page is a lower bound — the same silent truncation this command exists to surface, and it showed up in the command itself before it shipped.

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

Two vacuous checks turned up within an hour of each other. A workflow
step read an 8-digit date out of `yosys -V`, which contains no date, so
the guard skipped and the step passed without comparing anything. A
verifier had a lookup table on both sides of the identity it tested, so
flipping an entry cancelled with itself and left every claim green.

Neither was found by reading. Both were found by changing a constant and
noticing nothing went red. This automates that: perturb one numeric
literal at a time, re-run the checker, report the ones it did not notice.

Refuses to run on a file with uncommitted changes -- it rewrites the file
and restores it, and an interrupted run must be recoverable with git.

Dogfooded on the verifier that prompted it: 28 survivors, all but one in
prose. Masking comments and strings cut that to 17, and the one real
finding underneath was a quantiser compared only against itself, which
would have made the whole inertness claim pass vacuously.

Columns are reported because two identical literals on one line are
otherwise indistinguishable; I read such a report, hand-checked the wrong
one, and wrongly concluded the tool was lying.
@gHashTag

Copy link
Copy Markdown
Owner Author

Added a fourth command: tri mutate.

Two vacuous checks turned up within an hour of each other today — a workflow step that read an 8-digit date out of yosys -V (there is no date in that string, so the guard skipped and the step passed without comparing anything), and a verifier whose lookup table sat on both sides of the identity it tested and cancelled with itself. Neither was found by reading. Both were found by changing a constant and watching nothing go red.

tri mutate run --file conformance/phi_is_phi.py --cmd "python3 conformance/phi_is_phi.py"

Dogfooded on the verifier that prompted it:

run survivors note
first 28 of 80 all but one in comments and docstrings — mutating prose cannot fail a check
after masking comments/strings 17 of 80 and one real finding underneath: a quantiser compared only against itself, which would have made the inertness claim pass vacuously

It refuses to run on a file with uncommitted changes: it rewrites the file and restores it, so an interrupted run has to be recoverable with git checkout.

Report rows carry a column. Two identical literals on one line were otherwise indistinguishable — I read such a report, hand-checked the wrong 5, and wrongly concluded the tool was lying. It wasn't: one of the two was caught and the other is semantically equivalent.

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

A measurement taken against a file this command left perturbed is not a
measurement. That is not hypothetical: a perturbed constant survived a
hand-run mutation on this machine, was read back as if it were the real
value, and produced a written-up finding that did not exist -- an
'unreachable top binade' that the clean file does not have.

Each mutant now re-reads the file after restoring it and fails loudly
with the git command to recover, rather than continuing quietly.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

The clean-tree requirement was safe and was the wrong trade. Every
mutation run on work-in-progress needed a throwaway commit first, and a
throwaway commit is how a 'wip-for-mutation' subject reached a
repository whose format gate rejects it -- twice in two iterations, the
second time an hour after I wrote the lesson down.

A sibling backup gives the same recovery guarantee without asking the
caller to commit anything. Git cleanliness is still reported, because
git checkout is the nicer recovery path when it is available, and the
backup is removed on success.

Verified on a deliberately dirty file: the run completes, the file comes
back byte-for-byte, and no backup is left behind.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

Verifying the source came back byte-for-byte was not enough. Most
mutations here preserve the file's LENGTH -- 5 becomes 6, 16 becomes 17
-- and Python decides a .pyc is current by comparing the source's
(mtime, size). Restore inside the same filesystem second and both match,
so the interpreter serves bytecode compiled from the mutant.

Measured, not theorised: a benchmark's format table read back as e5m11
and then e6m10 on consecutive runs while the file on disk said e5m10
both times. Clearing __pycache__ made every assertion pass.

Second contamination of this kind in two iterations -- first the file
itself, now its cache. The restore has to reach derived artefacts or the
next measurement in that session is against a mutant nobody can see.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

Written after merging a pull request whose language audit was red. The
failure was in the list and I read a summary line I had written myself
instead of the list. The gate was correct; I was not.

The judgement that matters is not 'is anything failing' -- in these
repositories something always is -- but 'is anything failing HERE that
is not already failing everywhere else'. So it classifies every failure
against the default branch and the last few merged PRs, and ends with
one unambiguous verdict line.

Pending checks produce WAIT rather than a verdict: a verdict computed
from a partial list reads exactly like a complete one.

Checked against the two real cases: #588 (safe) and #832, the one I
merged red -- it names `checks` as the single failure unique to that PR
and says DO NOT MERGE.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

Written after getting the same task wrong twice in one session, both
times producing a plausible number instead of an error.

First: the stat parser matched `Number of cells:`, which yosys 0.33
prints and 0.65 does not. It found nothing and reported zero, and zero
looks exactly like a small design.

Second: the yosys output was captured into a shell variable and printed
with `echo`. zsh's echo interprets backslash escapes and yosys writes
identifiers with a leading backslash, so `\tern_node` became a tab and a
`\c` truncated the stream. Every count came back zero again, from data
that was correct on disk.

Now the counts are parsed by cell NAME (which did not change between
versions), both streams are read, the yosys banner is printed beside the
numbers, and a run that never reaches a stat block is an error carrying
yosys's own message rather than a row of zeros.

Reproduces the hand-measured layer numbers exactly: 4299/0/254/220 and
3906/3/123/204.

It also states what it does not measure. `ltp` is not a frequency
substitute: it counts topological hops in netlists whose structure
differs between designs, so it does not compare across them -- measured
today, where it reported 213 hops for a one-adder scale path and 20 for
a 32x16 multiplier.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

I merged a pull request while this command's own output said WAIT -- ten
checks still running -- because the merge was in the same batch as the
readiness check, so the verdict gated nothing.

The polling loop that fed it had its own bug: it counted rows matching
'pending' and exited on zero. Zero rows also means the checks have not
STARTED, which is exactly the state it hit.

So: --wait blocks until the list is both non-empty and quiet, and an
empty list is treated as 'not started' for four rounds before being
believed. The verdict can no longer be computed against a list that has
not appeared yet.

Nothing broke on main this time. That was luck, not process.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

The combined form
  [[.check_runs[]|select(.status!="completed")]|length, [...]|length]|@TSV
failed with 'expected an object but got: array' the first time --wait
ran, on the very PR it was added to gate. A wait loop that errors out is
worse than no wait loop: it turns a gate into an interruption.

Two plain queries instead. Verified against gHashTag/trinity-fpga#592.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

Written after two folds in one afternoon, both of which reported a
number rather than an error.

A layer whose operand memory had no write port: yosys propagated the
never-written memory as constant and pruned the design. 12 logic cells
for a 64-tap multiplier layer, and area flat across fan-in.

The same layer with one memory location per lane: reading N per cycle is
N read ports, so muxes rather than block RAM. 23052 cells on a
7680-cell part, zero RAM inferred.

Neither is an error and both were caught only because a human found the
number implausible. This checks the shape: sweep the parameter and say
plainly when the area does not respond to it, or responds far too
steeply. Both real cases are unit tests, with their real numbers.

It stays quiet on proportional growth, because a detector that fires on
the ordinary case stops being read.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

The first time --wait met a TLS handshake timeout it propagated the
error, the caller's merge ran anyway, and the gate protected nothing --
the third time in this project a verdict has failed to gate.

Now a failed poll is retried up to five times before giving up, and a
failed FINAL read makes the verdict WAIT rather than an invented clean
list. Unknown is not zero.
Inserting a new variant used `Red {` as the anchor, which put the new
command BETWEEN Red's doc comment and Red itself. Clap then read that
comment as the new command's description and left Red with none:

  mutate   What is failing on the default branch right now, and since
           when. Find the constants in a checker that nothing checks
  red      (blank)

Four commands were added against that same anchor this session, so the
help text has been wrong since the first of them. Nothing behaved
differently -- it just described itself incorrectly, which for a set of
commands whose whole point is being readable is not a small thing.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

The verdict cannot gate anything if the caller puts `gh pr merge` in the
same batch as this command. It prints WAIT, the merge runs anyway, and
nobody reads the line.

That happened four times in one session -- including on the pull request
that added the waiting mode, and again on the one that made the wait
survive network failures. Each time the fix addressed the tool and the
next failure came from the same place: the human batching past it.

So the action moves inside the verdict. With --merge, the command merges
on 'safe to merge' and refuses on WAIT or DO NOT MERGE, telling the
caller to re-run with --wait. There is no longer a gap to batch through.
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@gHashTag

Copy link
Copy Markdown
Owner Author

Superseded by #2223 — same content, rebuilt with L1-traceable commits (Closes #2222 on each).

@gHashTag gHashTag closed this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant