chore(specs): untrack specs/scratch -- 455 files, 578 MB, 64.5% of the tracked tree - #2283
Merged
Conversation
…e tree The repository owner asked for specs/scratch/ to stop being tracked. Added to .gitignore and removed from the index with `git rm -r --cached`. Nothing is deleted from any working tree. 455 files, 606,113,688 bytes at bffd389. 352 of them are one generated family (*_bench_*, 606,058,170 bytes); the other 103 are hand-written witnesses totalling 55,518 bytes. The tracked tree is 939,857,424 bytes, so this is 64.5% of it. The cost is accumulation: each wave committed its drafts on top of every prior wave's. Nothing outside scratch imports it (zero `use scratch` in the tree) and the corpus ratchet already excluded it by construction (bootstrap/src/suite.rs, is_scratch()). Two dependents move in this same commit: - 456 seals .trinity/seals/scratch_*.json are untracked with the specs. check_seal_coverage.py resolves spec_path against the filesystem, so on a fresh clone each would become dangling. None is in tools/seal_baseline.txt (0 of 209 lines), so leaving them would take Seal Coverage from 131 failures to 587. Removing them is the fix the gate's own message prescribes; baselining them would have written 456 lines of permanent debt. - 58 lines of tools/specs_generate_baseline.txt naming scratch specs are dropped (348 -> 290). The Emit Bit-Exact Gate enumerates via `git ls-files *.t27`, so the corpus drops 1196 -> 741 and those 58 would read as fixed, printing a false "58 spec(s) in the baseline now generate". The gate does not fail on fixed, so this is a truth fix, not a red one. Honesty limits, recorded in full in docs/NOW.md: - This does NOT shrink history. The blobs remain in git objects; clone size is unchanged. Only filter-repo plus a rewrite of every branch would reclaim it, at the cost of every open PR, fork, and recorded SHA. This stops growth from here. - Phase 3b gen-verilog yosys smoke silently falls 482 -> 27 targets on a fresh clone and Phase 3d/3e Icarus falls 155 -> 0, because collect_t27 returns an empty vec for a missing directory. Both print green; read them as untested. - 168 of the 455 files have no committed generator and exist only in history. - Seal Coverage is already red on master at 131 seals, unrelated to scratch. This change leaves it at exactly 131 -- measured against CI run 32324107310, whose failing set intersects this branch's exactly, with zero scratch seals in either. Closes #2282
Contributor
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This was referenced Aug 20, 2026
gHashTag
added a commit
that referenced
this pull request
Aug 20, 2026
…source (#2285) suite.rs has carried a non-empty floor for the ORACLE since W628 and never for the TARGETS. load_expectations returns Ok(None) for a missing ledger, documented as "never an empty ledger", and the ratchet turns that into "RATCHET: FAIL -- Absence is not amnesty (T31)". Ten lines away, collect_t27 returns Ok(Vec::new()) for a directory that is not there; every phase iterates it happily, run_phase returns (0, 0), and the phase prints "0 passed, 0 failed" into a suite that then prints ALL TESTS PASSED. No phase in run_comprehensive checked files.is_empty(), and no test anywhere asserted a phase's target count. It was already live. Untracking specs/scratch (#2283) took icarus-simulate and icarus-cocotb from 155 targets to 0 and gen-verilog-yosys-smoke from 482 to 27. 281 baselines are still tracked under .trinity/icarus-baselines/specs/scratch/ -- golden files whose subject no longer exists and whose comparing phase has no targets. Nothing said a word, and the ratchet could not: its 221 ledger entries include 0 under scratch. Adds require_targets(phase, source, files) and its single-file form require_target_file. Both name where the targets were supposed to come from, because every instance of this bug looks identical from the summary line and completely different at the source. Six call sites: the post-corpus-only spec lists (phases 1-6), phase 1c's gf16 spec (which reported "skipped (spec not found)" for a conformance gate that checked nothing), phase 3b's smoke list, phases 3d and 3e, the relative "specs" root shared by the phase 6 metrics, and phase 7's catalog gate -- which sat behind a bare "if cat.is_file()" with no else and printed "gate failures: 0" followed by "the catalog gate are all clean" for a gate that never ran. collect_t27 is deliberately unchanged: specs/scratch is legitimately absent since #2283, so the walker must keep returning an empty vec. Only the call site knows whether zero is a defect or a configuration. A test pins that division of labour. Four named opt-outs, all "the phase does not run": the standalone specs_scratch binding, phase 3b without yosys, phase 3d without its flag or iverilog, phase 3e without --cocotb. No CI effect today: corpus-ratchet.yml walks 650 corpus specs from the repo root, both fixed specs are tracked, and it installs neither yosys nor iverilog. Every guard passes. Running the suite with --icarus-simulate now fails, naming specs/scratch, which is the honest outcome. Does not restore the coverage the untrack removed, and does not detect a phase that shrank without reaching zero. Not compiled locally; suite.rs was parse-checked with rustfmt only. See docs/NOW.md. Closes #2284 Co-authored-by: Claude <claude@anthropic.com>
This was referenced Aug 20, 2026
gHashTag
added a commit
that referenced
this pull request
Aug 20, 2026
…ero (#2287) #2285 gave suite.rs a floor for the TARGET LIST. Two sites of the same shape on the INPUT side were left, and both reported a clean pass when what they read was missing or unreadable. The discipline was already in the file. load_expectations returns Ok(None) for a missing ledger and never an empty one -- "T31 is the bug where a gate treats 'no oracle' as 'pass'" -- and the ratchet turns that None into "RATCHET: FAIL -- Absence is not amnesty (T31)". Neither site followed it. A. Phase 7 catalog gate asked the working directory, not the tree. require_target_file closed the silent skip in #2285, but both operands stayed RELATIVE, so what the gate examined -- and whether the suite aborted -- depended on the cwd the binary was invoked from, and the failure named a relative path that cannot distinguish a wrong cwd from a lost file. Both are now joined onto the canonicalized repo root. The Err arm also still failed open: with the file proven present, reaching it meant the gate could not READ a file that exists, yet it printed one line, left gate_fail untouched, and the block below still printed "(lexer/parser conformance and the catalog gate are all clean)". It now bails. B. A corrupt baseline was byte-identical to a clean one. load_gen_verilog_smoke_baseline collapsed six conditions into HashSet::new() behind an eprintln!: missing file, unreadable file, invalid JSON, no expected_failures key, that key not an array, a non-string entry inside it. The caller set summary.baseline_failures = 0, which feeds summary.acceptable. The tracked baseline legitimately holds "expected_failures": [], so the fail-open path produced exactly the same output as the real file -- the right answer for the wrong reason on every run to date, which is why nothing caught it. It now returns anyhow::Result<Option<HashSet<String>>>: Ok(None) is absent, Err is present-but-unparseable, Ok(Some) is present-and-valid, and only the third may produce a count. That is load_expectations' signature and its reason. The caller decides in the open, and decides FAIL -- the file is tracked, so absence is a broken path, not a configuration. Tests: a_missing_baseline_is_none_not_an_empty_baseline is the counterpart, for the BASELINE, of a_missing_ledger_is_none_not_an_empty_ledger, which has kept absence out of the ORACLE since W628 while nothing kept it out of the baseline. a_corrupt_baseline_is_an_error_not_an_empty_baseline runs four corrupt payloads and asserts each is an Err naming the file. No criterion moved. CATALOG_ALLOWED, catalog_gate::run, the acceptable formula and the baseline file are untouched. With cwd == repo root -- the only configuration these gates have run in -- the resolved paths are identical and no finding changes. Effect on CI today: none. corpus-ratchet.yml runs from the repo root with both files present, so every guard passes and every printed number is identical. Honesty limits: this does not restore the 482 -> 27 coverage #2283 took from phase 3b, does not detect a phase that shrank without reaching zero, and phase 3b still cannot fail CI because it never calls record(...) -- a separate defect, unfixed. Not compiled: parse-checked with rustfmt only, so types and borrows are unproven until CI's cargo build. Closes #2286. Co-authored-by: Claude <claude@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2282
The repository owner asked for
specs/scratch/to stop being tracked.Added to
.gitignoreand removed from the index withgit rm -r --cached.Nothing is deleted from any working tree -- every file stays on disk for
anyone who has it, now ignored.
What this removes
Measured against the base commit
bffd38982withgit ls-tree -r -l:specs/scratch/*_bench_*The tracked tree at that commit is 939,857,424 bytes, so this is 64.5% of it,
and 99.99% of those bytes are one generated family. The cost was never one wave's
drafts -- each wave committed its own on top of every prior wave's.
Scratch is where a wave thinks out loud. It is not a source of truth: zero
use scratchappears anywhere in the tree, no spec outside it imports one, and thecorpus ratchet already excluded it by construction (
bootstrap/src/suite.rs,is_scratch()).Files changed: 914
Dspecs/scratch/**-- index onlyD.trinity/seals/scratch_*.json-- index onlyM.gitignore,docs/NOW.md,tools/specs_generate_baseline.txtDependents moved in the same commit
456 seals.
tools/check_seal_coverage.pyresolvesspec_pathagainst thefilesystem, and CI checks out a fresh clone -- so untracking the specs alone
would turn every
.trinity/seals/scratch_*.jsonintodangling. None is intools/seal_baseline.txt(0 of its 209 lines mention scratch). They areuntracked with the specs, which is the fix the gate's own message prescribes:
"the spec was committed and later deleted. Remove the seal with it, or restore
both." They were not baselined -- that would have written 456 lines of
permanent debt.
The two sets coincide exactly: every seal named
scratch_*has aspec_pathunder
specs/scratch/, and every seal with such aspec_pathis namedscratch_*. They cover all 455 specs (one spec,w825_bench_module_469x2p6_aos_var_call_write.t27, carries two seals).58 baseline lines. The Emit Bit-Exact Gate enumerates via
git ls-files *.t27, so the corpus drops 1196 -> 741 and 58 of the 348 lines intools/specs_generate_baseline.txtread asfixed, printingNOTE 58 spec(s) in the baseline now generate. That would be false -- they are gone, not fixed. All58 dropped (348 -> 290); each was verified to name a file in the untracked set.
The gate does not fail on
fixed, so this is a truth fix, not a red one.Gate impact, measured
Seal Coverage is already red on master and stays at exactly the same 131.
Run 32324107310 on
bffd38982-- the exact base of this branch -- reportsFAIL: 131 seal(s) newly do not hold, and the eight most recent master runs all failed.Those 131 are stale/dangling seals in
specs/tri/,specs/fpga/,compiler/and elsewhere; none has anything to do with scratch, and re-sealing them needs a
built
t27c.The delta was measured rather than assumed: CI's failing-seal name set from run
32324107310 was diffed against a run of the same script over this branch's tree.
0 seals fail on master that do not fail here, and 0 scratch seals fail here.
Without the seal removal the count would have gone 131 -> 587.
Corpus ratchet: unchanged, by construction.
docs/reports/suite_expectations.jsonholds 221 entries againstmax_entries: 221, none under scratch. The only difference is the log line741 of 1196->741 of 741.Honesty limits (BINDING)
objects forever.
git clonestill transfers all 578 MB and every existingclone is exactly the size it was. Only
git filter-repoplus a rewrite ofevery branch and tag would reclaim that space, and it would break every open
PR, every fork, and every commit SHA in the tree's own records. This stops
the growth from here. It reclaims nothing already paid for.
collect_t27returnsOk(vec![])for a missing directory rather thanerroring, so on a fresh clone Phase 3b gen-verilog yosys smoke falls from 482
to 27 targets (94.4%; this phase is not filtered by
--corpus-only, so itis the one real coverage loss), and Phase 3d/3e Icarus + cocotb falls 155 -> 0
(local-only; no workflow passes those flags). Both print green. Read them as
untested, not as passing.
generators
scripts/gen_w<NNN>.pyexist and are deterministic -- nonereferences
random,time,datetime,uuid, the environment, or thenetwork -- but they cover 279 of the 383 wave numbers present. The other 168
files exist only in git history. Recoverable, not reproducible.
.trinity/icarus-baselines/specs/scratch/is now unreachable in CI. It is20,248 bytes and removing it would also require editing 5 lines of
tools/json_parse_baseline.txt. Out of scope; it costs nothing and failsnothing.
before and after at the identical 131. It is not one of the four required
contexts (
check-now-freshness,check,validate,check-linked-issue), soit does not block merge -- worth saying out loud, because
docs/BRANCH-PROTECTION.mdlists it as required and the live ruleset does not.docs/CORPUS-RATCHET.md:96, the comment atsuite.rs:39, andcorpus-ratchet.yml:22quote 606,113,688 bytes as a livemeasurement of the tracked tree. Still true of history, no longer true of a
fresh checkout.