The withdrawal gate was green because it was under-scoped - #2181
Merged
Conversation
It scanned .md/.tex/.rst and reported OK on 975 documents, while three
machine-readable files still carried the withdrawn number -- including
specs/numeric/formats_catalog.t27:228, the canonical CATALOG row for gf16 that
feeds the published dataset. A gate that is green because it cannot see is the
same failure it was built to kill.
scope .md .tex .rst -> + .t27 .csv .json, minus conformance/vectors/
975 documents -> 4435
list 323 MHz -> + 330 MHz, 322 MHz (the same chain[19] net, read
on designs whose sizes differ by 62x, which is why
they agree to 2.5%)
Cleared what the widened gate then found: the canonical catalog row, the metrics
CSV, two self-citations in the competitor benchmark, and the .tex resource table
-- where the whole Max Freq column is now gone rather than one row blanked.
Two further live overclaims, found only because the scope widened:
97.67% MNIST MLP, 0.00% accuracy gap vs f32 -- nothing produces it. The only
MNIST run in the tree has every format at accuracy 0.1187, loss 2.3631, against
chance 0.1000 and ln(10)=2.3026. The model is untrained and one format scores
0.098, below chance. Identical scores across formats mean the scenario does not
measure the format; it is marked INVALID in place rather than deleted.
GF16 -- VERIFIED + FROZEN SILICON. STATUS.md:33 reserves SILICON for a received
die with written bring-up; STATUS.md:112 forbids the claim in t27 at all. No die
exists. Four other uses of the phrase meant the frozen tape-out design and are
disambiguated, not deleted -- they were ambiguous, not false.
fpga/HARDWARE_SSOT.md:1192 reports Fmax 322 MHz for a 70-LUT design, the same
value as the withdrawn reading on a 40,350-LUT one. That invariance is the tell,
but its clocking has not been re-checked, so it is marked UNVERIFIED rather than
asserted wrong.
Closes #2180
Refs #2179
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gHashTag
force-pushed
the
loop/t27-improve-2026-08-18
branch
from
August 18, 2026 10:44
5fc1360 to
aee0303
Compare
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tools/check_withdrawn_live.pyshipped scanning.md/.tex/.rstonly, and reportedOK … 975 documents scanned. It was green because it could not see the files that mattered most:specs/numeric/formats_catalog.t27:228— the canonical CATALOG row forgf16, which feeds the published dataset, still carriedFPGA 35/35 at 323 MHz Artix-7docs/metrics/numeric_formats_83_metrics.csv:12— same stringspecs/igla/coder/benchmark.t27:670,677— self-citation of the withdrawn figure in the competitor benchmarkA gate that is green because it is under-scoped is the same failure it was built to kill.
The list of withdrawn numbers was also too narrow: it held
323 MHzbut not330 MHzor322 MHz, which are the same ring-oscillator net read on designs whose claimed sizes differ by 62×.docs/arxiv-submission/trinity-gf16.tex:317-318still shipped both.Two further live overclaims found once the scope widened:
97.67 % MNIST MLP, 0.00 % accuracy gap vs f32(docs/NUMERIC_FORMATS_SSOT.md:226) — nothing in the tree produces this number. The only MNIST run present,conformance/gf_family_bench.json, has every format at accuracy 0.1187, loss 2.3631. Chance for 10 classes is 0.1000 and ln(10) = 2.3026, so the model is untrained; one format scores 0.098, below chance. Every format scoring identically to four decimals is itself the tell that the scenario measures nothing about the format.GF16 — VERIFIED + FROZEN SILICON—STATUS.md:33defines SILICON as "Physical die received and bring-up reported in writing. Asserted only on direct device evidence", andSTATUS.md:112states "No SILICON claim anywhere in t27." No die has been received.Refs #2179
What changed
.md .tex .rst→+ .t27 .csv .json; 975 → 4435 documents scanned+ 330 MHz, 322 MHz, 97.67% MNIST, FROZEN SILICONMax Freqcolumn in the.texconformance/gf_family_bench.jsonMNIST scenario marked INVALID in place, not deletedfpga/HARDWARE_SSOT.md:1192marked UNVERIFIED, not asserted wrong: 322 MHz on a 70-LUT design shares its value with the withdrawn reading on a 40,350-LUT one, but its clocking has not been re-checkedCloses #2180
🤖 Generated with Claude Code