Skip to content

fpga-build's yosys loop could never have passed as written — 1/32 → 32/32 - #2216

Merged
gHashTag merged 15 commits into
masterfrom
loop/t27-bench-synth
Aug 19, 2026
Merged

fpga-build's yosys loop could never have passed as written — 1/32 → 32/32#2216
gHashTag merged 15 commits into
masterfrom
loop/t27-bench-synth

Conversation

@gHashTag

Copy link
Copy Markdown
Owner

#2214 documented fpga-build.yml red on master since 2026-08-08. Root-caused: as written, its yosys loop could never have passed — and behind it sat four real emitter bugs.

The workflow (two defects in one loop, one in a Rust step)

Measured locally, same loop as CI: 1 of 32 modules passed.

  1. Top derived from the filenamemac.v holds module ZeroDSP_MAC, so hierarchy -top mac fails for 31 of 32. The top is now read from the file itself.
  2. Missing -sv -DSIMULATION — the repo's own convention (suite.rs:859, verify_emit_bitexact.py:184): -sv because the emitter uses SV static casts; -DSIMULATION strips ifndef SIMULATION bench blocks whose $display carries non-constant args synthesis cannot evaluate (mac.v:535).
  3. The fpga-synthesis step calls yosys from Rust (t27c fpga-build --synth-only) with the same missing flags — fixed in both script templates.

Four emitter bugs the fixed loop then exposed

bug symptom fix
escape-before-flattening \cross _data_width — the space terminating an escaped identifier split the flattened name (clock_domain.v:221) escape applied to the flattened name, 2 sites
assertion keywords missing function [31:0] assume; — TOK_ASSUME under -sv (formal.v:187) assert assume cover restrict added to the keyword list
call site unescaped declaration emits \assume but the call emits assume(...) — two different identifiers call site uses the same escape
fn-return lvalue unescaped a Verilog function returns by assigning to its own name: assume = 0; (formal.v:195) both return paths escape

Progression, measured at each step

as written on master:            pass= 1 / 32
+ module name from file:         pass=13 / 32
+ -sv -DSIMULATION:              pass=30 / 32
+ four emitter fixes:            pass=32 / 32

t27c fpga-build --docker false --synth-only — the exact step that failed in CI — now exits 0 locally, producing a 9.4 MB synth.json.

Zero regression, by the repository's own gates

verify_emit_bitexact (Verilog == model, iverilog): ALL SYNTHESIZE, exit 0. verify_exhaustive: 8 primitives, model == C == Rust exhaustively, Verilog agrees everywhere it runs. verify_igla_race: exhaustive ternary_mul intact. check_specs_generate: 768/346, unchanged. M5 ceremony performed for the compiler.rs change.

And the ceiling

Every job in fpga-build.yml (8) and emit-bitexact-gate.yml now carries timeout-minutes: 45 — on 2026-08-18 a hung apt step ran 6h0m16s to the GitHub ceiling and read as a red PR whose own steps were all skipped.

Not fixed here, still open in #2214: fpga-formal's pip install sby (packaging).

Closes #2214

Closes #2215

🤖 Generated with Claude Code

#2214 documented fpga-build.yml red on master since 2026-08-08. Root cause,
measured locally with the loop exactly as CI runs it: 1 of 32 modules passed.

Two defects in the workflow loop, one more in the Rust synth path:

  the top was derived from the FILENAME, while mac.v holds module ZeroDSP_MAC --
  31 of 32 died at hierarchy. The top is now read from the file itself.

  read_verilog ran without -sv -DSIMULATION, against the repo's own convention
  (suite.rs:859, verify_emit_bitexact.py:184): -sv because the emitter uses SV
  static casts, -DSIMULATION to strip ifndef SIMULATION bench blocks whose
  $display carries non-constant args synthesis cannot evaluate (mac.v:535).

  t27c fpga-build --synth-only builds its yosys script in Rust with the same
  missing flags; both script templates fixed.

Four emitter bugs the fixed loop then exposed, each verified on the module that
tripped it:

  escape-before-flattening: \cross _data_width -- the space that terminates an
  escaped identifier split the flattened name in two (clock_domain.v:221). The
  escape now applies to the flattened name, at both sites.

  assert/assume/cover/restrict missing from the keyword list: function assume
  is TOK_ASSUME under -sv (formal.v:187).

  call sites did not escape what declarations escape: function \assume ;
  declared, assume(...) called -- two different identifiers.

  the fn-return lvalue was unescaped on both return paths: assume = 0;
  (formal.v:195).

Progression measured at each step: 1/32 as written; 13/32 with the name fix;
30/32 with the flags; 32/32 with the emitter fixes.

Zero regression by the repository's own gates: verify_emit_bitexact ALL
SYNTHESIZE exit 0; verify_exhaustive 8 primitives model == C == Rust exhaustive,
Verilog agreeing everywhere it runs; verify_igla_race intact;
check_specs_generate 768/346 unchanged. M5 ceremony performed.

Every job in fpga-build.yml and emit-bitexact-gate.yml now carries
timeout-minutes: 45 -- on 2026-08-18 a hung apt step ran 6h0m16s to the GitHub
ceiling and read as a red PR whose own steps were all skipped.

Left open in #2214: fpga-formal's pip install sby packaging.

Closes #2215
Closes #2214

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 00:10:24 UTC

Summary

Status Count
Total Open PRs 29
PRs with Failing Checks 12
PRs with All Checks Green 17
READY 9
FAILING 12
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

…pdb was zeroes

Two more structural impossibilities in the same workflow, found after the yosys
loop landed:

  the Build nextpnr-xilinx step cloned YosysHQ/nextpnr, which does not contain a
  xilinx architecture -- cmake fails with "Architecture 'xilinx' not in list".
  The xilinx arch lives in the openXC7/nextpnr-xilinx fork. The step now clones
  that, builds nextpnr-xilinx and bbasm, and caches both.

  the Create chipdb symlink step wrote 1 MB of /dev/zero and called it a chipdb.
  A bitstream placed-and-routed against zeroes is not a bitstream, and the
  artefact upload and flashing guide downstream were outputs shaped like results
  with nothing inside -- the same class as the 323 MHz figure. The database is
  now exported for real (bbaexport + bbasm for xc7a100tcsg324-1, the part t27c
  actually expects at build/fpga/chipdb/) and cached.

fpga-bitstream's ceiling is 90 minutes rather than 45: the first uncached run
builds the toolchain and exports the chipdb; the cache makes later runs minutes.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 00:23:27 UTC

Summary

Status Count
Total Open PRs 29
PRs with Failing Checks 12
PRs with All Checks Green 17
READY 9
FAILING 12
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

The corrected clone reached cmake, which then failed with 'Could NOT find Boost
(missing: filesystem thread program_options iostreams)' -- libboost-dev provides
headers only. libboost-all-dev + libeigen3-dev, per the openXC7 recipe proven in
trinity-fpga.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 00:37:28 UTC

Summary

Status Count
Total Open PRs 29
PRs with Failing Checks 12
PRs with All Checks Green 17
READY 9
FAILING 12
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

The nextpnr-xilinx build now SUCCEEDS in CI for the first time in this job's
history -- '[100%] Built target nextpnr-xilinx', 'Built target bbasm' -- and the
step then died on my own cp path: an in-tree cmake build places both binaries at
the build root.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 00:48:44 UTC

Summary

Status Count
Total Open PRs 29
PRs with Failing Checks 12
PRs with All Checks Green 17
READY 9
FAILING 12
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Each fixed step reveals the next in a pipeline that has never run end to end.
prjxray was cloned bare; its yaml-cpp and friends are submodules, so cmake died
on 'target yaml-cpp not built by this project'. --recursive --shallow-submodules.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 01:03:15 UTC

Summary

Status Count
Total Open PRs 29
PRs with Failing Checks 12
PRs with All Checks Green 17
READY 9
FAILING 12
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

prjxray now BUILDS in CI -- '[100%] Built target xc7frames2bit' -- and the step
died on the original cp path, one more line that had never executed. The linker
line itself says tools/CMakeFiles/xc7frames2bit.dir.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 01:17:17 UTC

Summary

Status Count
Total Open PRs 29
PRs with Failing Checks 12
PRs with All Checks Green 17
READY 9
FAILING 12
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

cp: cannot create regular file '~/prjxray-build/': Not a directory -- the
destination was never created. One more line of this job that had never executed
before this PR.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 02:45:42 UTC

Summary

Status Count
Total Open PRs 30
PRs with Failing Checks 13
PRs with All Checks Green 17
READY 9
FAILING 13
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

…clone

The step's if-branch cd's into ~/nextpnr-xilinx, so the final relative
'cp ... build/fpga/chipdb/' pointed inside the toolchain clone rather than the
workspace -- my own bug this time. The important part of the log: bbaexport and
bbasm COMPLETED, so the first real chipdb this job has ever had now exists and is
cached; the next run is a cache hit plus this one-line copy.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 03:22:44 UTC

Summary

Status Count
Total Open PRs 30
PRs with Failing Checks 13
PRs with All Checks Green 17
READY 9
FAILING 13
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

…o 90

Layer 8: synthesis against the real chipdb SUCCEEDED (synth.json produced) and
t27c then looked for nextpnr-xilinx at its hardcoded build/ path; ours lives in
~/.local/bin. The CLI offers --nextpnr, so pass it.

And a correction to my own guard: the 45-minute ceiling I added against 6-hour
hangs killed a HEALTHY emit-bitexact run at 45m15s -- the cold-cache run
legitimately takes about that long. 90 now. A ceiling must clear the honest
worst case, not the median.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 04:10:23 UTC

Summary

Status Count
Total Open PRs 30
PRs with Failing Checks 13
PRs with All Checks Green 17
READY 9
FAILING 13
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

Layer 10, and the deepest point yet: nextpnr-xilinx now RUNS against the real
xc7a100tcsg324 database -- tiles load, placement starts -- and rejects the
hardcoded minimal-XDC pin C18 outright: the device has no such pin. These pins
had never been evaluated before, because every earlier run used a chipdb made of
zeroes.

Arty A7-100T (xc7a100tcsg324): clk = E3 (was already right), ck_rst = C2,
uart_txd_in (host->FPGA) = A9, uart_rxd_out (FPGA->host) = D10.

Verified by regenerating the XDC locally and reading the emitted pins.

Refs #2215

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 04:36:11 UTC

Summary

Status Count
Total Open PRs 30
PRs with Failing Checks 13
PRs with All Checks Green 17
READY 9
FAILING 13
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

gHashTag added a commit that referenced this pull request Aug 19, 2026
#2220)

ci-gates section 9, distilled from the ten-layer fpga-bitstream campaign
(#2215/#2216): every step past the historical failure point had never executed,
each was a fresh landmine at one CI round-trip apiece.

The key finding: a fake artefact masks real bugs downstream of it. The job's
chipdb was 1 MB of /dev/zero, and a zeroed database cannot reject a wrong pin --
so the emitted XDC's pin C18, which the device does not have, survived every
prior run unnoticed. Same class as the ring-oscillator MHz figure, in
infrastructure rather than a paper.

Rules recorded: dry-run the whole job's shell locally, not just the part being
fixed (the three layers I tested locally were found before CI; the six I did not
each cost a round-trip); treat every line after the historical failure point as
unreviewed code; assume everything downstream of a fake artefact is unvalidated,
including constants that look unrelated; edit workflow YAML by line number and
re-validate with a parser; and a timeout ceiling must clear the honest worst
case, not the median -- mine killed a healthy 46-minute run at 45.

Closes #2219
Refs #2215

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 07:36:02 UTC

Summary

Status Count
Total Open PRs 30
PRs with Failing Checks 13
PRs with All Checks Green 17
READY 9
FAILING 13
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 11:26:00 UTC

Summary

Status Count
Total Open PRs 30
PRs with Failing Checks 13
PRs with All Checks Green 17
READY 9
FAILING 13
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

First run ever to reach P&R died one layer later: t27c hardcodes
build/fpga/prjxray (clone + PYTHONPATH), build/fpga/prjxray/build/tools/
xc7frames2bit, and build/nextpnr-xilinx/xilinx/external/prjxray-db/artix7
-- while the job staged all three under ~/. Repoint the db clone, stage the
prjxray checkout + binary where the driver looks, and install fasm2frames'
real import chain (fasm, simplejson + friends; verified by executing
--help against exactly this package list, not against requirements.txt,
which pins editable submodule paths and py3.12-incompatible tools).

Refs #2215.
t27c emits build/fpga/zerodsp_top.bit for the minimal profile; the metrics
step demanded a file literally named bitstream.bit and failed the first run
in the job's history to produce a real bitstream (3,822,704 bytes, exactly
an xc7a100t configuration). Glob *.bit instead, print the real name, and
drop the second upload step that reused the same artifact name -- v4 errors
on duplicates, so it only ever passed because the first upload found nothing.

Refs #2215.
@gHashTag
gHashTag force-pushed the loop/t27-bench-synth branch from 466f02a to 53365e1 Compare August 19, 2026 12:05
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 12:10:52 UTC

Summary

Status Count
Total Open PRs 30
PRs with Failing Checks 13
PRs with All Checks Green 17
READY 9
FAILING 13
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=428770cb000d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

# Conflicts:
#	bootstrap/src/compiler.rs
#	bootstrap/stage0/FROZEN_HASH
#	docs/NOW.md
@gHashTag
gHashTag enabled auto-merge (squash) August 19, 2026 13:02
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-19 13:02:38 UTC

Summary

Status Count
Total Open PRs 29
PRs with Failing Checks 12
PRs with All Checks Green 17
READY 9
FAILING 12
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=54d19991b0c2 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant