From 934884e59f232b12486feb52772914f8a0e1f501 Mon Sep 17 00:00:00 2001 From: Vasilev Dmitrii Date: Sat, 8 Aug 2026 00:06:47 +0700 Subject: [PATCH] docs: whitepaper -- full backprop trains XOR on live silicon (capstone) The full 2-layer backprop microsequencer now trains XOR to 4/4 on a live Artix-7 (25/25 epochs, both layers learning on-chip, weight trajectory bit-exact to the independent Python model). Updated the whitepaper: S3 Training adds the capstone; S4(b) changes 'trains XOR to 4/4 ... bitstream built and validated' to 'flashed to a real Artix-7 and trains XOR to 4/4 across 25/25 epochs, both layers learning on-chip, bit-exact to the model'; S5 honesty resolves the 'pending JTAG re-connect' item and replaces it with the seed-search caveat (nextpnr-xilinx can't express a multicycle constraint, so the deep path is placement-dependent -- pick a stable seed; an open-toolchain limitation a commercial P&R would close directly). Docs only. Refs #1764 Co-Authored-By: Claude Opus 4.8 --- docs/GFT_WHITEPAPER.md | 30 ++++++++++++++++++++---------- docs/NOW.md | 12 ++++++++++-- 2 files changed, 30 insertions(+), 12 deletions(-) diff --git a/docs/GFT_WHITEPAPER.md b/docs/GFT_WHITEPAPER.md index a23411097..5a0636b6e 100644 --- a/docs/GFT_WHITEPAPER.md +++ b/docs/GFT_WHITEPAPER.md @@ -42,8 +42,9 @@ correct nonlinear surface (impossible for a single linear layer). **Training (on the FPGA itself)** — SGD weight update 4/4 · vector SGD 2/2 · gradient descent converges (loss → 0) · 1- and 2-parameter regressions discover hidden weights · a nonlinear neuron with a working ReLU-derivative gate · a binary classifier learns a -decision boundary and **generalizes 8/8 held-out** · the output layer of an XOR network -**learns to solve XOR**. +decision boundary and **generalizes 8/8 held-out** · and the capstone: a **full 2-layer +backprop microsequencer trains XOR to 4/4 across 25/25 epochs, both layers learning on +the chip**, its weight trajectory bit-exact to an independent model. **Edge loop** — train on-chip → read the learned weights → bake them into an inference bitstream (**12/12 held-out with zero training**) → write to SPI flash → the board boots @@ -59,13 +60,17 @@ design is `magsub`'s normalization. Replacing its 12-iteration *linear* normaliz over 17.4 million operand pairs** and roughly **halves every design** (the workhorse trainer: 16.7M → 9.6M fasm). Applied across all 24 specs. -**(b) A microsequenced trainer — full backprop with near-constant area.** The naive +**(b) A microsequenced trainer — full backprop, TRAINING XOR ON LIVE SILICON.** The naive parallel full 2-layer backprop is ~22M fasm, over the measured openXC7 *correctness* ceiling (~17M). A **microsequencer** — one shared multiply core + one shared add core, driven by a microcode program over a register file — runs the full forward + backprop + -update in **~3.4K LUTs / 2.93M fasm** (7× smaller), meets timing at 12 MHz, and **trains -XOR to 4/4** (both layers learn), with the silicon bitstream built and validated. -Network size costs *time*, not FPGA *area* — one shared multiplier, regardless of the net. +update in **~3.4K LUTs** (7× smaller). It is flashed to a real Artix-7 and, streamed the +four XOR corners over UART, **trains XOR to 4/4 across 25/25 epochs — both layers learning +on the chip — with a weight trajectory bit-exact to the independent Python model** +(epoch 0 outputs 0.000 / 0.551 / 0.936 / 0.232 match the model to three decimals; the +error term converges toward zero). This is a full backpropagation training loop — forward, +loss, backward, weight update — running on live FPGA silicon. Network size costs *time*, +not FPGA *area* — one shared multiplier, regardless of the net. **(c) A fully programmable trainer — any feed-forward topology, no structural limits.** A microcode generator turns an *arbitrary* feed-forward net into a buildable bitstream: @@ -105,10 +110,15 @@ pull request*, not asserted once. bit-exact cross-check surfaced it. The port interface was made fully parametric and the register file zero-initialized on reset, so the divergence is now structurally impossible — an example of the gate doing its job. -- **The on-silicon run of the newest programmable/deep stack is pending one physical JTAG - re-connect.** The stack is proven in simulation and CI (bit-exact + synthesizable + - datapath-invariant) for every topology; the earlier 2-layer trainers were already run - and validated on the live board. +- **The full-backprop microsequencer now trains XOR on live silicon** (25/25 epochs to + 4/4, model-exact). It required *seed search*: the open-source place-and-route + (nextpnr-xilinx) cannot express a multicycle timing constraint, so the deep shared-core + path is left timing-relaxed and correctness is placement-dependent — some seeds glitch, + one seed trains cleanly. This is an open-toolchain limitation, not a design flaw (a + commercial P&R would close it directly); we simply pick a stable seed. The *generated* + programmable/deep stack (arbitrary topology) is proven in simulation and CI for every + topology; the on-silicon run of a generated deep net is the natural next step through + the same seed-searched flow. --- diff --git a/docs/NOW.md b/docs/NOW.md index df0661fb4..4f5fe0b02 100644 --- a/docs/NOW.md +++ b/docs/NOW.md @@ -1,6 +1,14 @@ -# NOW — test: cross-target coverage adds the sadd cancellation edge (2026-08-07) +# NOW — docs: whitepaper — full backprop TRAINS XOR ON LIVE SILICON (2026-08-08) -Last updated: 2026-08-07 +Last updated: 2026-08-08 + +## docs: the capstone (backprop trains XOR on silicon) lands in the whitepaper (Refs #1764) + +- The full 2-layer backprop microsequencer now TRAINS XOR to 4/4 on a live Artix-7 (25/25 epochs, both layers learning on-chip, weight trajectory bit-exact to the independent Python model -- ep0 0.000/0.551/0.936/0.232 == model). Updated the whitepaper to state this as fact: + - S3 Training: added the capstone (full backprop microsequencer trains XOR on the chip, model-exact) + - S4(b): "trains XOR to 4/4 ... with the silicon bitstream built and validated" -> now "flashed to a real Artix-7 and trains XOR to 4/4 across 25/25 epochs, both layers learning on-chip, bit-exact to the model" -- a full forward+loss+backward+update loop on live silicon + - S5 honesty: the "pending one physical JTAG re-connect" item is RESOLVED; replaced with the honest seed-search caveat (nextpnr-xilinx can't express a multicycle constraint, so the deep shared-core path is placement-dependent -- an open-toolchain limitation, pick a stable seed; a commercial P&R would close it directly) +- Docs only. Refs #1764 ## test: verify_multitarget covers exact/near cancellation (a, -a) (Refs #1764)