perf(transcript): direct sponge-squeeze challenges, drop ChaCha20 (⚠️ proof-breaking)

[diegokingston]

What

The Fiat-Shamir challenge sampler seeded a fresh ChaCha20Rng from every 32-byte Keccak squeeze and pulled the field element from the keystream (default_transcript.rs). On the recursion guest, that ChaCha block is pure software — Keccak is a precompile, ChaCha is not — so it dominated the challenge-sampling cost (~2–3M guest instructions/proof across β/z/γ and the per-FRI-layer ζ's) while producing randomness the sponge already yields for free.

This replaces it with a Plonky3-style duplex challenger: DefaultTranscript holds a 32-byte output buffer, and sample_field_element/sample_u64 rejection-sample 64-bit candidates straight from the squeeze bytes (8 at a time), refilling with one squeeze when drained. A cubic-extension element (3 coords) usually costs a single squeeze instead of a squeeze + a ChaCha block.

Why it's sound

• Rejection sampling (< GOLDILOCKS_PRIME) is unchanged → distribution stays exactly uniform.
• Squeezing field elements directly from the sponge is the standard FS instantiation (Plonky3/Winterfell); ChaCha only expanded the same 32-byte seed, so removing it is arguably a cleaner ROM instantiation, not a weaker one.
• Prover and verifier share DefaultTranscript, so they derive identical challenges automatically (validated by the roundtrip suite).

Details

• HasDefaultTranscript::get_random_field_element_from_rng(rng) → sample_field_element_from(next_u64) (Goldilocks + cubic ext).
• Output buffer invalidated on every absorb (append_bytes/append_field_element) so a squeeze never reflects input appended after it; Clone copies the buffer, keeping the snapshot/restore contract byte-identical (the GPU-FRI fallback in try_fri_commit_gpu depends on it).
• Drops rand + rand_chacha from crypto's non-dev dependencies (ChaCha-only).

⚠️ Breaking

This changes the Fiat-Shamir hash-to-field, so all proofs and the pinned recursion ELFs must be regenerated — it is a transcript hard-fork, not a verifier-only change. CI must recompile the recursion ELFs; any committed proof fixtures / pinned program-id digests regenerate.

Validation

• 190 stark prove→verify roundtrips pass (prover↔verifier lockstep with the new sampler).
• 47 crypto tests pass (snapshot/restore + sampling determinism).
• Full workspace compiles (crypto, math, stark, prover); clippy clean.
• Guest-cycle benchmark = server (no local RISC-V toolchain).

[MauroToscano]

/bench-verify

[github-actions]

⏳ Benchmark started on the bench server. The verifier bench takes ~5 min; the recursion-guest cycle comparison then adds guest builds — a few minutes when cached, up to ~1h on a cold run. The bench server is occupied until it finishes.

[github-actions]

Verifier benchmark — a7a03227aa vs main (20 pairs)

=== Verify ABBA result ===

Metric	main	PR	Δ
Verify time (per-side)	3.586s	3.587s	+0.03% ⚪
Proof size	204.30 MiB	204.30 MiB	+0.00% ⚪

Per-side (⚠️ PR REJECTS the baseline's valid proof — likely a VERIFY REGRESSION, not a format change): A/B/B/A cancels machine drift but not proof-specific variance — read the Verify-time Δ as approximate.

  pairs: 20   mean A (PR): 3.587s   mean B (main): 3.586s
  [parametric] paired-t   mean +0.03%   sd 0.61%   se 0.14%
               95% CI: [-0.25%, +0.32%]   (t df=19 = 2.093)
  [robust]     median -0.01%   Wilcoxon W+=102 W-=88  p(exact)=0.7983  (z=+0.26)

  run-to-run jitter:    A CV 0.43%   B CV 0.33%        (lower = steadier)
  within-session drift: -0.19% over the run, 1st->2nd half -0.17%

⚪ INCONCLUSIVE — effect not separable from 0 at n=20 (point estimate ~-0.01%). Add pairs to resolve.

_{Drift-free interleaved A/B/B/A measurement. - = PR faster. Trust the verdict when paired-t and Wilcoxon agree.}

---

Recursion guest cycles (main vs PR)

=== Recursion-guest cycle comparison — single query (blowup=2, 1 query) — deterministic to ~±100k cycles ===
REF_B (baseline) origin/main a864832 guest=recursion-min.elf
REF_A (PR) a7a0322 a7a0322 guest=recursion-min.elf

Metric	REF_B (baseline)	REF_A (PR)	Δ (A-B)
Guest cycles	43.6M	42.0M	-1.6M (-3.75%)
Keccak calls	3025	3025	0

note: cycles reproduce to ~±100k (build codegen + proof nondeterminism); treat sub-100k deltas as noise, not signal.

raw (exact integer counts)

ref_b_sha=a8648320867f7f4242fe286a5b266ffed1fb5519 ref_b_elf=recursion-min.elf ref_b_cycles=43608950 ref_b_keccak=3025 ref_b_execute_wall_s=1
ref_a_sha=a7a03227aa03853a08e4292e9922900bd52642c9 ref_a_elf=recursion-min.elf ref_a_cycles=41974211 ref_a_keccak=3025 ref_a_execute_wall_s=2
delta_cycles=-1634739 delta_keccak=0

[diegokingston]

/bench-verify

[github-actions]

⏳ Benchmark started on the bench server. The verifier bench takes ~5 min; the recursion-guest cycle comparison then adds guest builds — a few minutes when cached, up to ~1h on a cold run. The bench server is occupied until it finishes.