perf(syscalls,crypto): in-place keccak sponge + direct finalize + fixed-shape parents (−40% recursion-verifier cycles at real query counts) by MauroToscano · Pull Request #847 · yetanotherco/lambda_vm

MauroToscano · 2026-07-16T21:59:55Z

What

Removes the software byte-plumbing around the guest's 1-cycle keccak_permute ecall — flamegraph attribution measured it at ~1,322 cycles per permutation, 48% of all recursion-verifier guest cycles at the multi-query regime. Six commits (three optimizations, tests, and a review-fix pass):

In-place sponge (syscalls/src/keccak.rs): the [u64; 25] state is the only buffer — input XORs directly into the rate lanes at a running offset, padding lands in place, the digest reads straight out of the state (tiny-keccak's design). Deletes the 136-byte staging buffer + zeroing, a full-block copy, the 17-lane byte re-extraction, the separate pad block. The whole-lane fast path gates on runtime pointer alignment (the VM traps on unaligned doubleword loads); misaligned input falls back to byte-wise absorption — correctness never depends on alignment.
Direct finalize into the node array (crypto/crypto/src/merkle_tree/backends/field_element_vector.rs): the hot Merkle backends skip the Digest::finalize blanket impl (GenericArray alloc + zeroing + two 32-byte copies per hash) and squeeze straight into the caller's [u8; 32]. TypeId + cfg(target_arch = "riscv64") specialization (same pattern as perf(crypto): specialize keccak256 for the fixed-shape Merkle parent hash #774); host and every non-keccak digest keep the generic path.
Fixed-shape keccak256_pair for 64-byte Merkle parents: loads the eight data lanes from the two nodes (byte-assembled — never an aligned load on a byte-aligned buffer), XORs the pad10*1 bits, one permutation, squeezes. Skips the incremental sponge entirely for the most frequent hash shape.
Differential tests + host-test enablement (syscalls/): see Testing below.
5.–6. Review-fix pass: tests wired into make test (test-syscalls target), endianness guard on the fast path, shared squeeze helper, unused dev-dep dropped, StdRng in tests.

Measurements (exact, deterministic)

Fixed (guest ELF, input blob) pairs — guest cycle counts are bit-reproducible, so each delta is an exact integer, no statistics. Blobs dumped once from the base, the same blobs fed to base and PR guests. Base = main (@ 3be1eed).

Regime	main	This PR	Δ
min (blowup2, 1 query, empty inner)	43,529,227	41,825,235	−3.9%
blowup8 (73 queries, empty inner)	368,070,066	261,062,738	−29.1%
ethrex 4-tx block, continuation guest, blowup2/219 queries	6,900,609,615	4,154,015,776	−39.8%

The real-block row was measured on the #825 stack (bench/recursion-full-queries), which provides the continuation guest and 219-query preset — this PR is independent of it; the min/blowup8 rows above are measured directly against main and reproduce main's baseline integers exactly.

Per-commit contribution on the real target: in-place sponge −1.545B, direct finalize −0.674B, fixed-shape pair −0.528B; the test commit is measured cycle-neutral (bit-identical counts). Keccak permutation counts are unchanged in every regime (3,025 / 134,173 / 3,150,816) — this removes plumbing around the permutations, not permutations.

A fourth optimization variant (lane-assembling misaligned absorb input) was measured at +4.7% on blowup8 across three formulations and dropped: on a 1-cycle-per-instruction VM the byte-wise absorb is already minimal and wider gathers only add instructions.

The min → blowup8 → real gradient (−4% → −29% → −40%) is expected: the plumbing scales with hash volume, i.e. queries × tables × tree depth.

Testing

New host differential tests (cd syscalls && cargo test): a software Keccak-f[1600] is injected in place of the ecall (test-only cfg; guest and host non-test builds untouched) and the sponge is checked for digest byte-identity against sha3::Keccak256: every input length through three rate blocks (all padding boundaries incl. 135/136/137), 300 randomized misaligned-chunking cases through update, and keccak256_pair against both the reference and the streaming sponge. A test-syscalls Makefile target runs them under make test, and CI runs them as a dedicated step in the cli-test job.
Enabling host tests surfaced a latent crate defect, fixed here: the guest global-allocator registration was unconditional and hijacked any host binary with a never-initialized heap; now cfg_attr(target_arch = "riscv64", global_allocator). Guest cycle counts re-measured bit-identical after the change.
End-to-end equivalence oracle: all three base-dumped proof blobs verify unchanged under the rewritten guest (any digest difference would diverge the Fiat-Shamir transcript and fail loudly); a bit-flipped blob is rejected; test_recursion_execute_empty (ignored/slow suite) passes with the committed attestation matching the host program_id recompute.
Host suites green: crypto (47), stark (199), stream-bytes parity (3). make lint green.

Notes

Targets main directly (originally developed against bench(recursion): measure the verifier at real query counts, over real blocks #825's benchmark branch for the realistic measurements; rebased with no content changes to the four files).
/bench-verify is now a clean PR-vs-main comparison; note it runs the min regime (1 FRI query), where this PR shows only −3.9% — the plumbing removed here scales with queries × tables, hence −29%/−40% at the realistic regimes.
A review pass (multi-agent, adversarially verified) was applied: differential tests are wired into make test, the whole-lane fast path is endianness-guarded, pad/squeeze logic is shared, and two proposed alternatives were rejected by measurement (a misaligned lane-assembly absorb path: +4.7%; a generic finalize_into replacement for the TypeId fast path: +0.5% — the by-value sponge through the trait layer can't be fully elided without LTO).
The sponge is not at its floor: residual ~420 cycles/permutation remains (per-element streaming overhead and the Digest trait boundary on the non-specialized paths). Follow-ups tracked in the optimization log.

MauroToscano · 2026-07-17T14:31:04Z

/review-ai

github-actions · 2026-07-17T14:32:15Z

Codex Code Review

No actionable issues found in the PR changes. The unsafe aligned lane read is correctly guarded, bounded, and valid for the little-endian RISC-V guest target.

claude · 2026-07-17T14:33:14Z

Reviewed the in-place Keccak sponge rewrite (single file, syscalls/src/keccak.rs). No issues found — the change is correct, self-contained, and clearly documented.

Verified the equivalence reasoning:

Padding: delimiter XORs at offset, final pad bit into byte 135. The bytes in offset+1..135 are correctly left untouched (equivalent to XOR-with-zero of the original fill region). The offset == 135 overlap collapses to 0x01 ^ 0x80 = 0x81, matching the old |= FINAL_PAD_BIT.
Invariant offset < RATE_BYTES holds across calls (new() = 0; both fast and byte paths reset to 0 right after a permute), so the finalize xor_byte_at_offset(DELIMITER) is always in range.
Fast-path bounds: with offset % 8 == 0, base + take <= 17, so lane writes stay within the 17 rate lanes and never touch capacity. The unsafe u64 read is correctly gated on runtime pointer alignment (VM traps on unaligned dword loads) and take*8 <= input.len() keeps it in-bounds; native LE load == from_le_bytes on the riscv64 target.
Multi-update / misaligned chunking: once byte-wise, offset stays off a lane boundary until it catches back up — correctness is independent of how input is split.
API surface unchanged: new/update/finalize, Clone, Default preserved; removed internals (buf/buf_len/absorb_block) have no external references — PlatformKeccak256 and all call sites route through the public API.

The unsafe block is well-scoped and its SAFETY comment accurately states the preconditions.

github-actions · 2026-07-17T14:37:17Z

AI Review

PR #847 · 1 changed files

Findings

No non-rejected structured findings were reported.

Reviewer Lanes

Lane	Model	Prompt	Status
glm	openrouter/z-ai/glm-5.2	general	success
kimi	openrouter/moonshotai/kimi-k2.7-code	general	success
minimax	minimax/MiniMax-M3	general	error: opencode failed (provider/auth/runtime error) and no findings were submitted
moonmath	zro/minimax-m3	general	success
nemotron	openrouter/nvidia/nemotron-3-ultra-550b-a55b	general	success

Native Codex and Claude reviews run separately and post their own comments. They are not included in this structured provenance report.

Raw lane outputs, candidates, final issues, and model metrics are uploaded as workflow artifacts.

diegokingston · 2026-07-17T14:45:29Z

/bench-verify

github-actions · 2026-07-17T14:50:26Z

⏳ Benchmark started on the bench server. The verifier bench takes ~5 min; the recursion-guest cycle comparison then adds guest builds — a few minutes when cached, up to ~1h on a cold run. The bench server is occupied until it finishes.

github-actions · 2026-07-17T14:55:39Z

Verifier benchmark — `c061b35539` vs `main` (20 pairs)

=== Verify ABBA result ===

Metric	main	PR	Δ
Verify time	3.588s	3.604s	+0.47% 🔴
Proof size	204.30 MiB	204.30 MiB	+0.00% ⚪

  pairs: 20   mean A (PR): 3.604s   mean B (main): 3.588s
  [parametric] paired-t   mean +0.47%   sd 0.86%   se 0.19%
               95% CI: [+0.06%, +0.87%]   (t df=19 = 2.093)
  [robust]     median +0.35%   Wilcoxon W+=159 W-=31  p(exact)=0.0082  (z=+2.56)

  run-to-run jitter:    A CV 0.67%   B CV 0.59%        (lower = steadier)
  within-session drift: -0.14% over the run, 1st->2nd half -0.02%

🔴 REAL REGRESSION — PR verifies ~0.47% slower (paired-t and Wilcoxon agree).

_{Drift-free interleaved A/B/B/A measurement. - = PR faster. Trust the verdict when paired-t and Wilcoxon agree.}

Recursion guest cycles (main vs PR)

=== Recursion-guest cycle comparison — single query (blowup=2, 1 query) — deterministic to ~±100k cycles ===
REF_B (baseline) origin/main 3be1eed guest=recursion-min.elf
REF_A (PR) c061b35 c061b35 guest=recursion-min.elf

Metric	REF_B (baseline)	REF_A (PR)	Δ (A-B)
Guest cycles	43.6M	42.3M	-1.3M (-3.08%)
Keccak calls	3025	3025	0

note: cycles reproduce to ~±100k (build codegen + proof nondeterminism); treat sub-100k deltas as noise, not signal.

raw (exact integer counts)

ref_b_sha=3be1eed35bd73e96c36a99bf03b58a87cf358af7 ref_b_elf=recursion-min.elf ref_b_cycles=43605029 ref_b_keccak=3025 ref_b_execute_wall_s=1
ref_a_sha=c061b35539996e8c708fbdd934b8e7d22f512863 ref_a_elf=recursion-min.elf ref_a_cycles=42261337 ref_a_keccak=3025 ref_a_execute_wall_s=1
delta_cycles=-1343692 delta_keccak=0

The state is the only buffer: input bytes XOR directly into the rate lanes at a running offset (tiny-keccak's design), padding lands in place, and the digest is read straight out of the state. This removes the staging buffer and its zeroing, the full-block copy, the 17-lane byte re-extraction, and the separate pad block — the largest slice of the ~1,322 cycles of software plumbing measured per 1-cycle keccak_permute ecall (measured for this commit alone: −1.545B cycles on the 219-query ethrex-block recursion benchmark; the residual is addressed by the follow-up commits). The whole-lane fast path gates on the input pointer's runtime alignment (the VM traps on unaligned doubleword loads); misaligned input falls back to byte-wise absorption, so correctness never depends on alignment. Digests are byte-identical to the previous implementation.

The field-element Merkle backends finalized every leaf and parent hash through the `Digest::finalize` blanket: it allocates a zeroed `GenericArray`, the riscv64 keccak adapter fills a local `[u8; 32]` and copies it into that `Output`, then the caller copies the `Output` once more into its own node array. That is two 32-byte memcpys plus a 32-byte memset of pure plumbing around the single permutation each hash actually costs. Route the leaf (`hash_data`, `hash_data_from_slices`, `hash_bytes`) and parent (`hash_new_parent`) hashes of `FieldElementPairBackend` and `FieldElementVectorBackend` through a shared `hash_streamed` helper. On the riscv64 guest, when `D` is the platform keccak digest and nodes are 32 bytes, it drives the syscall sponge directly and squeezes straight into the result array — no GenericArray, no intermediate buffer, no double copy. Every other digest / node size and the entire host build keep the generic `Digest` path, so output stays byte-identical (blobs still verify). Recursion verifier guest cycles: min 42,185,756 -> 41,991,672 (-0.46%), blowup8 291,147,000 -> 273,789,625 (-5.96%); keccak permutation counts unchanged (3,025 / 134,173).

Every Merkle parent hash is exactly 64 bytes (two concatenated 32-byte nodes), which fits the keccak rate in a single block. Add `keccak256_pair` to the syscall sponge: it loads the eight data lanes straight from the two nodes, XORs the pad10*1 bits in place, runs one permutation, and squeezes four lanes — skipping the incremental sponge's per-byte absorb, running-offset bookkeeping, and separate padding pass. Route `hash_new_parent` of both keccak field-element backends to it on the riscv64 guest via the same TypeId + node-size dispatch used for the direct finalize; every other digest / node size and the host build keep the generic streaming path, so output stays byte-identical (blobs verify, keccak counts unchanged at 3,025 / 134,173). Cumulative recursion verifier guest cycles after this commit: min 41,973,461, blowup8 271,979,593. C's isolated effect vs the prior commit: min -266,369, blowup8 -14,646,310. (The prior commit is a measured regression DROP-candidate; A and C together, with B dropped, are the recommended stack.)

The sponge's absorption chunking, padding, and squeezing had no unit coverage — its only oracle was end-to-end proof-blob acceptance. Host tests now inject a software Keccak-f[1600] in place of the ecall and check digest byte-identity against sha3::Keccak256: every input length through three rate blocks (all padding boundaries), 300 randomized misaligned-chunking cases, and the fixed-shape pair path against both the reference and the streaming sponge. The guest global allocator registration is now gated to riscv64 so a host `cargo test` of this crate uses the system allocator instead of aborting on the never-initialized guest heap. Guest builds unchanged (cycle counts bit-identical).

… critical-section dev-dep The syscalls crate is excluded from the workspace (riscv-only bare-metal allocator/entrypoints that assemble only for the guest target), so the root `cargo test` never reached its host keccak-vs-sha3 differential tests and CI couldn't run them. Add a `test-syscalls` target that runs `cargo test` in the crate dir and make `test` depend on it. Drop the `critical-section` dev-dependency: the embedded_alloc global allocator is gated to `target_arch = "riscv64"` (src/allocator.rs), so on host test builds it is never the active allocator and its critical-section path is never linked. A clean `cargo test` links and passes without it. critical-section stays in the lock transitively (via riscv/embedded-alloc) but needs no host impl. Commit syscalls/Cargo.lock (now that CI builds this crate standalone) to match the repo convention for excluded crates and pin dev-deps for reproducible tests.

…tdRng tests - update(): gate the whole-lane fast path on cfg!(target_endian = "little"). The raw *const u64 lane read equals the required little-endian value only on LE targets; the cfg! folds to a compile-time constant (codegen unchanged on every real target) and the byte fallback is endian-correct everywhere. - Deduplicate the squeeze: extract squeeze32_into(state, out), shared by Keccak256::finalize and keccak256_pair. Write-into (rather than returning [u8; 32]) so finalize fills its output reference in place — verified guest-codegen and cycle-identical to the pre-dedup loops. - Tests: replace the hand-rolled xorshift RNG with rand's StdRng + SeedableRng (matching src/random.rs), keeping fixed seeds for reproducibility. - Document at the byte-wise fallback that a from_le_bytes lane-assembly middle path was measured at +4.7% cycles (dropped commit f6d575ed) so it is not re-proposed on this 1-cycle-per-instruction VM. Digests remain byte-identical to sha3 Keccak-256 (host differential tests); guest cycles unchanged (recursion-min 41,747,766 / recursion-blowup8 260,967,578, keccak call counts identical).

…r code sites Two refactors that reviewers (and optimization-hunting agents) will keep re-proposing were implemented and measured slower on the guest; pin the numbers where the edit would happen so the effort isn't repeated: - replacing the TypeId dispatch in hash_streamed with a generic Digest::finalize_into route: +0.5% guest cycles at blowup8 across every formulation (by-value sponge through the trait layer, not elidable cross-crate without LTO) - return-value squeeze32: +81k cycles from the extra stack temporary The misaligned lane-assembly absorb path (+4.7%) is already documented at the byte fallback in update().

- CI never invokes 'make test', so the syscalls differential tests still did not run in CI despite the Makefile wiring; run 'make test-syscalls' as a dedicated step in the cli-test job. - Swap the test RNG from StdRng (algorithm unstable across rand releases) to ChaCha8Rng so the fixed seeds pin the exact fuzz case streams across versions and checkouts. - Correct the finalize_into dead-end comment to give both preset percentages (+0.14% min / +0.48% blowup8) instead of one figure. - Cite PR #847 instead of a commit hash unreachable from any ref for the dropped lane-assembly measurement.

The _start entry symbol (and its imported main) only exist for the guest; on a Linux host they collide with the C runtime / test harness entry and 'cargo test' fails with "entry symbol `main` declared multiple times" (macOS tolerates the duplicate, which is why local host tests passed). Same treatment as the global-allocator gating: guest builds are unchanged.

…n the adapter passthrough invariant Review feedback (two reviewers independently): the whole-lane fast path was only exercised because Vec<u8> bases happen to be 8-aligned on current platforms. A repr(align(8)) buffer now guarantees the aligned path and a +1-offset view guarantees the byte fallback, across all padding boundaries. Also documents two load-bearing implicit facts: the TypeId specializations depend on PlatformKeccak256 remaining a pure passthrough of the syscall sponge (INVARIANT note in the adapter), and the host tests' coverage boundary (the ecall and riscv64 branches are validated only by the proof-blob oracle).

MauroToscano · 2026-07-17T18:19:30Z

/bench-prove

github-actions · 2026-07-17T18:22:59Z

Benchmark — ethrex 20 transfers (median of 3)

_{Table parallelism: auto (cores / 3)}

Metric	main	PR	Δ
Peak heap	73480 MB	73457 MB	-23 MB (+-0.0%) ⚪
Prove time	38.095s	38.199s	+0.104s (+0.3%) ⚪

✅ No significant change.

✅ Low variance (time: 2.5%, heap: 1.1%)

_{Commit: 636ba67 · Baseline: cached · Runner: self-hosted bench}

MauroToscano marked this pull request as ready for review July 17, 2026 14:30

MauroToscano force-pushed the opt-rec/1-inplace-keccak-sponge branch from e0c47d1 to e2a715d Compare July 17, 2026 15:46

MauroToscano changed the title ~~perf(syscalls): absorb keccak sponge input in place (−22% recursion-verifier cycles at real query counts)~~ perf(syscalls,crypto): in-place keccak sponge + direct finalize + fixed-shape parents (−40% recursion-verifier cycles at real query counts) Jul 17, 2026

MauroToscano added 6 commits July 17, 2026 14:28

MauroToscano force-pushed the opt-rec/1-inplace-keccak-sponge branch from bd408a2 to 6db4c86 Compare July 17, 2026 17:28

MauroToscano changed the base branch from bench/recursion-full-queries to main July 17, 2026 17:28

MauroToscano added 4 commits July 17, 2026 14:32

MauroToscano enabled auto-merge July 17, 2026 18:18

diegokingston approved these changes Jul 17, 2026

View reviewed changes

ColoCarletti approved these changes Jul 17, 2026

View reviewed changes

MauroToscano disabled auto-merge July 17, 2026 18:19

MauroToscano enabled auto-merge July 17, 2026 18:23

MauroToscano added this pull request to the merge queue Jul 17, 2026

Merged via the queue into main with commit 68a120a Jul 17, 2026
15 checks passed

MauroToscano deleted the opt-rec/1-inplace-keccak-sponge branch July 17, 2026 18:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(syscalls,crypto): in-place keccak sponge + direct finalize + fixed-shape parents (−40% recursion-verifier cycles at real query counts)#847

perf(syscalls,crypto): in-place keccak sponge + direct finalize + fixed-shape parents (−40% recursion-verifier cycles at real query counts)#847
MauroToscano merged 10 commits into
mainfrom
opt-rec/1-inplace-keccak-sponge

MauroToscano commented Jul 16, 2026 •

edited

Loading

Uh oh!

MauroToscano commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Uh oh!

claude Bot commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Uh oh!

diegokingston commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Uh oh!

MauroToscano commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

MauroToscano commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Measurements (exact, deterministic)

Testing

Notes

Uh oh!

MauroToscano commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Codex Code Review

Uh oh!

claude Bot commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

AI Review

Findings

Reviewer Lanes

Uh oh!

diegokingston commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Verifier benchmark — c061b35539 vs main (20 pairs)

Recursion guest cycles (main vs PR)

Uh oh!

MauroToscano commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

Benchmark — ethrex 20 transfers (median of 3)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MauroToscano commented Jul 16, 2026 •

edited

Loading

Verifier benchmark — `c061b35539` vs `main` (20 pairs)