Skip to content

perf(profile): optimize the math cores so cargo test --workspace is runnable - #71

Open
Nexlab-One wants to merge 1 commit into
mainfrom
perf/dev-profile-math-cores
Open

perf(profile): optimize the math cores so cargo test --workspace is runnable#71
Nexlab-One wants to merge 1 commit into
mainfrom
perf/dev-profile-math-cores

Conversation

@Nexlab-One

Copy link
Copy Markdown
Contributor

Problem

At dev/test opt-level = 0 the crypto math is 1-2 orders of magnitude slow and the local workspace test gate does not terminate in practical time. Measured on a warm cache, running test binaries directly so no compile time is in the numbers: 8 binaries exceeded a 90s cap, the lib-q umbrella unit binary exceeded 420s, and a dozen more ran 25-70s each. CI already knew this per-crate via its skip lists; the local gate had no answer.

Change

Two layers, extending the existing fn-dsa precedent already in this file:

  • [profile.dev.package."*"] at opt-level 2 for all external deps. The "*" spec never matches workspace members, so members keep dev settings unless named.
  • Named entries at opt-level 2 for 49 crates with heavy in-crate math: the hash stack (Keccak/SHA3/K12/Poseidon — the shared substrate under SLH-DSA, SHAKE, the RNGs, HQC), the algorithm cores, the lattice/threshold/proof crates, and the STARK+plonky proving stacks.

Measured before → after

binary before after
lib-q-slh-dsa >90s 11.3s
lib-q-lattice-zkp >90s 11.2s
lib-q-zkp >90s 20.7s
lib-q-hqc simd_correctness >90s 8.4s
lib-q-zk-encryption-proof >90s 111.7s (completes)
lib-q-blind-token >90s 219.8s (completes)
lib-q-sig slh_dsa_context 68.7s 3.5s
lib-q-sig no_std 65.6s 3.1s
lib-q-mve 59.1s 2.4s
lib-q-sca-test 31.6s 1.6s
lib-q-ring-sig interop 28.0s 2.0s
lib-q-dkg wire_roundtrip 27.8s 2.1s
controls (types/utils/keccak) unchanged

No safety check is weakened

Only opt-level is overridden. debug-assertions and overflow-checks inherit from dev/test and stay ON — verified per unit via cargo --unit-graph, including the #[cfg(debug_assertions)] constraint checkers inside the STARK/plonky provers. Zero tests ignored, gated, or removed.

All 11 timing-sensitive suites still pass (k12/sha3 constant_time and performance, every hardened_dudect_smoke, threshold-kem-lattice perf_probe). Their codegen conditions do change — closer to what ships than unoptimized debug, but a different regime — so a CAUTION comment in the file names them.

This also corrects a comment that claimed a [profile.dev.package] override cannot reach the root crate being tested. On nightly-2026-07-24 it does; one mechanism covers both dependency and root positions.

Compile cost is 7m23s one-time on a warm cache. Dev and test artifact universes stay unified, so no double build, and incremental check/build on unlisted crates is unchanged.

Out of scope

Three binaries remain capped after this change — lib-q-fn-dsa unit, lib-q-sig/crypto_operations_tests, and the lib-q umbrella. That is not slowness and this PR does not fix it: they livelock, only under --all-features, because the no_avx2 feature selects a portable FN-DSA keygen path that appears not to terminate. Filed separately — it has a potential non-x86 correctness implication.

Complements origin/ci/trim-full-workspace-test-matrix rather than duplicating it: that branch is CI-row-scoped, this is the workspace-profile layer beneath it. No CI files touched.

… runnable

At dev/test opt-level 0 the crypto math is 1-2 orders of magnitude slow and
the local workspace test gate does not terminate in practical time. Measured
on a warm cache (test binaries run directly, so no compile time in the
numbers): 8 binaries exceeded a 90s cap, the `lib-q` umbrella unit binary
exceeded 420s, and a dozen more ran 25-70s each. CI already knew this
per-crate via its skip lists; the local gate had no answer.

Two layers, extending the existing fn-dsa precedent in this file:

  * `[profile.dev.package."*"]` at opt-level 2 for all external deps. The
    "*" spec never matches workspace members, so members keep dev settings
    unless named below.
  * Named entries at opt-level 2 for 49 crates carrying heavy in-crate math:
    the hash stack (Keccak/SHA3/K12/Poseidon - the shared substrate under
    SLH-DSA, SHAKE, the RNGs and HQC), the algorithm cores, the
    lattice/threshold/proof crates, and the STARK+plonky proving stacks.

Measured before -> after, same binaries, same machine:

    lib-q-slh-dsa               >90s -> 11.3s
    lib-q-lattice-zkp           >90s -> 11.2s
    lib-q-zkp                   >90s -> 20.7s
    lib-q-hqc simd_correctness  >90s ->  8.4s
    lib-q-zk-encryption-proof   >90s -> 111.7s (completes)
    lib-q-blind-token           >90s -> 219.8s (completes)
    lib-q-sig slh_dsa_context   68.7s -> 3.5s
    lib-q-sig no_std            65.6s -> 3.1s
    lib-q-mve                   59.1s -> 2.4s
    lib-q-sca-test              31.6s -> 1.6s
    lib-q-ring-sig interop      28.0s -> 2.0s
    lib-q-dkg wire_roundtrip    27.8s -> 2.1s
    controls (types/utils/keccak) unchanged

NO SAFETY CHECK IS WEAKENED. Only `opt-level` is overridden; `debug-assertions`
and `overflow-checks` inherit from dev/test and stay ON - verified per unit via
`cargo --unit-graph`, including the `#[cfg(debug_assertions)]` constraint
checkers inside the STARK/plonky provers. Zero tests were ignored, gated, or
removed.

All 11 timing-sensitive suites still pass (k12/sha3 constant_time and
performance, every hardened_dudect_smoke, threshold-kem-lattice perf_probe).
Their codegen conditions do change - closer to what ships than unoptimized
debug, but a different regime - so a CAUTION comment names them in this file.

Also corrects a comment that claimed a `[profile.dev.package]` override cannot
reach the root crate being tested. On nightly-2026-07-24 it does (the test
profile inherits dev's package overrides), verified via `--unit-graph`; one
mechanism therefore covers both dependency and root positions.

Thin API/type/util crates (lib-q, core, sig, random, types, utils, transcript,
the AEADs) deliberately stay at opt-level 0 for debuggability - measurement
confirmed they are fast once the math they call is optimized. Compile cost is
7m23s one-time on a warm cache; dev and test artifact universes stay unified,
so there is no double build, and incremental check/build on unlisted crates is
unchanged.

Three binaries remain capped after this change - lib-q-fn-dsa unit,
lib-q-sig/crypto_operations_tests, and the lib-q umbrella. That is NOT
slowness and this commit does not address it: they livelock, and only under
`--all-features`, because the `no_avx2` feature selects a portable FN-DSA
keygen path that appears not to terminate. Tracked separately.

Complements origin/ci/trim-full-workspace-test-matrix rather than duplicating
it: that branch is CI-row-scoped, this is the workspace-profile layer beneath
it. No CI files touched here.
@Nexlab-One

Copy link
Copy Markdown
Contributor Author

Refinement on the out-of-scope note, now that I've bisected it — the characterisation in the PR description is incomplete.

I wrote that the no_avx2 feature selects a portable keygen path that doesn't terminate. Accurate as far as it goes, but the trigger is broader: fn-dsa-kgen's test_keygen_self calls the portable keygen_from_seed directly (lib.rs:1239), bypassing the dispatcher at lib.rs:256-277, and it hangs identically with and without the feature. So the portable function is broken independent of any feature flag — no_avx2, and any non-x86 target, merely routes production code into an already-broken function.

Bisected by restricting the test loop to one logn at a time and rebuilding for each:

logn n result
2 4 ok, finished in 0.00s
3 8 hang >60s
4-10 16-1024 hang >60s

A keygen on 8 coefficients running over a minute is a livelock, not slowness. The dispatcher compiles its AVX2 branch only for x86/x86_64, so aarch64 / armv7 / wasm32 fall through to this same function — and CI covers those targets with cargo check only, never executing them. No CI test row runs --all-features either (grep -c → 0), which is why nothing has caught it.

Tracked on the board as t_1acb56d5 with full evidence and an explicit Not checked list — notably, this has still never been run on real non-x86 hardware, so the non-x86 impact remains inference from the shared code path.

None of this affects the diff in this PR, which is manifest-only.

@github-actions

Copy link
Copy Markdown

🔒 Security Validation Report

Generated: Wed Jul 29 00:23:25 UTC 2026

📊 Summary

  • NIST compliance: success
  • Cryptographic validation: success
  • Constant-time operations: success
  • Memory safety: success
  • Dependency security: success
  • WASM security: success

✅ Overall Security Status: PASSED

All critical security validations passed successfully.

🔍 Details

This report covers:

  • NIST post-quantum algorithm compliance
  • Constant-time operation verification
  • Memory safety and zeroization checks
  • Dependency vulnerability scanning
  • WASM build artifact validation

📋 Next Steps

✅ Security validation passed. Code is ready for deployment.


Security validation passed! This code meets all security requirements.

@github-actions

Copy link
Copy Markdown

🔍 Pull Request Summary

Generated: Wed Jul 29 00:26:27 UTC 2026

📋 Validation Results

  • Core Validation: success
  • Security Validation: success
  • Test Coverage: success
  • WASM Compatibility: success
  • Documentation: success

✅ Overall Status: PASSED

🔒 Security Checklist

  • No classical cryptographic algorithms
  • Only SHA-3 family hash functions
  • Constant-time operations
  • Proper memory zeroization
  • Input validation
  • Error handling

📝 Review Notes

Please review the security implications of this change carefully.
All cryptographic changes require security team review.


Automated validation passed! This PR is ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant