Skip to content

chore(deps): refresh Cargo.lock and re-persist canbench results - #245

Merged
mbjorkqvist merged 1 commit into
mainfrom
mathias/chore-refresh-deps-canbench-baseline
Aug 12, 2026
Merged

chore(deps): refresh Cargo.lock and re-persist canbench results#245
mbjorkqvist merged 1 commit into
mainfrom
mathias/chore-refresh-deps-canbench-baseline

Conversation

@mbjorkqvist

@mbjorkqvist mbjorkqvist commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Purpose

Land the dependency refresh that the v0.4.0 release PR (#244) would otherwise carry, so the benchmark delta is reviewable on its own and the release PR is left touching only version bumps and changelogs.

release-plz.toml sets dependencies_update = true, so the release PR includes a cargo update across 74 crates. That shifts instruction counts and leaves canister/canbench_results.yml stale, failing the benchmark gate. The baseline cannot be re-persisted inside #244: it is bot-authored, and canbench_results.yml is not in .github/repo_policies/BOT_APPROVED_FILES — and the whitelist itself cannot be widened from a bot PR without self-authorizing, which is exactly what that control prevents.

The benchmark delta is entirely dependency-driven

Reverting only Cargo.lock to main's version and re-running canbench on the same tree returns every benchmark to baseline:

Scenario instructions drift
With the refreshed lockfile 1 regressed, 4 improved — max +4.26%, min −23.43%
Same tree, old lockfile max +0.19%, median +0.03% — noise

The lockfile was the only variable, so no canister code regressed. The two bumps in the canister's wasm path are num-bigint 0.4.6 → 0.4.8 (candid's Nat/Int backend) and ic_principal 0.1.3 → 0.1.5 (pulled in by both
candid and ic-stable-structures). The rest are host-only build and test dependencies.

Net effect is favourable — 4 improvements against 1 regression, median −0.21%:

benchmark instructions Δ%
+ bench_write_events::Settling 29.38M +4.39%
+ bench_write_events 30.00M +4.26%
bench_get_order_book_depth_max 4.61M −23.43%
bench_get_order_book_depth_default 570.07K −21.54%
bench_get_order_book_ticker 8.60K −13.34%
bench_get_my_trades 48.62M −2.66%

Heap and stable-memory usage are unchanged across all 16 benchmarks.

Note on the lockfile

This is release-plz's own resolution verbatim, with only the two workspace-member versions kept at 0.3.0, rather than a fresh cargo update. A re-run could resolve different versions and would both conflict with #244's
lockfile and invalidate this baseline. Keeping it identical means merging main into #244 afterwards is conflict-free, and canbench_results.yml drops out of that PR's diff entirely.

Landing the dependency refresh on its own, ahead of the v0.4.0 release
PR. release-plz sets `dependencies_update = true`, so the release PR
carries a `cargo update` of 74 crates, which shifts instruction counts
and leaves `canbench_results.yml` stale. That baseline cannot be fixed
inside the release PR: it is bot-authored, and `canbench_results.yml`
is not in `.github/repo_policies/BOT_APPROVED_FILES` — nor can the
whitelist itself be widened from a bot PR without self-authorizing.

Reverting only Cargo.lock and re-running canbench returns every
benchmark to baseline (max +0.19%, noise), so the whole delta is
dependency-driven and no canister code regressed. The wasm-path bumps
behind it are num-bigint 0.4.6 -> 0.4.8 (candid's Nat/Int backend) and
ic_principal 0.1.3 -> 0.1.5 (via candid and ic-stable-structures).

Net effect is favourable: 4 improvements against 1 regression, median
-0.21%. bench_write_events::Settling rises 4.39% (29.38M instructions)
while the order-book queries drop 13-23% and get_my_trades 2.66%. Heap
and stable-memory usage are unchanged across all 16 benchmarks.

The lockfile is release-plz's resolution verbatim, with only the two
workspace-member versions kept at 0.3.0, so merging main into the
release PR afterwards is conflict-free.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 6, 2026 10:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Separates the v0.4.0 release’s dependency refresh into its own PR by updating the workspace lockfile and re-persisting the canister canbench baseline, so benchmark deltas can be reviewed independently and the release PR can stay focused on version/changelog changes.

Changes:

  • Refreshes Cargo.lock with the dependency resolution produced by release-plz.
  • Updates canister/canbench_results.yml to match the persisted canbench v0.4.1 baseline under the refreshed dependency set.

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.

File Description
Cargo.lock Updates resolved crate versions/checksums to the refreshed dependency set.
canister/canbench_results.yml Re-persisted benchmark instruction counts/metrics to align with the refreshed lockfile.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

canbench 🏋 (dir: canister) 713d219 2026-08-06 10:48:00 UTC

canister/canbench_results.yml is up to date
📦 canbench_results_benchmark.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   Regressions and improvements 🔴🟢
    counts:   [total 16 | regressed 1 | improved 4 | new 0 | unchanged 11]
    change:   [max +2.53M | p75 -1.02K | median -134.31K | p25 -358.71K | min -1.41M]
    change %: [max +4.26% | p75 -0.01% | median -0.21% | p25 -2.13% | min -23.43%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 16 | regressed 0 | improved 0 | new 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------

Only significant changes:
| status | name                                                    | calls |     ins |  ins Δ% | HI |  HI Δ% | SMI |  SMI Δ% |
|--------|---------------------------------------------------------|-------|---------|---------|----|--------|-----|---------|
|   +    | bench_write_events::Settling                            |     1 |  29.38M |  +4.39% | 14 |  0.00% |   0 |   0.00% |
|   +    | bench_write_events                                      |       |  30.00M |  +4.26% | 15 |  0.00% |   0 |   0.00% |
|   -    | bench_upgrade_full_depth::pre_upgrade::save_snapshot    |     1 |  13.28M |  -2.55% |  0 |  0.00% | 128 |   0.00% |
|   -    | bench_upgrade_1000_no_fills::pre_upgrade                |     1 |   1.47M |  -2.62% |  0 |  0.00% | 128 |   0.00% |
|   -    | bench_get_my_trades                                     |       |  48.62M |  -2.66% |  0 |  0.00% |   0 |   0.00% |
|   -    | bench_upgrade_1000_no_fills::pre_upgrade::save_snapshot |     1 |   1.39M |  -2.78% |  0 |  0.00% | 128 |   0.00% |
|   -    | bench_get_order_book_ticker                             |       |   8.60K | -13.34% |  0 |  0.00% |   0 |   0.00% |
|   -    | bench_get_order_book_depth_default                      |       | 570.07K | -21.54% |  0 |  0.00% |   0 |   0.00% |
|   -    | bench_get_order_book_depth_max                          |       |   4.61M | -23.43% |  0 |  0.00% |   0 |   0.00% |

ins = instructions, HI = heap_increase, SMI = stable_memory_increase, Δ% = percent change

---------------------------------------------------
CSV results saved to canbench_results.csv

@mbjorkqvist
mbjorkqvist marked this pull request as ready for review August 6, 2026 11:00
@mbjorkqvist
mbjorkqvist requested a review from a team as a code owner August 6, 2026 11:00
@zeropath-ai

zeropath-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown

No security or compliance issues detected. Reviewed everything up to 4ad0ff7.

Security Overview
Detected Code Changes
Change Type Relevant files
Enhancement ► canister/canbench_results.yml
    Update benchmark instruction counts and timings across multiple benches

@gregorydemay gregorydemay left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mbjorkqvist !

@mbjorkqvist
mbjorkqvist added this pull request to the merge queue Aug 12, 2026
Merged via the queue into main with commit e2655d5 Aug 12, 2026
26 checks passed
@mbjorkqvist
mbjorkqvist deleted the mathias/chore-refresh-deps-canbench-baseline branch August 12, 2026 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants