perf(nds): store only the nodes a commit changed

[emturner]

Part of RV-998.

What

Committing stores only the nodes a commit actually changed. One commit; the compaction and bloom
filter tuning that used to sit with it is now its own PR on top of this one, because it helps a
different thing and carries its own risk.

Why

Committing rewrote every node resolved in memory, changed or not: the recursion in
Storable for Node stopped only at a child that had never been loaded. The bytes matched what was
already stored, but they are new writes as far as RocksDB is concerned, so each commit flushed them
into fresh files that no earlier checkpoint shares.

The cost is cumulative, which is the key to reading the numbers below. The resolved set grows
monotonically as a state is committed against repeatedly — for a checked-out state it is everything
touched since — so the redundant write volume ramps with commit count.

How

StoreId identifies each live store, and a StoredIn memo on Node records which one the body
reached. Storable for Node returns early when the memo names the store being written, skipping the
subtree: nodes are content-addressed, so an unchanged node cannot have a changed descendant.
invalidate_hash clears the memo, so it is dropped by exactly the mutations that change what a node
hashes to. Loadable for Node sets it, which is what stops a checked-out tree rewriting what it
touches.

The memo names a store rather than setting a flag because copy_database gives the destination a
copy of the source's store and the same in-memory nodes, so one node owes a write to both. Its low
bit records whether the value data went too. It sits behind an AtomicU64 because store takes
&self.

Performance

gc_space, 200k keys, 1000 modified keys per commit, measured on this branch against
"emturner@working-tip". Per-commit figures are for commits that did not follow a compaction:

commit	bytes added, before	bytes added, after	commit ms, before	commit ms, after
1	2.0 MiB	2.0 MiB	87	94
2	3.3 MiB	2.0 MiB	103	83
3	4.4 MiB	2.0 MiB	115	77
6	7.1 MiB	2.0 MiB	158	73
10	11.0 MiB	2.0 MiB	190	71
14	12.4 MiB	2.0 MiB	230	71
19	14.9 MiB	2.0 MiB	277	70

Totals over 20 commits: repository 708.2 -> 560.6 MiB, bytes the history pins 703.7 -> 468.5 MiB
(31.0 -> 19.2 MiB per commit), sharing between consecutive checkpoints 92.7% -> 98.0%.

Node writes are identical in both arms: 139206, 6960 per commit, 260 B average. The same nodes
are stored either way; only redundant rewrites of unchanged ones went away. That is the check that
this is not trading correctness for bytes.

This is invisible to the lifecycle benchmark, by construction

database_lifecycle shows no change on any scenario, including the two second-commit ones. That
is a property of the harness, not of the commit: it re-clones a pristine, fully lazy checkout for
every iteration (BatchSize::PerIteration) and commits at most twice, so it measures commit 1-2 of
the table above, where the two arms are identical. On a lazy tree the pre-existing early return
already prunes everything unresolved, and resolving a path in order to modify a key also changes
every node on it — so there is nothing left to skip.

Verified rather than assumed: with OCTEZ_NDS_ROCKSDB_DISABLE_AUTO_COMPACTION=1, which removes the
compaction contention that dominates those scenarios (second commit 1.411 s -> 567 ms on
working-tip), the node-skip still shows no change. There is no saving being masked.

Reproduce:

gc_space --keys 200000 --commits 20 --modified-keys 1000 --sample-every 1 --repo-dir <dir>

Manually Testing

make all

Run here: cargo nextest run -p octez-riscv-durable-storage (225/225), cargo clippy -p octez-riscv-durable-storage --all-targets --features="rocksdb unstable-test-utils" (clean),
scripts/format.sh --all.

TMPDIR must be on a volume with room — the default /tmp is a 1.5 GB tmpfs and the database tests
exhaust it, which surfaces as OpenRocksDbFailed ... No space left on device rather than a real
failure.

Regressions

No goldenfiles move: hashing and the stored representation are unchanged, only which writes are
skipped.

Tasks for the Author

• Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes).
• Eliminate dead code and other spurious artefacts introduced in your changes. — StoreId::NONE
  had no call sites and is now the memo's sentinel. Default for StoreId is reached through
  #[derive(Default)] on InMemoryKeyValueStore, so it stays, with its docs corrected to say so.
• Document new public functions, methods and types.
• Make sure the documentation for updated functions, methods, and types is correct.
• Add tests for bugs that have been fixed. —
  test_copied_database_commits_shared_nodes_into_its_own_store replaces a checked-in proptest
  seed. Verified to bite: making the memo ignore the store id fails it on both backends with
  CommitDataMissing.
• Explain changes to regression test captures when applicable.
• Write commit messages in agreement with our guidelines.
• Self-review your changes to ensure they are high-quality.
• Complete all of the above before assigning this MR to reviewers.

[github-actions]

Benchmark results for revision 2fc210c:

Metric	Duration	TPS
Mean	1.49418744s	26.771
Worst	1.502498969s	26.622
Best	1.487233543s	26.896
Standard Deviation	±3.905503ms	±0.070

Full results

Run	Transfers	Duration	TPS
1	40	1.501129358s	26.647
2	40	1.502498969s	26.622
3	40	1.497523518s	26.711
4	40	1.494112342s	26.772
5	40	1.491649675s	26.816
6	40	1.490850669s	26.830
7	40	1.498392913s	26.695
8	40	1.490357767s	26.839
9	40	1.494101129s	26.772
10	40	1.489749136s	26.850
11	40	1.497599779s	26.709
12	40	1.493822549s	26.777
13	40	1.496310161s	26.732
14	40	1.491601406s	26.817
15	40	1.494014298s	26.774
16	40	1.493873679s	26.776
17	40	1.487233543s	26.896
18	40	1.488437341s	26.874
19	40	1.494883285s	26.758
20	40	1.495607286s	26.745

Compare the results above with those for the default branch.

[github-actions]

Durable storage space metrics for revision 2185dc0, at pull request
scale. The scale itself lives in durable-storage/Makefile.

Metric	Value	Unit
dead_node_bytes_per_commit	1.48	MiB
dead_node_fraction	37.4	%
repo_bytes_per_commit	11.76	MiB
checkpoint_shared_fraction	80.3	%
checkpoint_shared_fraction_steady	96.6	%

Full report

repository: /tmp/nix-shell.QNCklm/.tmpJlnMZ5
shape: 1 database(s) x 100000 keys x 32 B values, 10 commits x 1000 modified keys
prepopulated 100000 keys in 0.5s
committing the base state...
base state committed as bbcc53be48c9f9285a3607046839524e6d2152561d2e95d9e760e6a665d72d44 in 1.0s

 commit    blob MiB    live MiB    dead MiB   dead %   value MiB    disk MiB  commit ms  commits MiB     new MiB  shared %  rewrote MiB
      0        24.8        24.8         0.0     0.0%        15.4        46.1          0         42.0           -         -            -
      1        26.3        24.8         1.5     5.6%        15.4        47.8         92         43.7         1.7     96.1%          0.0
      2        27.7        24.8         2.9    10.6%        15.4        49.5         78         45.5         1.7     96.2%          0.0
      3        29.2        24.8         4.4    15.2%        15.4        97.8         75         47.2         1.7     96.3%          0.0
      4        30.7        24.8         5.9    19.2%        15.4        99.6         72         95.5        48.2      0.0%         47.1
      5        32.2        24.8         7.4    22.9%        15.4       101.3         69         97.2         1.7     96.6%          0.0
      6        33.7        24.8         8.9    26.3%        15.4       103.1         69         98.9         1.7     96.7%          0.0
      7        35.1        24.8        10.3    29.4%        15.4       141.8         78        101.2         2.2     95.9%          0.5
      8        36.6        24.8        11.8    32.3%        15.4       143.5         68        139.4        38.2     31.1%         36.2
      9        38.1        24.8        13.3    34.9%        15.4       145.3         69        141.1         1.7     97.0%          0.0
     10        39.6        24.8        14.8    37.4%        15.4       163.6         73        142.9         1.7     97.1%          0.0

over 10 commit(s):
  dead node data grew by 14.8 MiB, 1.48 MiB per commit
  blob column family grew by 14.8 MiB, 1.48 MiB per commit
  59689 node(s) written, 5968 per commit, averaging 260 B stored per node
  history pins 142.8 MiB (107.3 MiB Merkle, 35.3 MiB values, 0.1 MiB other), growing 10.1 MiB per commit
  75.2% of what the history pins is Merkle node data, which a shared store would stop duplicating per commit
  sharing: 80.3% of the bytes checkpoints pinned were already on disk (410.2 MiB of 510.9 MiB), adding 10.1 MiB in 2 new file(s) per commit
  3 of 10 measured commit(s) followed a compaction; without them sharing is 96.6%, so compaction costs 16.3 points of it, rewriting 83.8 MiB the earlier checkpoints still pin
  levels at the last commit:
    L0: 7 file(s), 5.8 MiB
    L6: 2 file(s), 53.0 MiB
  37.4% of the blob column family is now dead (14.8 MiB of 39.6 MiB)
  repository occupies 163.6 MiB across 11 commit directories (611.0 MiB apparent, and hard links save 451.6 MiB)

simulated directory-level collection: removed 9 database commit(s) and 9 manifest(s)
  repository went from 163.6 MiB to 121.7 MiB, freeing 42.0 MiB
  retained history now occupies 100.9 MiB
  still dead and now unreachable by any directory deletion: 14.8 MiB of node data (37.4% of the surviving blob column family)

These figures are for catching a regression before it merges. Compare them with
the longer run on the default branch, which measures more commits.

[codecov]

Codecov Report

❌ Patch coverage is 95.38462% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.46%. Comparing base (8828ce4) to head (58b2d8a).
⚠️ Report is 1 commits behind head on emturner@working-tip.

Files with missing lines	Patch %	Lines
durable-storage/src/persistence_layer.rs	70.00%	3 Missing ⚠️

Additional details and impacted files

@@                   Coverage Diff                    @@
##           emturner@working-tip    #1200      +/-   ##
========================================================
- Coverage                 90.50%   90.46%   -0.04%     
========================================================
  Files                       153      153              
  Lines                     32995    33060      +65     
  Branches                  32995    33060      +65     
========================================================
+ Hits                      29862    29909      +47     
- Misses                     2136     2151      +15     
- Partials                    997     1000       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.