feat: reset Dust state and re-apply cNight UTxOs during migration

[ozgb]

Overview

To support the upcoming Ledger 9 hard fork - this PR resets Ledger Dust state and re-applies cNight UTxOs to the ledger as a multi-block migration.

Limitation

ctime for all cNight UTxOs will not match real ctime from Cardano

The creation time for cNight UTxOs when applied to the ledger state is usually set to match the creation time of the UTxO on Cardano. This PR is implemented to retrieve cNight UTxO data using a mixture of cnight pallet data and data from the ledger state pre-dust-wipe.

The creation time for night utxos is only available as a commitment in the ledger state - it is not publicly visible. This means that to get the real creation time of the UTxOs, we would have to query db-sync.

Instead, the replay stamps fork block time - dust.time_to_cap() (~1 week) as the creation time for every re-applied UTxO.

DUST accrues linearly from ctime to a cap of night_value * night_dust_ratio, reached after time_to_cap. Backdating by exactly that much means every holder is at their DUST cap the moment the replay lands — i.e. the pre-fork steady state, since anyone who had held cNIGHT for a week was already capped. Holders should see nothing change.

The alternatives, for the record:

ctime	holder experience at the fork
fork block	0 DUST for everyone, refilling over ~1 week in proportion to holdings — large holders recover in minutes, small holders cannot pay a fee for days
fork - time_to_cap (this PR)	at cap immediately; over-credits only cNIGHT locked in the last week, bounded by a cap it would have reached anyway
real, from db-sync	also at cap for anything older than a week; needs a new consensus-critical db-sync query + inherent running during the fork

• Q: Should we query db-sync and get the real creation times? No. The pallet stores only hash("asset_create" ‖ tx_hash ‖ index), never the UTxO id, so db-sync cannot be queried by nonce — the node would have to snapshot the whole live cNIGHT set (a new unbounded db-sync query), match by hash, page it in through a new inherent, and have every validator reproduce it exactly in check_inherent. That is ~600-900 lines of consensus-critical code that runs once, during the fork, and cannot be fixed in flight. It also isn't more "correct": real ctime restores holders to as-if-never-spent, which over-credits recent spenders just the same.
  • Chosen solution: set all existing cNight UTXOs to their Dust Generation cap - this means that after the fork, all registered cNight holders will have max Dust up to their cap. Need to notify downstream wallets

👀 Watching the migration (QA — read this before hunting for events)

In Polkadot.js Apps, DustReapplyStarted is the only replay event you will see. That does not mean the replay stalled.

It is emitted from on_runtime_upgrade, so its phase is Initialization. Everything else the replay produces — DustReapplyCompleted / DustReapplySkipped / DustReapplyBatchFailed, the replay's SystemTransactionApplied, and pallet_migrations' own MigrationAdvanced / MigrationCompleted / UpgradeCompleted — is deposited from a multi-block-migration step, which FRAME runs in inherents_applied() after the block's last extrinsic. Their phase is ApplyExtrinsic(n) where n equals the block's extrinsic count — an index no extrinsic in that block has. PJS renders events grouped under the extrinsic that claims them, so it drops these entirely. They are in System::Events regardless.

Worked example from a local hardfork_e2e run (new code applied at #34):

block	event	phase	in PJS
35	CNightObservation.DustReapplyStarted	Initialization	✅
35	MultiBlockMigrations.UpgradeStarted	Initialization	✅
36	MidnightSystem.SystemTransactionApplied (52 dust Creates)	ApplyExtrinsic(3) — block has extrinsics 0–2	❌
37	CNightObservation.DustReapplyCompleted { applied: 52, skipped: 13 }	ApplyExtrinsic(3)	❌
37	MultiBlockMigrations.MigrationCompleted, UpgradeCompleted	ApplyExtrinsic(3)	❌

Raw System::Events at that block 37 contains 0d 07 34000000 0d000000 — pallet 13 (CNightObservation) · variant 7 (DustReapplyCompleted) · applied = 0x34 = 52 · skipped = 0x0d = 13.

How to confirm it actually ran

• Node log — every one of these events is logged under its own name, so grepping the log for the event name finds it whether or not an explorer shows it:

  DustReapplyStarted: recorded pre-fork ledger state key for the dust generation replay
  ObservationsSkippedForMigration: skipping process_tokens (on-chain storage version StorageVersion(1) < StorageVersion(2)); MBM in progress
  DustReapplyCompleted: dust generation replay complete, 52 applied, 13 skipped
  

• Events, keyed by event rather than by extrinsic — state_getStorage on System::Events at the block hash, or subxt's events(). Anything that walks extrinsics and collects their events will miss these.
• Storage version — CNightObservation reaches 2 once the replay winds up, and PreForkStateKey is cleared. The version key is twox128("CNightObservation") ++ twox128(":__STORAGE_VERSION__:") — note the trailing colon; without it you get None at every block.
• ObservationsSkippedForMigration (new here) — emitted from the process_tokens inherent every block the migration gate holds, so this one is visible in PJS and brackets the window in which Cardano observations are being ignored.

hardfork_e2e now asserts the outcome rather than just the wind-up: it decodes the DustReapply* events over the fork window, prints each one, and requires a DustReapplyCompleted with applied > 0.

🗹 TODO before merging

• Update Ledger to rc.4 (includes Dust fix + state translation with reset)
• Ready

📌 Submission Checklist

• All commits are signed off (git commit -s) for the DCO
• Changes are backward-compatible (or flagged if breaking)
• Pull request description explains why the change is needed
• Self-reviewed the diff
• I have included a change file, or skipped for this reason:
• If the changes introduce a new feature, I have bumped the node minor version
• Update documentation (if relevant)
• Updated AGENTS.md if build commands, architecture, or workflows changed
• No new todos introduced

🧪 Testing Evidence

Please describe any additional testing aside from CI:

• Additional tests are provided (if possible)

🔱 Fork Strategy

• Node Runtime Update
• Node Client Update
• Other:
• N/A

Links

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93ceb4766d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Regenerate runtime metadata for new pallet storage

This adds new pallet storage items, but the f027c98c diff does not update any checked-in metadata/static/midnight_metadata*.scale artifact. Consumers and tests that rely on the repository metadata will still see the old cnight-observation storage layout/events, so please run the metadata rebuild and commit the regenerated metadata with this change.

AGENTS.md reference: AGENTS.md:L146-L149

Useful? React with 👍 / 👎.