Problem
Crash recovery treats a connection-onboarding intent that cannot be replayed as fatal, and the failure is permanent: the intent journal is never cleared on that path, so every subsequent open of the interactive runtime-policy store fails the same way. The app cannot start against that storage root again without manually deleting runtime-policy-onboarding.json.
Mechanism (line refs at main@e05308be7):
openInteractiveRuntimePolicyStoresForWrite runs coordinator.recoverForWrite() before handing out any writer (packages/storage/src/runtime-policy-stores.ts:192).
recoverForWrite → recoverConnectionOnboarding (coordinator.ts:209, :1317) replays the journaled intent through applyConnectionOnboarding → prepareOnboardingUpsert.
- When the catalog has drifted so that the slug the intent resolves to is occupied by a connection with a different
connectionId, the upsert throws invalid_document — "Onboarding intent conflicts with the connection id" (connection-catalog-document.ts:534) — which recovery wraps as commit_outcome_unknown — "Connection onboarding recovery did not converge" (coordinator.ts:1330) and rethrows.
- Nothing clears the intent on this path, so the next open replays the same intent into the same conflict. Verified experimentally: three successive opens with fresh root leases all fail identically.
Reproduction (verified against real stores)
- Onboard a connection so an intent is journaled, and hard-kill the process between
writeConnectionOnboardingIntent and clearConnectionOnboardingIntent (the crash window the journal exists for).
- Out of band, delete and re-create the connection at the same slug (new
connectionId) — e.g. by another build, a restored backup, or manual edit of connection-catalog.json.
- Reopen the store:
commit_outcome_unknown / invalid_document: Onboarding intent conflicts with the connection id, permanently.
Reaching the state requires a hard kill plus out-of-band catalog drift, so it is rare — but the blast radius (the store never opens again) is disproportionate to the fault (one stale journal entry describing a row that provably no longer exists).
Why fail-closed is the wrong shape here
Fail-closed is right for I/O errors and undecodable documents: retrying may converge and throwing protects unknown state. This case is different — the conflict proves the intent is obsolete: the connection it named is gone, and its slug now belongs to a different identity. Replaying it can never become correct, so refusing to open the store protects nothing; it converts a completed-or-abandoned onboarding into a bricked installation.
Suggested direction
In recoverConnectionOnboarding, treat the id/slug-conflict (invalid_document from the upsert's identity cross-check) as an obsolete intent: log it, clearConnectionOnboardingIntent, and continue opening. Keep the current fail-closed behavior for every other error class (I/O failures, undecodable journal, vault errors), where retry can still converge. A regression test can hand-write the drifted catalog + stale journal and assert the store opens and the journal is cleared.
Provenance
Found during adversarial review of #3467 (which adds one more input to the same pre-existing throw sites but does not change the recovery semantics); reproduced there through a path untouched by that PR. Details in that PR's review-record comment.
Problem
Crash recovery treats a connection-onboarding intent that cannot be replayed as fatal, and the failure is permanent: the intent journal is never cleared on that path, so every subsequent open of the interactive runtime-policy store fails the same way. The app cannot start against that storage root again without manually deleting
runtime-policy-onboarding.json.Mechanism (line refs at
main@e05308be7):openInteractiveRuntimePolicyStoresForWriterunscoordinator.recoverForWrite()before handing out any writer (packages/storage/src/runtime-policy-stores.ts:192).recoverForWrite→recoverConnectionOnboarding(coordinator.ts:209,:1317) replays the journaled intent throughapplyConnectionOnboarding→prepareOnboardingUpsert.connectionId, the upsert throwsinvalid_document— "Onboarding intent conflicts with the connection id" (connection-catalog-document.ts:534) — which recovery wraps ascommit_outcome_unknown— "Connection onboarding recovery did not converge" (coordinator.ts:1330) and rethrows.Reproduction (verified against real stores)
writeConnectionOnboardingIntentandclearConnectionOnboardingIntent(the crash window the journal exists for).connectionId) — e.g. by another build, a restored backup, or manual edit ofconnection-catalog.json.commit_outcome_unknown / invalid_document: Onboarding intent conflicts with the connection id, permanently.Reaching the state requires a hard kill plus out-of-band catalog drift, so it is rare — but the blast radius (the store never opens again) is disproportionate to the fault (one stale journal entry describing a row that provably no longer exists).
Why fail-closed is the wrong shape here
Fail-closed is right for I/O errors and undecodable documents: retrying may converge and throwing protects unknown state. This case is different — the conflict proves the intent is obsolete: the connection it named is gone, and its slug now belongs to a different identity. Replaying it can never become correct, so refusing to open the store protects nothing; it converts a completed-or-abandoned onboarding into a bricked installation.
Suggested direction
In
recoverConnectionOnboarding, treat the id/slug-conflict (invalid_documentfrom the upsert's identity cross-check) as an obsolete intent: log it,clearConnectionOnboardingIntent, and continue opening. Keep the current fail-closed behavior for every other error class (I/O failures, undecodable journal, vault errors), where retry can still converge. A regression test can hand-write the drifted catalog + stale journal and assert the store opens and the journal is cleared.Provenance
Found during adversarial review of #3467 (which adds one more input to the same pre-existing throw sites but does not change the recovery semantics); reproduced there through a path untouched by that PR. Details in that PR's review-record comment.