fix(ntx-builder): actors self healing after error when submitting#2371
fix(ntx-builder): actors self healing after error when submitting#2371SantiagoPittella wants to merge 2 commits into
Conversation
Yes, I cannot check with the value in the database for the account commitment on the monitor, but the in-memory account is not getting updating after a failed transaction submission (the only cases where we re-fetch from the database the accoutn is when spawning the actor and when a submitted transaction expires). This changes adds a re-fetch when submission fails too.
Yes, this applies there too. I plan to cherry pick the commit there |
igamigo
left a comment
There was a problem hiding this comment.
LGTM! I left some questions mostly out of curiosity, so feel free to disregard.
I'm not entirely sure merging this to next first is preferable to merging it to main (I think you run some risks of still having conflicts when merging next back to main later on, but not super sure about this).
| // A rejected submission (e.g. an account-commitment mismatch) means our in-memory | ||
| // account has diverged from the committed chain. Reload it from the DB below so the |
There was a problem hiding this comment.
Do we know how it could diverge? The NTX should currently always get committed blocks, no?
There was a problem hiding this comment.
In the DB we have the account version of the streamed committed blocks, but in the actor we have an in-memory version of the account that gets updated applying in place each transaction's account patch. The divergence could happen if the account thinks that a submitted transaction was applied and the coordinator didn't receive that update yet or the account was updated by someone else
| /// Any error reaching this method is therefore terminal for the candidate: the batch's notes | ||
| /// are marked failed and the actor moves on. |
There was a problem hiding this comment.
nit: Maybe the new behavior could be specified here. If we also want to keep this function non-mutating, this could return an error so it can be reacted to from the outside
Conflicts will arise for sure no matter to which branch is merged first because the actors impls are a bit different in each. But those shouldn't be much work considering that this is a pretty small change. Though in any case the conflcits can be solved in the cherry-pick |
Summary
Why
The network transaction builder's actor held a single in-memory copy of its account for its whole lifetime, refreshed from the DB only at spawn and on transaction expiry. When a submission was rejected (e.g.
AccountCommitmentMismatch: the declared initial account commitment no longer matches the committed chain), the actor kept the stale in-memory account and re-executed against it, re-declaring the same stale commitment and failing repeatedly until an unrelated expiry or a respawn forced a reload. This showed up as repeatedSubmitProvenTxrejections for a network account whose committed state had already moved on.How
On a submission rejection (
NtxError::Submission), reload the account from the DB before the next selection, mirroring the existing expiry-reload path. Re-execution stays gated on the next committed block (the actor returns toNoViableNotes), so it re-syncs without tight-looping if the local DB is momentarily behind.Changelog