Skip to content

feat(mega-evme): batch replay and verification tooling - #366

Open
RealiCZ wants to merge 71 commits into
cz/chore/upgrade-revm-40from
cz/feat/evme-replay-tooling
Open

feat(mega-evme): batch replay and verification tooling#366
RealiCZ wants to merge 71 commits into
cz/chore/upgrade-revm-40from
cz/feat/evme-replay-tooling

Conversation

@RealiCZ

@RealiCZ RealiCZ commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Turns mega-evme replay into a self-contained equivalence-verification tool: replay whole blocks or transaction lists in one process, verify every replay against its on-chain receipt, and persist reusable fixtures and RPC caches safely under concurrency.

  • Batch replay: --block <N> / --tx-file <LIST> replay many transactions in a single process (one provider, one cache, each block executed once), emitting one NDJSON entry per target; single-transaction output stays byte-identical.
  • Receipt verification: --verify-receipt fetches each target's on-chain receipt and compares status / gasUsed / logs, reporting a structured diff; mismatches drive a dedicated exit code.
  • Fixture sweep: --dump-fixture-dir <DIR> bulk-dumps self-validating state-test fixtures (bench corpus format) with per-target fidelity gating.
  • Concurrent-safe caching: cache persistence takes a file lock and merges with on-disk state, so multiple processes can share one --rpc.cache-dir; cache merge consolidates existing per-worker caches; capture envelopes get optimistic-concurrency protection for the external-env snapshot.
  • Cache capacity: --rpc.cache-size is replaced by --rpc.cache-max-entries (0 = effectively unlimited), so long verification runs stop silently evicting early entries.
  • Rate-limit clarity: --rpc.rate-limit is renamed to --rpc.cu-per-sec (old name kept as a visible alias), with a warning for self-throttling values.
  • Request timeout: --rpc.request-timeout (default 30 s, 0 disables) bounds every HTTP request, so a stalled endpoint fails fast and retries instead of hanging the process.
  • Exit codes and structured errors: one documented taxonomy (0 success / 1 execution or input error / 2 verification mismatch / 3 RPC failure); --json runs always end with a machine-readable error object on failure.

All changes are confined to bin/mega-evme and docs/mega-evme; no consensus code is touched.

Testing

  • Full suite (cargo test -p mega-evme) plus envelope-gated offline batch integration tests.
  • Live mainnet verification: whole-block batch replays with --verify-receipt (29/29 and 25/25 receipt matches on fresh blocks), two-process concurrent cache sharing with no lost entries, and fixture sweeps self-validated through the state-test runner.
  • CI-grade lint: workspace clippy under -D warnings, rustfmt, cargo-sort, Prettier on docs.

Follow-up fixes (2026-08-11)

Rebased onto the current cz/chore/upgrade-revm-40 tip (repository rules forbid force-pushes, so the base sync landed as merge commit 62e1dc0; zero conflicts), plus three fixes:

  • Replay correctness — RPC account existence (36081e7): JSON-RPC cannot express "this account was never created" (eth_getBalance / eth_getTransactionCount / eth_getCode all answer zero), so the forked backend materialized never-created accounts as existing empty accounts. That flipped the EIP-7702 per-authorization refund condition: a brand-new authority was judged already-in-trie and each replayed authorization refunded 12,500 gas the chain did not, making replayed gasUsed under-report the on-chain receipt. The forked state now maps the all-zero answer back to None. This is safe because an existing-but-empty account cannot occur post-EIP-161, which every chain this tool replays has had from genesis. Tests: DB-level boundary matrix (all-zero → nonexistent; balance-only / nonce-only / code-only → still existing) plus an execution-level mock-transport test asserting a type-4 transaction with a fresh authority costs exactly 12,500 gas more than with an existing authority (verified to fail without the fix).
  • Batch replay cache policy (409de9a): online batch replay (--tx-file / --block) now engages the on-disk RPC cache only when --rpc.cache-dir is passed explicitly, and otherwise behaves as --rpc.no-cache-file. Rationale: a linear history scan's request keys are block-scoped and essentially never repeat across runs, so a shared cache file buys almost no hits — while the clean-exit persist re-reads, merges, and rewrites the whole file under a cross-process lock, a tail that grows linearly with the file (minutes at multi-GB sizes) and serializes concurrent batch workers into hour-long queues. Defaulting batch to no persistence makes the exit cost zero and independent of cache size, which dominates the alternative (incremental/sharded persist) that would still pay the per-process load cost and add format complexity. Single-transaction replay, run/tx --fork, and capture mode keep their previous defaults; --rpc.no-cache-file, --tx-file / --block / --verify-receipt semantics, and the exit-code taxonomy are unchanged. Tests: a batch run against a seeded default-path cache file leaves it byte-identical and creates no cache file anywhere (so a 26k-target batch exits with no cache tail at all), an explicit --rpc.cache-dir still persists, and single-transaction replay still persists by default.
  • tx --raw decoding (a77ed86): DecodedRawTx held the real envelope yet hand-mapped every variant into a TxEnv; it now derives the transaction through the upstream FromTxWithEncoded impl (which also fills the deposit parts and the enveloped bytes for L1 fee calculation), so new transaction types are picked up with the dependency instead of a manual mapping. The CLI-flag path (TxArgs) has no real envelope and is unchanged. Tests: EIP-155 signed-vector decode, deposit-envelope decode, and CLI-override behavior.

Review-fix batch + coherent --override.spec (2026-08-12)

22 commits (16 changes + 2 test-hardening + 4 merges), all in bin/mega-evme/ and docs/mega-evme/; workspace suite 1846 green, clippy/fmt/prettier/cargo-sort clean.

  • Batch target classification is now total over the (blockNumber, blockHash) resolution space: a mined answer without an inclusion hash and the contradictory null-number-with-hash shape both fail the target as rpc instead of being queued or misread as pending; a block-body hash that resolves to null aborts as a typed RPC inconsistency (exit 3) that still names the vanished transaction for the abort sweep.
  • The single-transaction path now matches batch on fetch coherence: target metadata is classified exhaustively before any block fetch, the parent block must link to the fetched block, the target must anchor to it (reported inclusion hash and body membership), block-body nulls reuse the same typed rpc variant (the user-supplied hash keeps its definitive not-found, exit 1), and a pending replay fetches the latest block once and reuses it for both roles.
  • replay --override.spec is now a coherent what-if: the executor receives a schedule synthesized from the forced spec (activation from the spec, per-fork parameters delegated to the chain config), so pre-block predeploys, EIP-2935/4788 gating, block-level limits, and EVM semantics all derive from one spec; without an override the behavior is byte-identical (verified by a binary-output diff probe). Replaying an old block under a newer spec deliberately installs predeploys that never existed at that height. Note: mainnet/testnet schedules currently publish only the Rex5 registry parameters, so --override.spec Rex6+ on those chains fails closed with a message naming the missing config (previously it crashed with a code-hash mismatch).
  • Every cache-file write now takes the sidecar lock: cache merge locks its output and folds the current on-disk file in under the lock (two-process serialization tests), lock-acquisition failure fails closed instead of degrading to an unlocked write, --rpc.clear-cache unlinks and loads inside one critical section, and the provider persist classifies the on-disk shape before writing so it can no longer replace a capture envelope. Safeguard diagnostics (chain-identity, output replacement) now reach stderr at default verbosity.
  • Capture hygiene: the capture transport no longer bakes result: null answers into fixtures — offline replay reports a cache miss naming the request instead of a frozen not-found.
  • Robustness: the panic hook writes fallibly, so a closed stdout (--json | head) ends with the documented exit 1 instead of a SIGABRT; the fixture pre-map's absent-means-nonexistent shape is pinned by tests; raw-tx decoding gains signed EIP-2930/1559/7702 vectors with full field assertions.

Review threads fixed by this batch are resolved; the batch-mode --rpc.clear-cache thread stays open pending a semantics decision.

Exit-taxonomy unification + batch-cache opt-in (2026-08-12, second batch)

11 commits, all in bin/mega-evme/ and docs/mega-evme/; workspace suite 1891 green, clippy/fmt/prettier/cargo-sort clean.

  • --rpc.clear-cache is a disk-cache opt-in for batch replay: the documented recovery flag now engages the cache (delete under the sidecar lock, start empty, persist on exit) instead of being silently dropped; the --rpc.no-cache-file combination's actual behavior (clear does not run) is documented as is.
  • Receipt handling is one contract across all modes: dump and verify classify a null, divergent, or unfetchable receipt identically (rpc, exit 3) with byte-identical error objects, and every receipt must belong to the transaction it was requested for — a receipt served for another transaction is an RPC inconsistency in single verify, single dump, and batch alike.
  • Batch classification is order-independent and truthful: per-target inclusion validation against the fetched block replaces the first-seen anchor; an anchored target absent from the block body is an endpoint contradiction (rpc); a non-target transaction aborting the block drives the run's exit class through a separate floor without corrupting the per-target totals; one unavailable receipt under --verify-receipt --dump-fixture-dir is counted once while both result fields are emitted; NDJSON output follows the documented (block, tx_index) order with absent targets last; a block containing none of its targets is answered without being executed.
  • Failed receipt fetches keep the replayed result: the target's execution facts stay on its NDJSON line with the failure under verification.error / fixture.error, per the documented keep-your-result policy; the documented jq selectors cover both failure shapes.
  • Pre-block RPC failures classify correctly: an unanswered state read inside the EIP-2935/EIP-4788 system calls exits 3 (was 1), matched at the cause boundary through named wrapper constants so an execution error merely embedding "RPC error:" text cannot misclassify. Known residue, deliberately out of scope: a keyless-deploy sandbox DB failure is collapsed to a selector-only revert inside the executor and cannot be classified from the tool side.
  • The --block 0 (invalid request, exit 1) vs endpoint-reported block-0 inclusion (contradictory endpoint data, exit 3) asymmetry is now documented rationale rather than an accident.

All four previously open review threads are fixed by this batch and resolved.

Served-answer authentication + classification consistency (2026-08-12, third batch)

Four fixes from the Codex review round on 8a8cad0, each with red-green regression tests:

  • Fetched transactions are authenticated before execution: all three eth_getTransactionByHash consumers (batch loop, single-path preceding loop, single-path target) recompute the hash from the served consensus encoding and re-derive the sender from the signature, refusing mismatches as an inconsistent-endpoint failure (exit 3). The response's own hash/from fields are never trusted — notably, alloy's trie_hash()/tx_hash() return the cached server-supplied hash for RPC-deserialized transactions, so authentication hashes the encoding explicitly.
  • Hashless block aborts attribute to the in-flight transaction: a rejection that names no hash (block-gas admission, for one) now lands on the transaction whose iteration raised it instead of sweeping the aborter itself as an unanswered peer.
  • A null endpoint-resolved block is an RPC failure: on the single path, the replayed block and its parent are fetched at heights the endpoint itself resolved, so a null answer is the divergent-views class (exit 3), matching batch; BlockNotFound (exit 1) stays reserved for user-supplied heights.
  • Local clear-cache failures exit 1: lock-acquisition and unlink failures classify as input errors, not RPC failures — retrying or switching the endpoint cannot fix the local filesystem.

Docs: the exit-code taxonomy in overview.md now records the authentication and resolved-block-null conventions; state-management.md records the clear-cache failure class. All four review threads from this round are fixed and resolved.

RealiCZ added 13 commits August 4, 2026 14:42
Canonical flag is now --rpc.cu-per-sec; --rpc.rate-limit remains a
visible alias. Clarify that the value is a CU/s budget, not RPS, and
warn once at provider build when retries are on and the budget is <100.
Delete --rpc.cache-size and introduce --rpc.cache-max-entries (default 0 =
never evict) so verification workloads keep large RPC caches. The cache
layer is always installed; 0 maps to u32::MAX capacity. Capture/replay
conflict lists and docs updated.
Alloy SharedCache preallocates its LRU hash table to full capacity, so
mapping "unlimited" to u32::MAX caused multi-GB RSS on every default
online run. Cap at 2^20 entries (cheap preallocation, covers far more
than any observed corpus) and add a construction-reality unit test.
Replay many transactions in a single process: build one provider and one
RPC cache, group targets by their containing block, and execute each block
once while recording every target's result. Batch mode emits NDJSON with
--json (one line per target, result or error entry), exits non-zero when a
target hit an infrastructure failure, and rejects the flags that only have
single-transaction semantics (fixture dump, overrides, forced spec, trace,
state dump).

The single-transaction path is unchanged; its output stays byte-identical.
Command dispatch now maps errors instead of propagating with ?, so the
error handler runs and diagnostics stay off stdout.
Share --rpc.cache-dir across processes via exclusive sidecar lock plus
re-read-merge on persist (provider cache and capture envelopes). Add
mega-evme cache merge for offline consolidation of both shapes.
Add `replay --verify-receipt`, which fetches each replayed transaction's
on-chain receipt and compares it against the receipt the replay produced:
success status, gas used, and the emitted logs (count plus each log's
address, topics, and data). Works in single-transaction and batch mode.

The comparison lives in a new `replay/verify` module as a pure function
over the consensus facts of both receipts, so it is independent of how
either receipt was obtained. The verdict is reported as a `verification`
object in JSON (absent without the flag) and as one verdict line in
human-readable output; a mismatch fails the run through a dedicated
`VerificationMismatch` error.

Anything that prevents the comparison from running is reported as an
infrastructure failure rather than a mismatch: a receipt the endpoint
cannot serve or has pruned, a receipt describing a different inclusion
than the replayed block (reorg or divergent endpoint), and pending
transactions.
Add batch sedimentation of self-validating EEST fixtures so a tx list or
block can be swept into <DIR>/<tx_hash>.json in one run. Per-target
fidelity/BLOCKHASH/unsupported-shape gates skip with a recorded reason
instead of failing the run; write failures stay infrastructure errors.
…e output

Every failure now maps onto one of four documented exit codes through a single
module: 0 success, 1 execution/internal error, 2 receipt verification mismatch,
3 RPC/transport failure. The mapping matches the error enums exhaustively, and
`main` is the only place that turns a command result into a process status.

Batch replay aggregates its per-target failures into a structured error carrying
the counts by class, so the run-level precedence (execution before rpc before
mismatch) is resolved from data instead of a formatted message.

On failure a run reports once: an `error: <message>` line on stderr, plus — with
`--json` — a compact `{"error":{"code","kind","message"}}` object as the last
stdout line, so a machine-readable run never ends with empty stdout. Per-target
NDJSON lines and all success-path output are unchanged.
…n checks

Reject conflicting non-null external_env on envelope persist (same rule as
cache merge), validate rpc-cache-{id}.json chain identity on provider merge,
classify malformed --rpc URLs as InvalidInput (exit 1), and type envelope
re-read hard vs degradable without substring matching.
Report a failed fixture dump on the target's own result line instead of
replacing it, so the receipt verification still runs and its mismatch is
counted; the failed dump remains an execution-class failure.

Classify targets swept up by a mid-block abort as unanswered (`rpc`) with
a message naming the aborting cause, reserving `not_found` for the target
the endpoint actually denied, and emit them in block transaction-index
order so the stream stays ascending by (block, index).

Route argument-parsing failures through the structured output, report a
capture-persist failure on stderr next to the run error that owns the
exit code, and stop mapping a `BatchFailed` with no counts to success.

Classify a block execution error by the database failure behind it, so a
state read that fails mid-execution exits 3 instead of 1; the pre-block
system calls and the keyless-deploy sandbox render their cause into a
message before it reaches the mapping and stay execution-class.
…c JSON

Carry load-time external_env through capture persist so intentional
--bucket-capacity refreshes win while true concurrent conflicts hard-error;
canonicalize bucket lists; always report non-aborting swept targets as rpc;
emit structured JSON on panic under --json.
Stop RpcCacheStore::new_envelope from re-reading the capture file after
build_capture_provider already loaded it. A concurrent writer between the
two reads could make C look like the load-time baseline so an A-derived
persist silently overwrote C. Carry the first-load snapshot as a parameter.
Default 30s bounds hung endpoints so they surface as retryable transport
errors (exit 3 after retries) instead of hanging the process forever.
@mega-maxwell

mega-maxwell Bot commented Aug 4, 2026

Copy link
Copy Markdown

Claude review status

Living comment — rewritten in place. The review workflow keeps this single comment up to date instead of posting a new one each round, so it always describes the latest reviewed commit and the earlier text is intentionally gone. No reply is needed here; reply to a finding in its own review thread, and answer an open question in a reply on this PR. The next review round reconciles your answer.

🛠️ Review did not finish

Attempted bd5bedc2..7da1ae12 · updated 2026-08-12T11:51:47+00:00

This round did not publish: MODEL_ACTION_FAILED in phase review_retry. Anything listed below is from the last round that did. Re-run the workflow or push a new commit to try again.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

🧬 Mutation testing — ✅ PASS

Nothing to test — no mutants were generated on the changed lines.

@claude

claude Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Documentation Impact

This PR adds a new top-level mega-evme cache command (bin/mega-evme/src/cache/, wired in src/cmd.rs). The docs/mega-evme/ spec pages and bin/mega-evme/AGENTS.md have since been updated to cover it (STRUCTURE now lists src/cache/ and src/common/exit.rs, thanks). One more reference was missed:

Agent / Skill Files

File Reason
AGENTS.md (repo root, also CLAUDE.md via symlink) The Workspace Structure table's mega-evme row still reads "CLI tool for EVM execution (run, tx, replay)" — missing the new cache subcommand added by this PR.

This update can be included in this PR or in a follow-up.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

🧬 Mutation testing

No results at target/mutants/mutants.out — nothing was mutated (e.g. no mutatable changes).

@codspeed-hq

codspeed-hq Bot commented Aug 4, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 382 untouched benchmarks


Comparing cz/feat/evme-replay-tooling (7da1ae1) with cz/chore/upgrade-revm-40 (98ad6bd)

Open in CodSpeed

@RealiCZ RealiCZ added comp:mega-evme Changes to the `mega-evme` tool spec:unchanged No change to any `mega-evm`'s behavior dependencies Pull requests that update a dependency file rust Pull requests that update rust code api:compatible Only new interface or API is introduced. Existing software is compatible. comp:doc Changes in the documentation labels Aug 4, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8c47fd128

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/mega-evme/src/replay/batch.rs Outdated
Comment thread bin/mega-evme/src/replay/batch.rs Outdated
Comment thread bin/mega-evme/src/replay/batch.rs
Comment thread bin/mega-evme/src/replay/verify.rs Outdated
Comment thread bin/mega-evme/src/replay/batch.rs
RealiCZ added 2 commits August 4, 2026 15:09
Defer fixture writes until block finish succeeds; classify construction
failures as fixture errors rather than skips; validate parent-block hash
linkage; reject receipts with a null blockHash as infrastructure errors;
stamp receipt inner logs with block/tx identity and block-global indices.
The single-transaction replay path derives its preceding-transaction hashes
from the block body the endpoint already served, so a lookup resolving to
null contradicts an answer that endpoint gave itself. Report it as
BlockBodyTransactionNull (rpc class, exit 3) instead of TransactionNotFound
(execution class, exit 1), matching the batch driver.

The initial user-supplied target lookup keeps TransactionNotFound: nothing
the endpoint served claimed that hash exists, so the null is a definitive
answer about the caller's own question.
Serialize --rpc.clear-cache under the same sidecar exclusive lock as
persist/merge and fail closed when the lock cannot be acquired. Classify
on-disk reread in save_cache_atomic so corrupt degrades to ours-only while
foreign shapes skip the write with a visible warning.
The parent/block linkage guard passes even when both numbered fetches
coherently answer from a replacement block the target is not part of: the
lookup reported inclusion in one block, the endpoint served another. The
target then executed after every transaction of that block, which counted
as preceding, and the run reported a plausible wrong result with exit 0.

Anchor a mined target to the block that is about to be replayed: the
inclusion hash the lookup reported must equal the fetched block's hash, a
mined lookup without an inclusion hash is rejected as an unanchored view,
and the fetched body must list the target, since its position there is
what defines the preceding transactions. Each violation is an RPC
consistency failure (exit 3), matching the batch driver's classes.

A pending target fetched the latest height twice, once as its state base
and once as the block it is replayed in, and skipped every coherence
check: two divergent answers produced a mixed-view execution reporting
exit 0. Fetch that block once and fill both roles from it, so the two
cannot disagree at all.
Keep the sidecar exclusive lock from before unlink until after the
exists-check and load_cache so a concurrent persist or cache merge
cannot recreate the file in the released window and have the clearing
invocation load the entries the user asked to remove.

Add a cross-process test that injects a cache file while clear is
queued on the lock and asserts the injected entries do not survive.
The single-transaction replay path decided "pending" from `block_number`
alone, so an inclusion hash paired with a null number was replayed against
latest with exit 0 — every inclusion and body guard skipped — while a block
number paired with a null hash was only rejected after the block and parent
fetches, where a missing block or a broken parent linkage could mask it.

Match the `(block_number, block_hash)` pair exhaustively right after the
target lookup: both present is mined, neither is pending, and the two mixed
shapes fail as RPC failures from the metadata alone, before any block is
fetched. The mined arm carries its inclusion hash forward, so the later
inclusion guard no longer needs a null case at all.

@mega-maxwell mega-maxwell Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Review needs attention — 1 finding(s)

0 blocking · 0 should-fix · 1 suggestion(s) · 0 open question(s)

Reviewed head d75183e1.

Details are attached inline.

Comment thread bin/mega-evme/src/replay/cmd.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d75183e145

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/mega-evme/src/replay/cmd.rs Outdated
Comment thread bin/mega-evme/src/replay/verify.rs
RealiCZ added 11 commits August 12, 2026 11:08
Validate each --tx-file target's reported inclusion hash against the
fetched block instead of a job-level first-seen anchor, so same-height
stale/canonical pairs get order-independent outcomes. Classify
anchored-but-absent targets as rpc (endpoint self-contradiction) rather
than not_found; genuine null resolution stays not_found.
Batch replay defaults the on-disk RPC cache off and forced `no_cache_file`
whenever `--rpc.cache-dir` was absent, which silently swallowed
`--rpc.clear-cache`: the documented recovery flag parsed, did nothing, and
left the polluted default-path cache for the next non-batch run.

Deleting the cache file only means something while the disk cache is
engaged, so `--rpc.clear-cache` now opts a batch run back in exactly as
`--rpc.cache-dir` does. The file is cleared under the sidecar lock, the run
starts empty, and the cache persists on exit. An explicit
`--rpc.no-cache-file` still wins over both flags and keeps its existing
meaning: with no cache file in play there is nothing to delete, load, or
persist.

Covered by unit rows over the flag combinations and binary-level tests that
seed a default-path cache under a fake HOME.
When a non-target aborts a block, count classify(err) into the batch tally so
the run exit reflects the root cause while swept targets stay rpc/unanswered.
Deferred fixture discards inherit the abort class, and body-tx transport
failures carry the hash via BlockBodyTransactionFetch.
The single-transaction --dump-fixture fidelity gate fetched the target's
on-chain receipt with its own error mapping: a null receipt became
TransactionNotFound (exit 1) and a divergent-inclusion receipt became
Other (exit 1), while --verify-receipt classified both identical endpoint
conditions as RpcError (exit 3).

Route the dump path's fetch through verify::fetch_receipt and map the
inclusion check to RpcError, so both modes report the same exit code,
the same failure class, and the same message for an unanswered or
divergent receipt.
Route dump-dir and verify-receipt unanswered receipt questions as rpc-class
findings on kept result lines (exit 3), early-return when no job targets sit
in the fetched body, and document the --block 0 vs resolved-into-0 asymmetry.
eth_getTransactionReceipt is queried by transaction hash, but nothing
checked that the answer describes that transaction: the guard compared
only block hashes, and ReceiptFacts discards transactionHash. An
inconsistent endpoint or a tampered capture could therefore have verify
report a verdict about a different transaction — including a spurious
match when the consensus facts coincide — and have the dump path anchor
a fixture to it.

Validate the identity in verify::fetch_receipt, the single seam that
single verify, single dump and batch all fetch through, and classify a
mismatch as an RPC failure whose message names both the served and the
requested hash.
Non-target aborts floor the run exit without inflating per-target
failure counts. Shared receipt failures under --verify-receipt and
--dump-fixture-dir count once while both result fields stay present.
Same-block entries emit in documented (block, tx_index) order with
absent-last placement. Docs match anchored-absence rpc and the dual
unverified jq selector.
# Conflicts:
#	docs/mega-evme/commands/replay.md
Pre-block EIP-2935/EIP-4788 system-call database failures are rendered into
BlockValidationError message fields by mega-evm, so the typed EvmeError chain
is gone before exit classification. Recover the RPC class from stable Display
prefixes this crate owns (RPC error: / RPC transport error:), so offline cache
misses and transport failures during pre-block no longer look like permanent
execution errors to retry scripts.
Strip only the documented pre-block Display wrappers before matching RPC
prefixes with starts_with, so an execution-class message that merely embeds
"RPC error: " no longer misclassifies as exit 3.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a8cad003b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/mega-evme/src/replay/batch.rs
Comment thread bin/mega-evme/src/replay/batch.rs Outdated
Comment thread bin/mega-evme/src/replay/cmd.rs Outdated
Comment thread bin/mega-evme/src/common/provider/mod.rs
…ed hash

A load-balanced endpoint or a tampered offline capture can answer
eth_getTransactionByHash(H) with another transaction. All three fetch
sites (batch loop, single-path preceding loop, single-path target) now
recompute the hash from the served consensus encoding and re-derive the
signer from the signature before executing anything, refusing mismatches
as an inconsistent-endpoint failure (exit 3). The response's own hash
field and from field are never trusted: alloy caches the served hash
into trie_hash()/tx_hash(), and as_recovered() trusts the served from.

Mock harnesses now serve internally-authentic transactions (hash and
from computed from the signed body); the execution-abort tests switch
from body tampering (now refused at the fetch) to draining the sender's
parent-block balance.
…saction

A rejection raised about a transaction does not always embed its hash in
the error (the block-gas admission check's
TransactionGasLimitMoreThanAvailableBlockGas, for one). Attribution from
error introspection alone then swept the aborter itself as an unanswered
peer: its NDJSON line said rpc with a 'aborted before this transaction'
message while the run exit was already floored to execution.

The batch loop now records which transaction's iteration raised the
abort and attributes the error there first; introspection remains the
fallback for errors raised outside the loop (a failed finish can still
name a transaction).
…lure

On the single-transaction path, the replayed block and its parent are
fetched at heights the endpoint itself resolved — the target's inclusion
metadata, or the reported latest height for a pending target. A null
answer there is the endpoint contradicting itself (reorg in progress, or
a load-balanced endpoint serving divergent views), which the batch path
already reports as a retryable rpc failure; the single path reported a
definitive exit-1 'Block not found' for the same context. Both fetches
now fail with the divergent-views rpc classification (exit 3);
BlockNotFound stays reserved for user-supplied heights.
… errors

A --rpc.clear-cache that fails locally — the sidecar lock cannot be
acquired, or the cache file cannot be unlinked — was wrapped as an RPC
error, telling automation the endpoint failed to answer (exit 3) even
though retrying or switching the RPC cannot fix the local filesystem.
Both sites now classify as InvalidInput (exit 1), the same class as
cache merge's lock failure. Docs cover the new classification, plus the
transaction-authentication and resolved-block-null conventions from the
two preceding fixes.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd5bedc28f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ReplayMode::Batch(batch_mode) => self.run_batch(&mut pctx, batch_mode).await,
};

let persist_result = pctx.cache_store.persist();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid persisting pending transaction metadata

When --verify-receipt is used on a transaction that is still pending, get_transaction_by_hash returns Some(tx) with no inclusion metadata, run_single rejects it, and this unconditional failure-path persist writes that response to the provider cache. Subsequent runs then keep receiving the cached pending object instead of querying the endpoint after the transaction is mined, so verification remains stuck on the pending error until --rpc.clear-cache is used. Fresh evidence beyond the earlier null-result rebuttal is that the cited Alloy behavior caches every Some response, which includes this pending shape; skip provider-cache persistence for this failure or evict unmined transaction responses.

Useful? React with 👍 / 👎.

/// would fire twice.
fn maybe_warn_low_cu_per_sec(&self) {
if let Some(msg) = cu_per_sec_warning(self.max_retries, self.compute_units_per_sec) {
warn!("{msg}");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Emit the low-CU warning at default verbosity

When a user configures --rpc.cu-per-sec below 100 without also passing at least -vv, this warn! event is discarded because LogArgs::init installs an off filter at the default verbosity. The documented warning therefore never appears in the normal invocation where it is intended to explain severe self-throttling, potentially leaving batch replays running unexpectedly slowly; emit this user-facing diagnostic directly on stderr (or through the existing always-visible warning helper) rather than only through tracing.

Useful? React with 👍 / 👎.

@mega-maxwell mega-maxwell Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Review needs attention — 1 finding(s)

0 blocking · 0 should-fix · 1 suggestion(s) · 0 open question(s)

Reviewed head bd5bedc2.

Details are attached inline.

let is_pending = mined.is_none();

let (state_base_block, block_number) = if let Some((n, _)) = mined {
(n - 1, n)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] Single-tx replay path lacks batch's block-0 guard, underflows on n - 1

For an inconsistent / load-balanced endpoint that resolves a hash into block 0, debug builds panic on u64 overflow; release builds wrap to u64::MAX, then surface a generic missing_resolved_block(u64::MAX) RPC error rather than the batch path's typed contradictory-endpoint-data message. This breaks the PR-body invariant "the single-transaction path now matches batch on fetch coherence" and the documented --block 0 (exit 1) vs endpoint-reported block-0 inclusion (exit 3, contradictory data) asymmetry.

Suggested fix: Before line 574, mirror the batch guard: if n == 0 { return Err(ReplayError::RpcError("endpoint resolved the target into block 0, which has no parent block to fork from: contradictory endpoint data".into())); } (or extract a shared helper the batch and single paths both call). Add a mock-transport regression test alongside the existing single-target verification tests.

Three mainnet CREATEs on the Rex spec halted after their constructor's
checkpoint had been committed, so the constructor's log reached a receipt the
chain records as empty — a receipts-root divergence caught by the full-history
replay gate.

Capture them together with their on-chain receipts and replay them offline
under --verify-receipt, so the expectation is the chain's own receipt rather
than a hand-written value. Neutering the result-seam log strip turns both tests
red.

The capture is committed compressed; the shared fixture helper extracts it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api:compatible Only new interface or API is introduced. Existing software is compatible. comp:doc Changes in the documentation comp:mega-evme Changes to the `mega-evme` tool dependencies Pull requests that update a dependency file rust Pull requests that update rust code spec:unchanged No change to any `mega-evm`'s behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants