docs(release): make "disclosure never yields to a stale test" an invariant - #478
Merged
Merged
Conversation
…riant The integration suite is red, the quickest route to green is to stop showing a screen, and the commit that does it looks like a test fix. Writing the rule down is the cheapest defence against that, because the pressure arrives disguised as housekeeping and the result would look like progress. Tests adapt to newly required disclosure. Firmware disclosure is never removed to satisfy a stale test. If a test fails because the device now shows more, the test is wrong. The extra THORChain screens disclose affiliate-fee bytes that were previously signed invisibly. Also replaces the known-red section, which was wrong in two ways. It said the integration stall was pre-existing at 4406197 -- a controlled run (same test files, only firmware varying) showed 1af2ffe passes and head hangs, so this release introduced it. And it could not enumerate the failures at all, because the job was killed before producing JUnit XML. With #477 the suite completes in 134s and the list is finite: 22 failed, 362 passed, 47 skipped, every failure a stale pinned expectation and none a firmware defect. Recorded as a table with an owner per row, since that is now the concrete work list for the repin. Adds the two measurement traps that cost real time here: do not run the suite sequentially against one emulator, and treat the PINNED submodule commit as the test source of truth rather than the working tree -- reading master's copy of test__sign_transformERC20 produced a confident and wrong "this breaks nothing". Refs #466 #474 #477
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation only. Two changes.
1. Writes down the invariant as a release rule
The pressure to invert this is real and arrives disguised as housekeeping: the integration suite is red, the quickest route to green is to stop showing a screen, and the commit that does it looks like a test fix. The extra THORChain screens disclose affiliate-fee bytes that were previously signed invisibly — deleting them to make CI pass would reintroduce the defect this release exists to close, while looking like progress.
If a test fails because the device now shows more, the test is wrong.
2. Replaces the known-red section, which was wrong twice
It said the integration stall was pre-existing. It is not. A controlled run — same test files, only the firmware varying — shows
1af2ffe7depasses both stalling tests andrelease/7.14.2head hangs them. This release introduced it. My earlier comparison used440619703, which already contained most of the release work.It could not enumerate the failures at all, because the job was killed before producing JUnit XML. With #477 the suite completes in 134s, so the list is now finite and has an owner per row:
Transaction signing disabled by policy(TON)Chain Id out of boundsEnable AdvancedMode to blind-sign(TRON)Timeout >60.0sArbitrary contract data signing disabled by policyEnable AdvancedMode to blind-sign typed hashesStructured EIP-712 disabled…22 failed, 362 passed, 47 skipped. No firmware defect among them. Thedeps/python-keepkeyrepin is the single action that resolves most of it.Two measurement traps, recorded so they are not rediscovered
test__sign_transformERC20enables AdvancedMode on python-keepkey master and does not at the pinned81e581f. Reading the wrong one produced a confident and wrong "this breaks nothing" on fix(evm): bind transformERC20 clear-signing to full calldata #468.Note on the red state
The branch reports red, and after #474 that is a fact rather than an appearance:
ci-gatefails when any required job isfailure,cancelledorskipped, so a gate-stage failure can no longer make the build graph vanish behind green ticks.