fix(appliance): commit gate consumes doctor --json, not just a curl (#852) - #864
Merged
Conversation
…852) The A/B slot-commit gate in os/overlay/pithead-boot marked a slot good on a single `curl https://localhost/` — any HTTP answer. "The dashboard answers" is a strict subset of "the stack is alive": a slot whose monerod/p2pool/tari crashed while caddy+dashboard keep serving still committed, defeating the case-2 self-heal the A/B design promises. The plan (dual-distribution-plan.md) always named the gate `pithead doctor --json` and claimed it "checks containers" — but doctor's Containers section only printed `docker compose ps` and never FAILed on a crashed revenue container, so wiring doctor's exit code alone would not have fixed the bug. - doctor now genuinely fails on a crashed revenue container. A new pure classifier `revenue_container_verdict` + `check_revenue_containers` split the revenue path by sync behaviour: chain nodes (monerod/tari + payout wallets) run throughout a days-long initial sync and report healthy early (liveness probes / #718 scan-grace), so a down/unhealthy one is a real crash; the sync-gated miners (p2pool/xmrig-proxy, stopped by the dashboard until the node syncs, #35) are expected down, so only a running-but-unhealthy miner is a fault. This keeps a slow initial sync from ever blocking the commit. - pithead-boot now gates mark-good on the curl AND `pithead doctor --json` (curl first, cheap; doctor only has to pass once — a still-starting node just loops). A slot serving a dashboard while mining is dead no longer commits. - docs reconciled: appliance-wizard boot contract step 4, the plan's gate bullet, and the appliance-release phase table now describe the real gate. Coverage: - tests/stack: unit tests for revenue_container_verdict (syncing vs crashed, incl. empty-.State robustness) and the pithead-boot gate wiring. - tests/os: the provision reboot leg — where the real self-commit gate runs on a provisioned stack — now asserts the gate PASSES on the healthy still-syncing stack yet REFUSES once monerod is stopped (the mining-dead-but-serving slot a curl-only gate committed). This is the assertion whose absence let the curl-only gate ship green. Closes #852 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The A/B slot-commit gate in
os/overlay/pithead-bootmarked a slot good on a singlecurl https://localhost/— any HTTP answer. "The dashboard answers" is a strict subset of "the stack is alive": a slot whose monerod/p2pool/tari crashed while caddy+dashboard keep serving still committed, defeating the case-2 self-heal the A/B design promises.The plan always named the gate
pithead doctor --jsonand claimed it "checks containers" — but doctor's Containers section only printeddocker compose psand never FAILed on a crashed revenue container. So wiring doctor's exit code alone would not have fixed the bug. This PR fixes doctor first, then uses it.Health signal chosen
Commit now requires both: the
localhostcurl (proves the derived-config → caddy → dashboard chain) andpithead doctor --jsonexiting clean. doctor gains an honest revenue-container check.Syncing vs crashed (the distinction that avoids bricking)
A new pure classifier
revenue_container_verdictsplits the revenue path by how it behaves during a days-long initial sync:#718scan-grace), so they report healthy from early on regardless of height. Down or unhealthy = a real crash → hold the commit.#35), so a down miner is the expected steady state on a fresh box → fine, commit. Only a running-but-unhealthy miner is a fault.So a still-syncing box commits (the
#35sync gate holds mining but the OS is good) while a genuinely broken one reverts. The boot loop retries, so a still-startingnode just waits rather than committing on partial evidence.Coverage
revenue_container_verdictacross syncing vs crashed states (incl. empty-.Statecross-engine robustness) + the pithead-boot gate wiring.Docs
Boot-contract step 4 (appliance-wizard), the plan's gate bullet (dual-distribution-plan), and the appliance-release phase table all now describe the real curl + doctor gate.
Closes #852
🤖 Generated with Claude Code