release(v8.4.0): failure-class coverage by default (Phase A+B+C) - #89
Merged
Conversation
Turn the depth-gated verification machinery ON by default and make the gates GENERATE failure-class coverage, not only grade volunteered ACs. Phase A — split the evidence-independence floor off the Size×risk depth axis (evidence_floor = max(tier floor, AC-shape floor)); broaden Gate 7's adversarial trigger to every external-input boundary; tier-independent strongest-derivation oracle + per-AC [ORACLE-AUDIT]. Phase B — a default, always-on failure-class eval panel (L-CORRECTNESS / L-ROBUSTNESS / L-CONTRACT-CONFORMANCE / L-UNIFORMITY / L-SIMPLICITY, with a per-ticket [EVAL-PANEL]); Gate 9 Failure-Class Coverage (a per-boundary R1-R4 matrix); plus a dogfood-driven adversarial-key / prototype-pollution vector in L-ROBUSTNESS + Gate 9 R2. Phase C — an EC-SELFDOC evidence channel + a read-only doc-verifier agent (classes A/E); Gate 10 Peer-Set Uniformity (class D); a refute-then-synthesize merge replacing the lone-non-critical-FAIL-demoting majority-merge; and a Gate 9 R1 round-trip-losslessness cue. Every mechanism is kill-switched and feature-named (no bare measure labels — they collide with the prior plan's M-numbering). Validated by a 2-subject dogfood A/B (a CSV<->JSON library and a stateful key-value CLI). 821/821 skill-contracts, 144/144 path-consistency, full sweep green; the ac-evaluator <-> ac-evaluator-hi body stays byte-identical (CT-EV-MODEL-1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
v8.4.0 turns the harness's depth-gated verification machinery ON by default and makes the gates generate failure-class coverage rather than only grading volunteered ACs — bringing the default tier to parity-or-better with a max-effort build on a 2-subject dogfood A/B.
What ships (three bundled phases)
evidence_floor = max(tier floor, AC-shape floor)(any behavioral AC floors at+1-independent, Size-independent); Gate 7's adversarial trigger broadens to every external-input boundary; tier-independent strongest-derivation oracle + per-AC[ORACLE-AUDIT].L-CORRECTNESS/L-ROBUSTNESS/L-CONTRACT-CONFORMANCE/L-UNIFORMITY/L-SIMPLICITY,[EVAL-PANEL]); Gate 9: Failure-Class Coverage (per-boundary R1–R4 matrix); plus a dogfood-driven adversarial-key / prototype-pollution vector inL-ROBUSTNESS+ Gate 9 R2.EC-SELFDOCchannel +doc-verifieragent (classes A/E); Gate 10: Peer-Set Uniformity (class D); a refute-then-synthesize merge replacing the lone-non-critical-FAIL-demoting majority-merge; and a Gate 9 R1 round-trip-losslessness cue.Migration
Non-breaking; every mechanism is kill-switched (
constraints.eval_panel/failure_class_coverage/peer_uniformity/selfdoc_verification/refute_merge, plus the existingindependent_evidence/oracle_verification), all defaultauto. The refute-then-synthesize merge flips a default onexhaustivetickets (a lone non-critical FAIL now survives unless refuted);constraints.refute_merge: offrestores the prior majority-merge. See the[8.4.0]CHANGELOG entry for the full migration paragraph.Verification
bash tests/test-skill-contracts.sh821/821,bash tests/test-path-consistency.sh144/144, fullrun-all.shALL SUITES PASSED.plugin.json8.4.0== newest CHANGELOG[8.4.0](CT-MODE-14), real ISO date (CT-MODE-13).ac-evaluator↔ac-evaluator-hibody byte-identical (CT-EV-MODEL-1) across all paired edits.See
CHANGELOG.md[8.4.0].🤖 Generated with Claude Code