release(v8.5.0): Advertised-Accept-Set Conformance + agnosticism codification + P0 safety hardening - #92
Merged
Merged
Conversation
…cket sibling-guard Comprehensive eval-P0-batch increment on impl/eval-p0-batch. Multiple efforts are committed together because the contract-test file (tests/test-skill-contracts.sh) entangles their CTs and partial staging is unavailable in this environment. - Agnosticism (charter P0): bind a "Product/Language/Domain Agnosticism" section in CLAUDE.md (property-not-membership rule + the (A) user-product-substrate=agnostic / (B) harness-own-substrate=fixed line + meet-or-beat + HARD-LINE-vs-JUDGMENT enforcement boundary); de-falsify the detect-tautological-assertions.sh header (it is JS/Jest-Vitest-targeted, not "language/framework-agnostic"; other languages rely on the ac-evaluator semantic pass); demote the verification-depth criticality domain-name list to illustrative and promote the domain-independent property trigger (shared-boundary + correctness-critical) to primary. - De-contamination (CT-DECONTAM-1): product-instance recidivism guard over the 13-file normative set held at 0 hits; independent-oracle-harness uses a neutral worked example while keeping its four-part contract. - Cross-ticket sibling-guard (P-A): the decomposer emits a mandatory-present, unconditional cross-ticket `shared_input_boundary` signal naming the sibling tickets that share one external input class; create-ticket forwards each ticket's membership into BOTH its planner and its ticket-evaluator spawn; a "delegation is NOT an automatic n/a" clause forces each delegating sibling to carry its own hostile / accept-set failure-class row. New RED-first contract test CT-EV-SHARED-INPUT-XTICKET pins the signal + the clause across producer, schema, forwarder, canonical rubric, both ac-evaluator twins, planner, and ticket-evaluator. - eval-P0 batch + strictness-leniency hardening across skills / agents / hooks. Verification: test-skill-contracts 838/838, test-path-consistency 144/144; hook suites all green (pre-bash-safety 200, pre-write-safety 33, pre-edit-safety 35, pre-bash-contract-guard 8, hooks-lib 163, state-parsers 33, autopilot-continue 53); ShellCheck --severity=warning rc=0; ac-evaluator twins byte-identical (CT-EV-MODEL-1); CT-DECONTAM-1 / CT-AR-13 / CT-EV-GATE10-2 green. P-A sibling-artifact omission rationale (CLAUDE.md ## Modifications): the delegation-not-n/a clause is propagated to the canonical rubric + both ac-evaluator twins + planner + ticket-evaluator + the ticket-template R2 row, but intentionally NOT to the other CT-AR-13 sibling-guard carriers (test-authoring-guidance, verification-depth, implementer, test-writer) -- those are downstream authoring / impl guidance, not graders of the cross-ticket signal, which reaches the grading path via the planner self-audit + ticket-evaluator + ac-evaluator without them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NAC #4 in tests/test-ac-evaluator-static-rules.sh asserts that tautological- assertion-rules.md names no test framework "outside example blocks", but the implementation grepped the WHOLE file (fence-unaware), so a framework token inside a fenced worked example would false-trip it. Align the implementation with its own comment by filtering fenced code blocks before the count, transcribing the proven three-backtick toggle awk from hooks/pre-write-safety.sh:51-64. Behaviour-identical on the current tree (the rules file has 0 framework tokens and 0 fences, so the count stays 0 -> PASS). RED-first demo confirmed: on a fixture with one prose token + one FENCED token, the old whole-file grep counts 2 (false trip) while the fence-aware filter counts 1 (prose only). Suite 21/21, ShellCheck --severity=warning rc=0. Prerequisite for AASC: a future accept-set worked example carrying a runner token must not RED-fail this guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s (AASC prereq) The scratch-carve-out note said its "single exception ... for a computational AC", but point 5 of the same section ALREADY directs a behavioral, time-bounded watchdog probe of an external-input boundary via the .simple-workflow/scratch/ carve-out (and the EC-SELFDOC real-build probe likewise). That is a pre-existing internal inconsistency, independent of any new mechanism. Reconcile it: the carve-out permits EVIDENCE-GATHERING on already-built behaviour (computational oracle probe AND the behavioral watchdog / EC-SELFDOC probes this section already directs), NEVER a substitute for missing test coverage (the :199 ban is preserved in intent). Deliberately scoped: this names NO accept-set / complement-sweep (that is the gated AASC mechanism, deferred until the charter F-4 discriminating dogfood passes). Applied BYTE-IDENTICALLY to both ac-evaluator twins. New RED-first CT-EV-SCRATCH-EVIDENCE pins the clarification in both twins (token HEAD=0 before this change). Verification: test-skill-contracts 839/839, ac-evaluator twins byte-identical (CT-EV-MODEL-1 + cmp), CT-DECONTAM-1 green, ShellCheck --severity=warning rc=0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pe (charter F-1) Preserve the charter section I probe as a durable, runnable design-oracle under design-oracles/aasc-accept-set/ (it previously lived only in gitignored scratch). It pairs an independent hand-coded spec oracle with a property-generated complement corpus (the Unicode decimal-digit codepoints that are NOT ASCII, BMP + astral, enumerated by Unicode property so the generator names no script) and diffs each implementation black-box against the oracle. Reproduced: node probe.js -> buggy caught, correct clean (astral 0 is correct, the JS native parser is ASCII-only); python3 probe.py -> buggy leaks the full Unicode-Nd alphabet incl ~390 astral codepoints caught by construction, correct clean. Source names no script and carries no product-instance denylist token. This is a PROOF ARTIFACT, not normative plugin content: the AASC mechanism is NOT wired into the harness and its implementation stays gated on the charter section F-4 discriminating-subject live dogfood. The directory is run by no test suite or CI; test-path-consistency 144/144 and test-skill-contracts 839/839 stay green with it present. The illustrative boundary (strict unsigned decimal octet) is illustrative only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add loop-engineering framing in three README locations without growing the body or naming skills in the top blurb: - opening blurb: the Harness brings loop engineering to the development lifecycle via a closed inner loop (act -> verify -> correct -> continue) bounded by contract stopping conditions - Usage > Execution chains: annotate the autopilot per-ticket loop in patterns 2 and 3 with 'loop engineering fires'; the all-manual pattern 1 stays unannotated because the human drives that loop there - Why simple-workflow?: tie the three pillars to the inner loop of loop engineering and the engineering around it (no new bullet) Docs-only; path-consistency 144/144 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire Advertised-Accept-Set Conformance into the ac-evaluator: when a boundary's
contract advertises strict/canonical/lossless/limit (or a shared-input-class
sibling exists), the evaluator EXECUTES a generative grammar-complement sweep in
scratch against an independent hand-coded oracle, rather than only reasoning about
the accept set. Per the charter's innovation-theater analysis, the load-bearing
addition is the executed act, not more prose contract.
Load-bearing edits (twin/sibling byte-identical):
- ac-evaluator{,-hi}.md L-ROBUSTNESS lens: EXECUTE mandate + Grammar Card
(A/U/W/K) + four metamorphic relations (MR-FINITE/ALPHABET/CANONICAL/KEYFAITH,
MR-ALPHABET enumerating the Unicode decimal-digit property complement naming no
script) + two-tier oracle-authoritative FAIL gating (no false-positive storm) +
always-on + no-runnable-artifact caveat arm.
- ac-evaluator{,-hi}.md scratch carve-out: admit the sweep as a behavioral
evidence probe.
- ac-evaluator{,-hi}.md persisted observability: a "## Accept-set sweep" section
in the terminal-rewrite report body (per-boundary triggered/ran/astral/
divergences/authoritative lines) so non-execution is falsifiable.
- implementer.md / test-writer.md: producer-side commit-on-leak retained corpus
(a fixed rejection characterization test, conditioned on a sweep-found leak).
Scaffold: new skills/impl/references/accept-set-conformance-harness.md (copyable
shape, non-product worked example, illustrative-only); EC-METAMORPHIC in
evidence-channels.md; a test-authoring-guidance cross-link.
Tests: CT-AASC-1/2/3/5 (RED-first proven), new doc added as the 14th
CT-DECONTAM-1 normative-set file. test-skill-contracts 843/843, path-consistency
144/144, twins byte-identical, ShellCheck clean, CT-DECONTAM-1 zero
product-instance hits.
Efficacy (dogfood46, record-locator library, 3 siblings, patched tree): QUALIFIED
PASS. Self-elicitation is REAL (not theater) -- an evaluator actually executed a
property-enumerated BMP Nd-complement sweep (corpus-size corroborated by the tool
call and an independent oracle); the committed test layer now locks in the
Unicode-digit reject class (a one-character [0-9]->\d / isdigit regression makes
the committed suite go RED, the opposite of the prior baseline); product correct
with zero new bugs. meet-or-beat achieved over the baseline.
Known iterate targets (follow-up hardening): execution is non-uniform (only one of
three siblings ran a genuine property-enumerated sweep; one hand-picked a small
sample, one stood the sweep down on delegation grounds), the astral planes were
not swept, and the stderr observability twin did not fire. The kill switch
(constraints.accept_set_conformance) is intentionally deferred to the hardening
pass so this efficacy delta stays cleanly attributable; the absent-field default
keeps the mechanism active.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…x self-incrimination (load-bearing core)
Hardens the AASC accept-set sweep after the dogfood46 efficacy run showed
self-elicitation is real but non-uniform (1 of 3 evaluators ran a genuine
property-enumerated sweep; one hand-picked a sample, one stood the sweep down on
delegation grounds; astral was never swept). An adversarial design review trimmed
~60% of the proposed hardening as recognition-gated cue-accretion (the astral /
no-script / property-enumeration prose already existed and was ignored); this
ships only the load-bearing core.
- skills/impl/SKILL.md: the orchestrator computes a per-AC deterministic trigger at
the Step-15 handoff (a lexical strict/canonical/lossless/limit scan + the
shared_input_boundary sibling signal) and inlines it into the evaluator spawn as
`Accept-set conformance: {auto|off} triggered-on={AC-ids}`, emitting
[ACCEPT-SET-TRIGGER] to stderr. Roster updated (fields a-m, + Accept-set
conformance).
- agents/ac-evaluator{,-hi}.md (byte-identical): the lens READS triggered-on= and
sweeps exactly those ACs without re-recognizing; a triggered-on AC may NOT be
stood down on delegation grounds (kills the dogfood46 ticket-002 archetype by
coupling the obligation to the orchestrator's list); a black-box
no-implementation-peek prohibition (do not skip astral by inspecting the unit's
internals to argue an input class cannot occur); and a self-incrimination rule
(ran=y astral=n on a triggered AC is a NON-CONFORMANT shallow sweep, recorded as
such -- turning the persisted `## Accept-set sweep` line into a falsifiable
instrument).
- tests/test-skill-contracts.sh: CT-AASC-6 (the trigger wire) + CT-AASC-7
(implementation-peek + shallow-sweep), RED-first proven.
Deliberately NOT changed: the astral / no-script / property-enumeration prose
(already maximal); accept-set-conformance-harness.md (re-stating it is accretion);
the deferred kill-switch declaration L1/L2 + the stderr twin.
Tests: test-skill-contracts 845/845, path-consistency 144/144, twins
byte-identical, ShellCheck clean, CT-DECONTAM-1 zero product-instance hits.
Honest scope: the deterministic trigger MOVES the recognition from the proven-weak
evaluator to the orchestrator (a variance reduction, not a hard mechanical gate --
the orchestrator is still instructed, not hooked); the depth's only load-bearing
part is the self-incrimination falsifiability (astral coverage is made visible, not
forced). Efficacy is UNPROVEN until a re-dogfood shows the delegating sibling now
sweeps (triggered=y ran=y astral=y) where dogfood46 stood it down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(light/dark) Visualize the default /brief chain=on path below the README opening paragraph, tuned for how the README actually renders on GitHub: - inline: a short, large-text OVERVIEW (phases + the per-ticket loop and the in-/impl verify loop) that stays readable at GitHub's ~860px width - collapsed under <details>: the FULL flow with every skill, agent, hook, inter-phase handoff/harness note, and a harness-mechanism summary - both served via <picture> + prefers-color-scheme so they adapt to GitHub light/dark mode instead of glaring as a white block Loops are named PER-TICKET LOOP / VERIFY LOOP (not OUTER/INNER) to avoid colliding with loop engineering's scheduler 'outer' loop, which the plugin delegates to Claude Code (/loop, /schedule). Diagrams are Pillow-generated; the generators are committed beside the PNGs for regeneration. Glyphs are restricted to ASCII + safe arrows and on-line labels use opaque backgrounds, so no tofu boxes or overlaps. Verified by rendering at 860px (light + dark); path-consistency 144/144. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pus directive + worked shape + K-trigger binding
Brings the class-(b) STRUCTURAL-KEY-INJECTION (MR-KEYFAITH) lens to parity with the
class-(c) MR-ALPHABET directive, after the dogfood48 (b) run self-elicited the K-axis
sweep but generated a HAND-PICKED literal key list (the lens MR-KEYFAITH clause was
bare/advisory prose while MR-ALPHABET had a strong enumerate-by-property directive).
A /tmp prototype proved (b) is catchable by construction with a reflection-derived
generator that names no key.
- skills/impl/references/accept-set-conformance-harness.md: a worked REFLECTION-DERIVED
MR-KEYFAITH (b) shape (mirrors the MR-ALPHABET example) -- an independent
round-trip-faithfulness oracle + a generator that derives reserved/accessor keys by
reflection over the structure's type/prototype (naming NO key literal) + generic
hostiles (empty/duplicate/normalized-collision) + a black-box diff. ILLUSTRATIVE.
- agents/ac-evaluator{,-hi}.md (byte-identical): the MR-KEYFAITH clause gains a MUST
DERIVE-the-corpus-BY-REFLECTION directive (names no key, enumerates via reflection,
covers all reserved/accessor/colliding keys + generic hostiles; a single hand-picked
list is not conformant) pointing at the worked shape. The gating qualifier stays
ASSUMED/advisory (not upgraded this round).
- skills/impl/SKILL.md: the deterministic trigger now binds the K axis for a
shared_input_boundary sibling that builds a keyed structure from untrusted input,
even when no lexical strict/canonical/lossless/limit word is present.
- tests/test-skill-contracts.sh: CT-AASC-8 (the (b) shape) + CT-AASC-9 (the
reflection-derived corpus directive in both twins), RED-first.
Tests: test-skill-contracts 847/847, path-consistency 144/144, twins byte-identical,
ShellCheck clean, CT-DECONTAM-1 zero product-instance hits (directive + shape name no
key; verified by forced-text grep).
Honest scope: EDIT A/B are PROSE directives mirroring the (c) directive that works WITH
run-to-run variance (the AASC trigger+sweep is LLM-mediated, not a hard gate). This
strengthens (b) elicitation from bare/advisory prose to a MUST + worked shape; efficacy
(does the K sweep now derive-by-reflection) is to be validated by a re-dogfood on the
COMMITTED tree -- the prior dogfood49 was invalid because the shape was uncommitted and
did not load. The MR-KEYFAITH gating stays advisory; determinism would require a runtime
hook (deferred).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pgrade + reflection-corpus completeness + (b) design-oracle
Finishes class-(b) STRUCTURAL-KEY-INJECTION (MR-KEYFAITH) to full parity with the
class-(c) MR-ALPHABET mechanism, after dogfood50 validated the (b) sweep as
reflection-derived-effective (the K corpus self-elicited via dir(type) naming no
key; K bound reliably; product (b)-faithful; baseline (a)/(b)/(c) all validated,
with a run-to-run variance caveat).
- agents/ac-evaluator{,-hi}.md (byte-identical): the MR-KEYFAITH clause's temporary
"ASSUMED, not proven / advisory unless a concrete drop" qualifier is removed and
folded into the SAME standard two-tier oracle-authoritative FAIL gating as the
other MRs (a concrete round-trip-faithfulness violation on a lossless/strict
keyed boundary => FAIL; a legitimately-wide keyed boundary => advisory). The
corpus directive now names private/internal slot names and the FULL reflected
set (never truncated/sliced).
- skills/impl/references/accept-set-conformance-harness.md: the MR-KEYFAITH worked
shape gets the same gating-upgrade (lockstep), an oracle-independence hardening
(the round-trip expectation is computed from the INPUT PAIRS by last-write-wins,
never re-invoking the builder -- closes the dogfood50 circularity residual), and
private/internal-slot + FULL-reflected-set coverage (closes the dogfood49
private-slot-shadow + dogfood50 sliced-corpus residuals).
- design-oracles/aasc-keyfaith/{probe.js,probe.py,README.md}: a committed (b)
design-oracle (mirrors the committed (c) design-oracles/aasc-accept-set/) -- a
reflection-derived key generator (names no key literal) paired with an
independent round-trip-faithfulness oracle, proving the (b) class is caught by
construction, language-agnostic, false-positive-free. Verified: node probe.js
"buggy caught = true | correct clean = 0"; python3 probe.py "buggy caught = True
| private-slot leak caught = True | correct clean = True". Proof artifact, not
normative, not run by CI.
- tests/test-skill-contracts.sh: CT-AASC-10 (gating proven-upgrade, both twins,
dual-lever: folded-gate phrase present + "ASSUMED, not proven" removed),
CT-AASC-11 (oracle from input pairs), CT-AASC-12 (private/internal slots + FULL
reflected set, doc + both twins), RED-first.
Tests: test-skill-contracts 850/850, path-consistency 144/144, twins byte-identical,
ShellCheck clean, CT-DECONTAM-1 zero product-instance hits (the gating upgrade +
corpus directive name no key literal; forced-text grep over both twins + the doc = 0).
Honest scope: the gating upgrade inherits the already-validated two-tier gate (no
new over-FAIL risk -- the wide-boundary => advisory arm is unchanged); the corpus
completeness + oracle-independence are robustness hardening. None of this eliminates
the AASC sweep's run-to-run variance (that needs the deferred runtime hook). A live
(b)-leaky-narrow + (b)-wide A/B dogfood is owed as follow-up confirmation of the
gating upgrade (validated-by-inheritance + constructive-proof; not a blocker).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t-sweep conformance gate (metric-only) + 3-spawner kill switch Closes the #1 AASC residual: run-to-run VARIANCE. The accept-set sweep and its self-incrimination rule are ALREADY normative MUST in the ac-evaluator lens ("an `## Accept-set sweep` line with `ran=n`, or with `ran=y astral=n`, is a NON-CONFORMANT shallow sweep"), yet live dogfoods leaked them run-to-run because the rule is honoured only by the model RECOGNISING it (dogfood49 stood the K-axis sweep down on a subject dogfood48/50 had swept). A hook is the only lever independent of model recognition. Triaged adversarially (multi-agent): of 8 candidate items only ONE is a genuinely additive lever -- a runtime hook that reads an already-emitted artifact and gates deterministically. Prose-only items (stderr-twin, oracle-roundtrip tightening, private-slot mandate) were rejected as recognition-gated theater per charter section H; a spawn-prompt-rewriting "grep promotion" was rejected as infeasible (no hookable event between trigger-compute and subagent-spawn -- the cron-create wall); a cross-language negative-control proof was DEFERRED (no rustc/cargo/go on this host -- faking it would violate the don't-claim-unrun rule). - hooks/accept-set-verify.sh (new): PostToolUse(Write|Edit) gate over the persisted `## Accept-set sweep` section of an eval-round-{n}.md report. Reads the EMITTED 8-field line from disk and applies the lens's own rule with zero model recall: P1 stand-down (triggered=y ran=n, all axes), P2 shallow-astral (A/U axes), P3 sliced-corpus (A/U axes, < SW_AASC_CORPUS_FLOOR), P4 gating-consistency (authoritative divergence not driven to FAIL, all axes). P2/P3 are scoped to the alphabet/unicode axes (a reflection-derived key corpus for the K/W axes is legitimately small/ASCII, so flooring them would false-trip); caveat=no-runnable-artifact exempts P1/P2/P3 (compiled-language fail-open). Ships SW_ACCEPT_SET_CONFORMANCE_MODE=metric-only (fail-OPEN, exit code ALWAYS 0; unknown collapses to metric-only); `on` emits a PostToolUse decision:block surfacing the violated predicate to the evaluator. Skeleton (IN_PROGRESS) / n/a / non-eval / jq-missing all silent exit 0. - hooks/hooks.json: two top-level PostToolUse entries (Write + Edit) per the ordering rule. - skills/brief/references/policy-template.md, skills/create-ticket/references/autopilot-policy-reference.md, skills/impl/references/ac-evaluator-orchestration.md: the per-brief `constraints.accept_set_conformance` kill switch documented symmetrically across all 3 spawner surfaces (the field was already consumed by /impl Step 15 + honoured by the lens, but undocumented -- a half-wired L1/L2/L3 gap; mirrors constraints.refute_merge). - CLAUDE.md: SW_ACCEPT_SET_CONFORMANCE_MODE + SW_AASC_CORPUS_FLOOR env knobs. - tests/test-accept-set-verify.sh (new): 22 behaviour fixtures (every predicate, the no-runnable-artifact escape, K-axis no-false-trip, the dogfood50 clean shape no-false-positive, kill-switch tri-state, jq-absent fail-OPEN). - tests/test-skill-contracts.sh: CT-AASC-13 (hook registered >=2) + CT-AASC-14 (3-spawner policy-field parity), RED-first. Verify-by-construction BEFORE wiring: a /tmp fixture matrix proved the predicate gates correctly for ALL inputs (P1 reproduces the exact dogfood47/49 regression -> flagged; the dogfood50 clean shape -> NO false trip; K small-corpus + the no-runnable-artifact arm -> NOT over-blocked) -- and caught a real design bug (the no-runnable-artifact escape must exempt P3, not only P2) before any wiring. Tests: test-skill-contracts 852/852, test-path-consistency 144/144, test-accept-set-verify 22/22, ShellCheck --severity=warning clean, CT-DECONTAM-1 0 hits (14-file normative set; the hook + docs name no product/key/script literal), ac-evaluator twins byte-identical (CT-EV-MODEL-1; the GO set does not touch the twins). Honest scope: the hook catches STRUCTURAL self-incrimination (a triggered boundary not run / shallow / sliced), NOT field fabrication (a model could emit `ran=y astral=y corpus-size=750` without executing -- the R1 bound). It ships metric-only so the confirmation dogfood is its promotion gate and it cannot break the harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ensitive header (dogfood51 confirmation findings) The dogfood51 confirmation run (subject lossless-record, 3 tickets; ran the committed fdbbf74 -- version confound ruled out: the generated autopilot-policy.yaml carries the v8.5.0+ accept_set_conformance field) VALIDATED the AASC determinism hook's core: the full chain fired end-to-end across all 3 tickets, the K/(b) boundary self-elicited a reflection-derived corpus (dir(Record): 38/35/12 names incl __proto__/constructor/_data) with NO dogfood49 stand-down regression, the sweep EXECUTED for real (003: 440 inputs, 390 astral found, 0 divergences vs an independent oracle -- the R1 fabrication bound intact), and the product did not leak (95/95 + independent adversarial probes). But an offline re-run of the committed hook against the 6 real persisted reports showed the P3 corpus floor (256) would, in `on` mode, decision:block 2 of 4 sweep-bearing reports -- both triggered=y ran=y astral=y divergences=0 CONFORMANT sweeps (002-r1 A=5, itself a PASS report; 003-r2 A=105/U=105) -- a false trip on a wide-spec subject's legitimately-thin sweep. Two format-drift cases also blunted the hook (a capital-S `## Accept-set Sweep` header skipped a whole report; a descriptive corpus-size dodged the numeric P3 guard). Per charter section H (do not ship a heuristic blocker that false-trips), the fix is in the HOOK (recognition-independent), not in lens prose: - hooks/accept-set-verify.sh: DEMOTE P3 (corpus-size < floor) from BLOCKING to ADVISORY -- it now emits `[ACCEPT-SET-VERIFY] advisory: ...` (a note, never a block) in both modes. corpus-size is a weak depth proxy (a few astral probes can be deeper than hundreds of ASCII ones); astral (P2) is the real A/U depth gate. P1 (stand-down) / P2 (astral) / P4 (gating) remain the BLOCKING set. Plus: match the `## Accept-set sweep` header case-insensitively (a mis-cased header no longer skips the whole report -- it was blunting P1/P2/P4 too); and parse the leading integer of a descriptive corpus-size so an annotation cannot dodge the advisory. - CLAUDE.md + autopilot-policy-reference.md: document P3 as advisory + the case-insensitive header + the leading-int parse. - tests/test-accept-set-verify.sh: F4 now expects advisory (not block); plus Fcap (capital-S header + P1 -> block), Fthin (the 002-r1 thin-conformant PASS shape -> advisory, NO block), Fsuffix (descriptive corpus-size -> leading-int parsed -> advisory). 26/26. Authoritative re-test: the updated hook against all 6 REAL dogfood51 reports in `on` mode -> 0 blocks (3 advisory, 3 clean), exit 0 everywhere -- the promotion gate (no false trip on a conformant report) is now CLEARED. A genuine P1/P2/P4 violation still blocks (Fcap: capital-S header + triggered=y ran=n -> block). Tests: test-skill-contracts 852/852, test-path-consistency 144/144, test-accept-set-verify 26/26, ShellCheck --severity=warning clean, CT-DECONTAM-1 0 hits, ac-evaluator twins byte-identical (CT-EV-MODEL-1; twins untouched). Honest scope: this validates no-false-positive on real artifacts; the on-mode BLOCK path is still proven only by construction + the offline re-run, never fired live (every dogfood51 sweep self-reported divergences=0). Promote SW_ACCEPT_SET_CONFORMANCE_MODE=on only after a deliberately-leaky subject exercises the live BLOCK path. Generation-side depth variance (5 vs 750 keys on the same boundary in one run) is SURFACED by the advisory, not fixed here -- the hook is the backstop, not the primary lever. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… branch audit)
A holistic pre-dogfood audit of impl/eval-p0-batch (12 commits vs main, both axes
of the product/language/domain-agnostic x meet-or-beat philosophy, plus complete
resolution of every conversation/dogfood problem) returned fully-agnostic /
all-resolved / meets-or-beats / GO, and surfaced a small set of robustness +
doc-consistency residues in the determinism backstop. All are mechanism or
sibling-uniformity fixes (NOT recognition-gated prose); each closes a
silent-escape / contradiction class; none re-introduces the dogfood51 false-trip
class (every real dogfood51 report still 0 blocks in on-mode).
hooks/accept-set-verify.sh (header-drift + parse robustness; fail-OPEN intact):
- Gate 2b + the section awk now match the header at ANY hash depth (`#{1,6}`),
not only `##` -- a mis-LEVELED header (`### Accept-set sweep`, live-observed in
the dogfood51 003 eval-round-3) no longer lets a whole report silently skip the
gate (same drift class as the dogfood51 capital-S header, on the depth axis).
- the section awk start-pattern is now fully case-insensitive via tolower(), so it
accepts the SAME set as Gate 2b's `grep -i` (any case incl ALL-CAPS, not only a
flexed first letter) -- the two header-matchers can no longer disagree (an
all-caps header that passed Gate 2b would previously extract nothing -> silent
blunt).
- field_of value class `[^ ]+` -> `[^[:space:]]+` so a TAB-separated sweep line
still has its fields extracted (was silently un-parsed).
- line selector `boundary=*` -> `*boundary=*` so a reordered (boundary-not-first)
but complete line is no longer skipped (field_of is order-independent).
- an off-grammar boundary label (not A/U/W/K; live `boundary=pair-shape`) now
emits a stderr `[ACCEPT-SET-VERIFY] off-grammar boundary label=...` WARN -- a
relabeled alphabet boundary would otherwise dodge the A/U astral gate silently.
Deliberately a WARN, NOT a block (fail-toward-observability; never re-introduce
the dogfood51 false-trip class). Exit code stays 0 (fail-OPEN invariant intact).
skills/impl/references/evidence-channels.md: the EC-taxonomy reference both
evaluators cite still carried the stale "MR-KEYFAITH is ASSUMED, not proven /
advisory-only" posture that 3999964 already upgraded in both twins + the harness
doc -- restated as the SAME two-tier oracle-authoritative FAIL gate, so a reader
there no longer downgrades a genuine narrow-keyed-boundary leak the twins FAIL.
tests/test-skill-contracts.sh: CT-AASC-10 now ALSO scans evidence-channels.md for
the absence of the stale posture -- the sibling-artifact uniformity rule (CLAUDE.md
## Modifications) is now MECHANIZED, closing the gap that let the contradiction ship.
tests/test-accept-set-verify.sh: + Fhash (mis-leveled `###` header + P1 -> block),
Ftab (tab-separated line + P1 -> block), Fcaps (all-caps header + P1 -> block),
RED-first regressions for the three closed header/parse escapes.
Audit verdict (HIGH confidence, 11-agent adversarial workflow): agnostic_verdict=
fully-agnostic (CT-DECONTAM-1=0; every trigger/rubric/MR keys off a property --
lexical strict/canonical/lossless/limit, shared_input_boundary,
keyed-structure-from-untrusted-input -- not a named product/lang/domain;
MR-ALPHABET selects by the Unicode decimal-digit property, MR-KEYFAITH derives the
key corpus by reflection, both name no script/key); problems_resolved=all-resolved
(cue-accretion, AASC generation gap, non-uniform execution, run-to-run variance,
(b) hand-picked corpus, dogfood51 P3/format-drift each cite a concrete artifact);
quality_verdict=meets-or-beats (the four baseline MR classes are property-driven +
EXECUTED + proven by the committed design-oracles; the fail-OPEN hook can never
REMOVE a catch). GO for the next dogfood.
Tests: test-skill-contracts 852/852, test-accept-set-verify 29/29
(+Fhash +Ftab +Fcaps), test-path-consistency 144/144, ShellCheck --severity=warning
clean, CT-DECONTAM-1 0 hits, ac-evaluator twins byte-identical (CT-EV-MODEL-1;
twins untouched), all 6 real dogfood51 ground-truth reports still 0 blocks /
0 nonzero-rc in on-mode (no false trip re-introduced).
Open residuals (honestly-bounded NON-BLOCKERS, unchanged): the on-mode BLOCK path
has never fired LIVE (ships metric-only + fail-OPEN); run-to-run variance is
mitigated not eliminated (the triggered-on= compute is still orchestrator-LLM-
mediated -- full determinism would need on-mode promotion + a pre-spawn trigger);
generation-side depth variance (uniform in KIND, not always exhaustive DEPTH); one
inherited pre-main domain-list label nit (out of branch scope).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…force by default) Promote SW_ACCEPT_SET_CONFORMANCE_MODE default from metric-only to `on`, per the repo promotion discipline (ship metric-only, promote after a dogfood confirms no false trips) -- now satisfied with strong evidence: - dogfood51 (6 reports) + dogfood52 (8 reports) = 14 real conformant eval-round reports, 0 false-trips under the committed hook in on-mode (re-verified here under the NEW unset default = 0 blocks over all 14). - dogfood53 live de-risk: a fault-injected non-conformant report (triggered=y ran=n) Written under SW_ACCEPT_SET_CONFORMANCE_MODE=on produced a real live decision:block -- the AASC reason surfaced cleanly, the model handled it gracefully (no thrash/loop, Claude Code stayed alive) and was NOT pressured into fabricating a conformant line (it declined to flip ran=y/astral=y without actually running). decision:block is the same mechanism the other PostToolUse guards (pre-write-safety etc.) already use on subagent writes. Change: the hook now ENFORCES by default -- a genuinely non-conformant persisted `## Accept-set sweep` (P1 stand-down / P2 shallow-astral / P4 ungated authoritative divergence) emits a PostToolUse decision:block surfacing the reason to the evaluator to re-run the sweep. P3 thin-corpus stays ADVISORY (never blocks). The default is `on`, so the env var is set ONLY to downgrade; metric-only / off remain available, and a typo collapses to the safe observe (metric-only) mode (never a surprise enforce, never a silent disable). This is the INITIAL default of a NEW v8.5.0 feature (the AASC hook did not exist in v8.4.2), shipped on-by-default with a kill switch -- additive, not a change to any prior RELEASED behaviour. - hooks/accept-set-verify.sh: default `:-metric-only` -> `:-on` (+ header doc). - CLAUDE.md, skills/create-ticket/references/autopilot-policy-reference.md, skills/impl/references/ac-evaluator-orchestration.md: document on-by-default + the downgrade path. - tests/test-accept-set-verify.sh: + 3 unset-default tests (no env -> P1 blocks, thin-corpus advises, clean passes); the unknown->metric-only test stays valid. Tests: test-accept-set-verify 32/32, test-skill-contracts 852/852, ShellCheck --severity=warning clean, ac-evaluator twins untouched; 14 real dogfood51+52 reports under the NEW unset default = 0 blocks (no false-trip in enforce). Honest residual: the "real evaluator re-runs the sweep after a block" path is not yet exercised in a live autopilot run (dogfood51/52 produced conformant sweeps -> nothing to block; dogfood53 was a fault-injection where re-running was N/A). It is bounded by the autopilot round-cap + the metric-only/off kill switch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…fication + P0 safety hardening See CHANGELOG.md [8.5.0]. plugin.json bumped 8.4.2 -> 8.5.0. Pre-flight: skill-contracts 852/852, path-consistency 144/144, accept-set-verify 32/32, full per-suite sweep green, ShellCheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v8.5.0 — Advertised-Accept-Set Conformance + agnosticism codification + P0 safety hardening
Releases the AASC arc (the product-/language-/domain-agnostic meet-or-beat verification upgrade), the agnosticism codification, and the P0 safety/state hardening batch. 14 commits on top of v8.4.2; main is an ancestor (clean merge).
Highlights (full detail in
CHANGELOG.md[8.5.0]):strict/canonical/lossless/limit(or a shared-input sibling), theac-evaluatorEXECUTES a generative grammar-complement sweep (Grammar CardA/U/W/K,MR-FINITE/MR-ALPHABET/MR-CANONICAL/MR-KEYFAITH) in scratch vs an independent hand-coded oracle — catching parse-accepted overflow, input-alphabet leaks (incl. astral Unicode digits), non-canonical accepts, and structural-key / prototype-pollution injection by construction. Per-AC deterministic trigger + persisted## Accept-set sweepobservability +constraints.accept_set_conformancekill switch + committeddesign-oracles/proofs.hooks/accept-set-verify.sh) — a recognition-independentPostToolUse(Write|Edit)gate over the persisted sweep line; enforces by default (SW_ACCEPT_SET_CONFORMANCE_MODE=on;metric-only/offto downgrade), fail-OPEN.CLAUDE.md(A)/(B) substrate line + the meet-or-beat rule +CT-DECONTAM-1product-instance recidivism guard over the 14-file normative set.pre-bash-contract-guard+ jq-missing / state-field / bash-state guards + element-scoped state parsers + loop-guard / post-ship integrity self-heal (allSW_*_MODEkill-switched).Verification:
test-skill-contracts852/852,test-accept-set-verify32/32,test-path-consistency144/144, full per-suite sweep green, ShellCheck--severity=warningclean.plugin.json8.5.0== newest CHANGELOG[8.5.0](CT-MODE-13/14). Validated across dogfood43-53 (a leak-inviting strict subject produced correct strict artifacts on every historically-failing trap; a livedecision:blockde-risk confirmed the enforce path). Additive + kill-switched —constraints.accept_set_conformance: off(orSW_ACCEPT_SET_CONFORMANCE_MODE=off) reverts byte-for-byte to the pre-v8.5.0 read-only behaviour.🤖 Generated with Claude Code
https://claude.ai/code/session_019irBw7wTgiT1YG3rUy5T5a