test: freeze Agent Spec field-change contract matrix

CATALOG.md

@@ -1,7 +1,7 @@
 <!-- Generated by bin/generate-catalog.sh from cell KDL and evidence/*.tsv. Do not edit. -->
 # Current eval catalog and evidence
 
-This is the sole current corpus surface: **58 included cells** (18 model-free,
+This is the sole current corpus surface: **59 included cells** (19 model-free,
 40 model-backed) and **4 retired exclusions**. Inventory fields are derived from the active
 KDL and executable judge scripts. Latest accepted PASS evidence remains distinct from the append-only run
 history: a failed last run is visible without being advertised as accepted evidence, and cells with no
@@ -17,6 +17,7 @@ harness-native loader plus canonical hook file.
 | Cell | Harness | Model(s) / effort | Model seats | Cost | Timeout | Held-out judges | Latest accepted PASS | Last recorded run |
 |---|---|---|---:|---|---|---:|---|---|
 | `adopt-only-migration` | model-free | — | 0 | none | `90s` | 6 | — | **NO STRUCTURED RUN** |
+| `agent-spec-field-change-matrix` | model-free | — | 0 | none | `180s` | 5 | — | **NO STRUCTURED RUN** |
 | `agent-spec-resource-bindings` | model-free | — | 0 | none | `60s` | 4 | — | **NO STRUCTURED RUN** |
 | `assignment-contract-cold-assignment` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** |
 | `assignment-contract-cold-focus` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** |
@@ -95,6 +96,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
 ## Harness hook exclusions
 
 - `adopt-only-migration` / `cell` — deterministic native adoption/replacement lifecycle probe; no harness seat
+- `agent-spec-field-change-matrix` / `cell` — deterministic native Agent Spec field-change contract probe; no harness seat
 - `agent-spec-resource-bindings` / `cell` — deterministic native Agent Spec Resource-envelope probe; no harness seat
 - `context-resource-continuity` / `cell` — deterministic native context/resource restart probe; no harness seat
 - `docs` / `judge:cold-reader` — one-shot offline Claude print grader; no bus identity, DING, or hook surface
@@ -114,7 +116,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
 - `targeted-reconcile-isolation` / `cell` — deterministic selected reconcile isolation probe; no bus-connected agent
 - `two-networks-coexist` / `cell` — deterministic catalog-isolation probe; no harness seat
 
-These **19** rows are generated from structured exclusions. A cell-level row is required for
+These **20** rows are generated from structured exclusions. A cell-level row is required for
 every derived model-free cell. The one-shot docs cold reader is a model-backed grader, but not a bus-connected
 agent: it has no st2 identity, DING sidecar, or lifecycle hooks.
 

cells/agent-spec-field-change-matrix/README.md

@@ -0,0 +1,53 @@
+# agent-spec-field-change-matrix
+
+Contract-first, model-free acceptance matrix for the field-change rules in merged
+[`compoundingtech/st2#102`](https://github.com/compoundingtech/st2/pull/102) at exact merge
+[`e54a04a3ee8af6fe0b0bce4cd961f8188ac90525`](https://github.com/compoundingtech/st2/commit/e54a04a3ee8af6fe0b0bce4cd961f8188ac90525).
+
+This eval is expected to fail until st2 implements the documented field-change rules. PR #102 changes
+documentation, not runtime behavior, and names eight implementation gaps. The classification judges accept a
+known failure only when its public CLI result matches the exact map below. They never count a known failure as
+a conformance pass. The run step continues to fail until all contract cases pass and matching product evidence
+updates the manifest.
+
+**Capabilities required:** `st2,pty,jq,awk,grep,sed,coreutils`. No model and no provider. Every catalog, workspace, exec state,
+message, PTY root, process, and receipt is created below the eval-owned temporary root.
+
+## Closed-set map
+
+| Outcome | Cases | Field / gap |
+| --- | --- | --- |
+| Existing pass | `source-noop-heals`, `compact-lowering-equal`, `provider-fields-core-noop`, `invalid-agent-isolated` | F01, F14-F16 |
+| Expected red | `invalid-type-refuses` | F04 / G01 |
+| Expected red | `workspace-survivor-event`, `resource-survivor-event`, `render-survivor-event` | F06-F08 / G02, G04, G06 |
+| Expected red | `identity-remove-add`, `task-set-remove`, `task-id-change`, `spawn-drift-visible`, `invalid-policy-refuses`, `retire-removed-child` | F02, F09-F13 / G03, G04 |
+| Expected red | `host-projection` | F03 / G05 |
+| Expected red | `role-metadata-adopts`, `plan-dry-run-order` | F05, shared R05 / G07 |
+| Expected red | `moved-intent-refusal` | unsupported moved intent / G08 |
+
+The probes use only public `st2 validate`, `ls`, `agents`, `tasks`, `up --once`, message, and task observation
+surfaces plus isolated exec, process-generation, and PTY state. The F14 arms compare compact argv lowering with
+an explicit PTY and send a real inbox message through both native DING sidecars; the F06-F08 oracle requires one durable stable-ID event with class,
+affected paths or Resource descriptors, post-commit visibility, coalescing, replay idempotence, unchanged
+silence, and no file or secret bytes. The probes do not import st2 source modules or treat unit tests as
+conformance evidence.
+
+## Gates
+
+- `judges/self-test.sh` plants missing-field, unknown-gap, expected-red-as-pass, stale-map-with-equal-counts,
+  result-drift, and residue mutations against the static oracle.
+- The focused eval runs all public-behavior probes, prints one receipt row per closed-set case, cleans every
+  exact eval-owned process/session, and exits nonzero while any contract case is red.
+- `bin/check-corpus.sh` remains the model-free repository preflight and does not execute this expected-failure
+  cell.
+
+Run the static oracle and the expected-failure product matrix with:
+
+```sh
+bash ./cells/agent-spec-field-change-matrix/judges/self-test.sh
+st2 eval ./cells/agent-spec-field-change-matrix/ --keep
+```
+
+At the `e54a04a3` product pin the honest focused verdict is expected to contain five green classification
+judges and one red run-step gate. Any different pass/red map is an unexpected product change and fails the
+classification judge rather than silently changing the contract.

cells/agent-spec-field-change-matrix/agent-spec-field-change-matrix.kdl

@@ -0,0 +1,28 @@
+// Contract-first, model-free acceptance for the field-change rules proposed by merged st2 PR #102.
+// The run step is expected to fail while the exact G01-G08 product gaps remain.
+eval {
+  copy "./fixture"
+  max-timeout "180s"
+
+  run "matrix" {
+    command "bash ./matrix.sh"
+  }
+
+  judges {
+    judge "COVERAGE - every F01-F16 field rule plus dry-run and moved intent has one closed-set case" {
+      exec "bash ./judges/contract.sh coverage $RUNS_DIR/matrix.out ./fixture/cases.tsv"
+    }
+    judge "GAPS - every expected red maps only to a named G01-G08 implementation gap" {
+      exec "bash ./judges/contract.sh gaps $RUNS_DIR/matrix.out ./fixture/cases.tsv"
+    }
+    judge "CLASSIFICATION - exact product observations match the frozen pass/red map" {
+      exec "bash ./judges/contract.sh classification $RUNS_DIR/matrix.out ./fixture/cases.tsv"
+    }
+    judge "HONESTY - no expected product red is counted as a conformance pass" {
+      exec "bash ./judges/contract.sh honesty $RUNS_DIR/matrix.out ./fixture/cases.tsv"
+    }
+    judge "CLEANUP - isolated exec and PTY state has zero live residue" {
+      exec "bash ./judges/contract.sh cleanup $RUNS_DIR/matrix.out ./fixture/cases.tsv"
+    }
+  }
+}

cells/agent-spec-field-change-matrix/fixture/cases.tsv

@@ -0,0 +1,19 @@
+case_id	fields	expected	gaps	contract
+source-noop-heals	F01	PASS	-	real KDL-to-JSON path/form move preserves normalized effects while independently dead work heals
+identity-remove-add	F02	RED	G03	exact old identity is removed before the new identity boots
+host-projection	F03	RED	G05	independent old/new projections converge after both overlap-first and absence-first orders without a shared receipt
+invalid-type-refuses	F04	RED	G01	unsupported type refuses validation and reconciliation before launch write or teardown
+role-metadata-adopts	F05	RED	G07	role-only change preserves the generation and becomes observable declaration metadata without notification
+workspace-survivor-event	F06	RED	G02,G04,G06	workspace survivor keeps its generation and gets one durable coalesced privacy-safe post-commit event
+resource-survivor-event	F07	RED	G02,G06	Resource field changes commit before one durable coalesced privacy-safe event and unchanged replay stays quiet
+render-survivor-event	F08	RED	G02,G06	two changed render targets commit before one durable coalesced privacy-safe event and unchanged bytes stay quiet
+task-set-remove	F09	RED	G03	removed child is cleaned by exact ownership without changing its sibling
+task-id-change	F10	RED	G03	exact old task id is removed before the new id boots
+spawn-drift-visible	F11	RED	G03,G04	healthy spawn mismatch stays live and reports desired and observed fingerprints
+invalid-policy-refuses	F12	RED	G03	invalid restart or lifecycle policy refuses instead of defaulting
+retire-removed-child	F13	RED	G03	retirement stops every owned child including one removed in the same revision
+compact-lowering-equal	F14	PASS	-	compact argv env cwd lifecycle equal the explicit PTY while native DING lowering delivers a real message
+provider-fields-core-noop	F15	PASS	-	live provider-only mutation preserves generation and launch fingerprint with zero wake event or action
+invalid-agent-isolated	F16	PASS	-	distinct invalid last-known-good generation is held while an unrelated valid runtime id boots
+plan-dry-run-order	R05	RED	G07	true dry-run reports the fixed action order and writes nothing
+moved-intent-refusal	MOVED	RED	G08	unsupported moved intent rejects cycles conflicts and host changes