Eval catalog plan versus direct brief A/B scaffold

AGENT-SPEC.md

@@ -245,6 +245,85 @@ task named `agent`. Network startup materializes active declarations for the sel
 reconciliation; retired and other-host declarations are skipped. A gating failure suppresses only that agent.
 Prefer a catalog-owned `.st2/` overlay and locally excluded tool loaders.
 
+## Experimental read-only plans
+
+This candidate contract is not part of the released `0fed14b` corpus
+conformance claim above. The paired model-free experiment is isolated to
+[`catalog-plan-vs-direct-brief`](cells/catalog-plan-vs-direct-brief/) and runs
+against st2 draft PR
+[#115](https://github.com/compoundingtech/st2/pull/115) at exact current-main source
+`8a76b6e71355140e5b89cd9313fcfd88c82b5cad`.
+
+The experiment shape descends from
+[source sketch revision `5c1d142`](https://gist.github.com/myobie/d5ecfac24cd3965e095a5031cd2e00cb/5c1d1427c0556d95d13890e5c5086cd85b25d994);
+the executable discovery and schema authority is the exact PR115 source above.
+An agent discovers a plan only through a childless Resource link:
+
+```kdl
+agent "app-web" {
+  resource "ship-remote-approvals" _tag="plan" uri="file:plans/ship-remote-approvals/plan.kdl"
+}
+```
+
+The positional Resource name is an agent-local role. The referenced
+`plan.kdl`, not the Resource envelope, owns all plan truth:
+
+```kdl
+plan "ship-remote-approvals" {
+  owner "app-web"
+  version "0000" content="file:versions/0000.md"
+  version "0001" content="file:versions/0001.md" {
+    parent "0000"
+    why "Browser proof exposed an approval race."
+  }
+}
+```
+
+Instead of external Markdown content, a version may keep its complete intent
+inline in the same referenced `plan.kdl`:
+
+```kdl
+plan "review-follow-up" {
+  owner "app-web"
+  version "0000" {
+    intent "Review every unresolved comment and report the exact final head."
+  }
+}
+```
+
+Plan identity is the explicit KDL value, never its directory. External plans
+require one `owner`. Each version requires exactly one `content="file:..."` or
+one child `intent`; both or neither fail. A version may repeat `parent` for
+multiple declared parents; parents must exist, be unique, and form an acyclic
+graph. A version with parents requires one non-empty `why`. Versions are sorted,
+and the frontier is derived as every version with no child, retaining concurrent
+siblings. Resource URIs resolve relative to the agent KDL, while content URIs
+resolve relative to the referenced `plan.kdl`; both must be relative `file:`
+references to regular files inside the selected catalog. The Resource adds the
+agent to `referencedBy` but owns no plan fields. Legacy `plan-ref`, childful plan
+Resources, and agent-owned inline plan truth are unsupported. The experiment
+stores no content digest or history. It cannot prove that an earlier declaration
+or content file stayed unchanged.
+
+The only supported CLI is read-only:
+
+```sh
+st2 plan validate [PATH] [--json]
+st2 plan list [PATH] [--json]
+st2 plan show <identity> [PATH] [--json]
+st2 plan inspect <identity> [PATH] [--json]
+```
+
+Global `--catalog` selects the same input. The experiment has no current
+pointer, execution, controller, schedule, step graph, retry, claim, receipt,
+event, reconciliation mutation, agent restart, CAS, merge, or deployment
+behavior. Direct KDL and direct human-to-agent planning remain supported.
+The current model-free A/B finds no plan advantage for cold resume, exact
+intent recovery, or acceptance evidence: both durable paths recover equally,
+and neither surface reports worker acceptance. Static validation and resolved
+provenance are plan-only authoring evidence; live correctness, traffic, cost,
+and duration remain unresolved.
+
 ## Validation, health, and lifecycle
 
 Canonical validation is:

CATALOG.md

@@ -27,6 +27,7 @@ harness-native loader plus canonical hook file.
 | `assignment-contract-hot-assignment` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 4 | — | **NO STRUCTURED RUN** |
 | `assignment-contract-hot-focus` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 4 | — | **NO STRUCTURED RUN** |
 | `assignment-contract-hot-resources` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 4 | — | **NO STRUCTURED RUN** |
+| `catalog-plan-vs-direct-brief` | model-free | — | 0 | none | `60s` | 22 | — | **NO STRUCTURED RUN** |
 | `context-resource-continuity` | model-free | — | 0 | none | `90s` | 4 | — | **NO STRUCTURED RUN** |
 | `crash-ding` | mixed | `claude-sonnet-5+gpt-5.6-sol` / medium | 4 | high | `180s` | 5 | — | **NO STRUCTURED RUN** |
 | `ding-mode` | Claude | `claude-sonnet-5` / medium | 2 | medium | `1200s` | 4 | — | **NO STRUCTURED RUN** |
@@ -95,6 +96,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
 
 ## Harness hook exclusions
 
+- `catalog-plan-vs-direct-brief` / `cell` — deterministic plan-versus-brief fixture and recovery contract; no harness seat
 - `adopt-only-migration` / `cell` — deterministic native adoption/replacement lifecycle probe; no harness seat
 - `agent-spec-resource-bindings` / `cell` — deterministic native Agent Spec Resource-envelope probe; no harness seat
 - `context-resource-continuity` / `cell` — deterministic native context/resource restart probe; no harness seat

cells/catalog-plan-vs-direct-brief/README.md

@@ -0,0 +1,65 @@
+# catalog-plan-vs-direct-brief
+
+Model-free contract for the first experiment proposed by
+[`st2 plans: today → dream`](https://gist.github.com/myobie/d5ecfac24cd3965e095a5031cd2e00cb/5c1d1427c0556d95d13890e5c5086cd85b25d994):
+compare one realistic repository task delivered as a versioned local catalog
+plan (A) with the same task delivered as an ordinary durable direct brief (B).
+
+The frozen task, repository, tools, budgets, intent revisions, judges, and done
+condition are identical. The two intent documents are byte-identical at each
+revision. Both arms receive durable local input and use the same evaluator-owned
+receipt fields, so direct planning remains a valid control rather than an
+intentionally fragile baseline.
+
+The deterministic fixture now runs both durable product paths:
+
+- the seeded repository needs a real implementation and the arm-neutral
+  reference solution passes public and held-out correctness tests;
+- arm A recovers an initial and then revised catalog snapshot through exact
+  `st2 plan show`/`inspect` calls;
+- arm B receives the byte-identical revisions through real isolated
+  `st2 message send` deliveries with `inReplyTo` lineage, then recovers them
+  through `message ls`/`read`;
+- both arms recover the same intent after cold state loss with their remote
+  source offline, so cold resume, intent recovery, and steering are ties;
+- neither read-only plan inspection nor direct-message delivery reports worker
+  acceptance, so acceptance evidence is evaluator-owned in both arms and is
+  also a tie;
+- the plan adds native static validation and resolved provenance, but the
+  current model-free result is `no-measured-advantage`; and
+- no model or provider is launched.
+
+Correctness, coordination traffic, token use, cost, and wall duration remain
+unresolved live-run endpoints. The reference solution establishes a valid task
+and neutral judge; it is not substituted for two agent executions.
+
+Arm A uses the exact Resource-linked external `plan.kdl` contract from
+[st2 draft PR #115](https://github.com/compoundingtech/st2/pull/115) at source
+`8a76b6e71355140e5b89cd9313fcfd88c82b5cad`, while retaining the experiment
+shape from source gist revision
+`5c1d1427c0556d95d13890e5c5086cd85b25d994`. A childless Agent Spec Resource
+with `_tag="plan"` supplies only the agent-local role and source-relative file
+link. The referenced `plan.kdl` owns the plan identity, owner, versions, and
+intent truth. The comparison path keeps its external Markdown content, and one
+focused second target proves the supported inline-intent form without changing
+the A/B scenario.
+
+The accepted Linux artifact SHA256 is
+`214e08874720bc546d4adf7d7977e614237baf7989cc09f6932cd991f497a753`;
+hosted Nix run
+[30835684680](https://github.com/compoundingtech/st2/actions/runs/30835684680),
+job `91760161352`, passed. The fixture exercises only `plan validate`, `list`,
+`show`, and `inspect`, and proves they do not alter the catalog. Legacy
+`plan-ref` and agent-owned inline plan truth are not used.
+
+This plan model stores no content digest or history, so it cannot prove that an
+earlier declaration, parent link, or content file stayed unchanged.
+
+Arm B uses the product's ordinary durable message store rather than a synthetic
+thread sidecar. Neither arm receives a pre-authored acceptance receipt. The
+experiment uses plain copied folders, requires no server, and sets
+`casRequired` to false. The product experiment adds no current pointer,
+execution, scheduling, steps, retries, progress claims, events,
+reconciliation, or CAS. A provider-backed A/B remains a separate authorization
+with an exact model, effort, budget, run order, spend ceiling, cleanup, and
+rollback.

cells/catalog-plan-vs-direct-brief/catalog-plan-vs-direct-brief.kdl

@@ -0,0 +1,81 @@
+// Neutral, model-free fixture contract for a catalog-plan versus direct-brief A/B.
+eval {
+  copy "./fixture"
+  max-timeout "60s"
+
+  run "fixture-contract" {
+    command "bash ./check-fixtures.sh"
+  }
+  run "plan-contract" {
+    command "bash ./check-plan-contract.sh"
+  }
+
+  judges {
+    judge "TASK — frozen repository starts red and its arm-neutral reference implementation passes" {
+      exec "grep -Fqx TASK-CONTRACT-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "PARITY — task intent, tools, budgets, judges, and done condition match across arms" {
+      exec "grep -Fqx ARM-PARITY-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "RESTART — catalog plan and real direct-message inbox both recover after cold state loss" {
+      exec "grep -Fqx COLD-RESTART-TIE-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "PARTITION — both arms recover locally while their remote sources are unavailable" {
+      exec "grep -Fqx LOCAL-PARTITION-TIE-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "INTENT — both arms recover byte-identical initial and revised intent" {
+      exec "grep -Fqx INTENT-RECOVERY-TIE-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "STEERING — a complete plan version and reply-linked direct brief recover equally" {
+      exec "grep -Fqx HUMAN-STEERING-TIE-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "ACCEPTANCE — neither read-only plan inspection nor message delivery proves worker acceptance" {
+      exec "grep -Fqx ACCEPTANCE-EVIDENCE-TIE-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "STATIC VALUE — plan validation and provenance are plan-only authoring evidence" {
+      exec "grep -Fqx PLAN-STATIC-EVIDENCE-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "DIRECT REALISM — the control uses durable st2 messages and reply lineage" {
+      exec "grep -Fqx DIRECT-MESSAGE-EVIDENCE-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "MEASUREMENT — correctness gates are separate from neutral traffic and cost observations" {
+      exec "grep -Fqx MEASUREMENT-CONTRACT-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "PROVENANCE — every authoritative input has a verified content hash" {
+      exec "grep -Fqx PROVENANCE-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "PAIRING — exact experimental source and artifact are frozen while live execution stays blocked" {
+      exec "grep -Fqx PRODUCT-PAIRING-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "STORAGE — plain local folders suffice and CAS is not required" {
+      exec "grep -Fqx PLAIN-FOLDER-NO-CAS-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "VERDICT — the model-free endpoints show no plan advantage and live endpoints stay unresolved" {
+      exec "grep -Fqx MODEL-FREE-OUTCOME-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+    judge "RUNNER — the exact experimental st2 binary matches the accepted source and SHA256" {
+      exec "grep -Fqx PLAN-RUNNER-PIN-GREEN-43af $RUNS_DIR/plan-contract.out"
+    }
+    judge "PLAN VALIDATE — Resource-linked external plans normalize without errors" {
+      exec "grep -Fqx PLAN-VALIDATE-GREEN-43af $RUNS_DIR/plan-contract.out"
+    }
+    judge "PLAN TARGETS — external Markdown content and inline intent are both exact" {
+      exec "grep -Fqx PLAN-TARGET-FORMS-GREEN-43af $RUNS_DIR/plan-contract.out"
+    }
+    judge "PLAN LIST — explicit identity, owner, and derived frontier are exact" {
+      exec "grep -Fqx PLAN-LIST-GREEN-43af $RUNS_DIR/plan-contract.out"
+    }
+    judge "PLAN SHOW — normalized intent exposes versions but not provenance paths" {
+      exec "grep -Fqx PLAN-SHOW-GREEN-43af $RUNS_DIR/plan-contract.out"
+    }
+    judge "PLAN INSPECT — provenance and resolved content are exact and read-only" {
+      exec "grep -Fqx PLAN-INSPECT-READONLY-GREEN-43af $RUNS_DIR/plan-contract.out"
+    }
+    judge "PLAN BOUNDARY — a mutable current pointer is rejected with a classified error" {
+      exec "grep -Fqx PLAN-BOUNDARY-GREEN-43af $RUNS_DIR/plan-contract.out"
+    }
+    judge "HERMETIC — the scaffold launches no model, provider, network client, or mutating product runtime" {
+      exec "grep -Fqx HERMETIC-SCAFFOLD-GREEN-43af $RUNS_DIR/fixture-contract.out"
+    }
+  }
+}

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog-initial/agents/eval/receipt-worker/agent.kdl

@@ -0,0 +1,6 @@
+agent "receipt-worker" {
+  host "eval"
+  command "true"
+  resource "receipt-report" _tag="plan" uri="file:plans/receipt-report/plan.kdl"
+  resource "inline-intent" _tag="plan" uri="file:plans/inline-intent/plan.kdl"
+}

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog-initial/agents/eval/receipt-worker/plans/inline-intent/plan.kdl

@@ -0,0 +1,6 @@
+plan "inline-intent" {
+  owner "receipt-worker"
+  version "0000" {
+    intent "Keep the complete inline intent in plan.kdl."
+  }
+}

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog-initial/agents/eval/receipt-worker/plans/receipt-report/plan.kdl

@@ -0,0 +1,4 @@
+plan "receipt-report" {
+  owner "receipt-worker"
+  version "0000" content="file:versions/0000.md"
+}

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog-initial/agents/eval/receipt-worker/plans/receipt-report/versions/0000.md

@@ -0,0 +1,37 @@
+# Receipt report: latest accepted pass and last run
+
+Revision: `0000`
+
+## Goal
+
+Implement the report core in `src/report.mjs`. The report must retain the most
+recent accepted `PASS` for each cell independently from that cell's last run,
+which may be a failure.
+
+## Scope and owner
+
+You own the frozen `receipt-report` repository for this task. Change only
+`src/report.mjs`.
+
+## Required behavior
+
+- `parseJsonLines(text)` ignores blank lines and returns records in input order.
+- Malformed JSON, non-object JSON, or a record missing non-empty `run_id`,
+  `cell`, or `result` fields throws a `TypeError` that identifies the input line.
+- `summarize(records)` returns one row per cell, sorted by cell name.
+- Each row is `{ cell, accepted_pass, last_run }`.
+- `accepted_pass` is the complete most recent `PASS` record for the cell, or
+  `null` when the cell has no pass.
+- `last_run` is the complete last record for the cell.
+- Neither exported function mutates caller-owned records.
+
+## Invariants and allowed actions
+
+Use only the repository's existing Node, Bash, and Git tools. Do not access the
+network or add dependencies. Do not weaken or replace tests. No action outside
+the frozen repository is authorized.
+
+## Evidence and done condition
+
+Run `npm test`. Completion additionally requires the evaluator's held-out tests
+to pass and a diff containing only `src/report.mjs`.

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog/agents/eval/receipt-worker/agent.kdl

@@ -0,0 +1,6 @@
+agent "receipt-worker" {
+  host "eval"
+  command "true"
+  resource "receipt-report" _tag="plan" uri="file:plans/receipt-report/plan.kdl"
+  resource "inline-intent" _tag="plan" uri="file:plans/inline-intent/plan.kdl"
+}

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog/agents/eval/receipt-worker/plans/inline-intent/plan.kdl

@@ -0,0 +1,6 @@
+plan "inline-intent" {
+  owner "receipt-worker"
+  version "0000" {
+    intent "Keep the complete inline intent in plan.kdl."
+  }
+}

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog/agents/eval/receipt-worker/plans/receipt-report/plan.kdl

@@ -0,0 +1,9 @@
+plan "receipt-report" {
+  owner "receipt-worker"
+
+  version "0000" content="file:versions/0000.md"
+  version "0001" content="file:versions/0001.md" {
+    parent "0000"
+    why "Human steering tightens input validation without changing scope."
+  }
+}

cells/catalog-plan-vs-direct-brief/fixture/arm-a/catalog/agents/eval/receipt-worker/plans/receipt-report/versions/0000.md

@@ -0,0 +1,37 @@
+# Receipt report: latest accepted pass and last run
+
+Revision: `0000`
+
+## Goal
+
+Implement the report core in `src/report.mjs`. The report must retain the most
+recent accepted `PASS` for each cell independently from that cell's last run,
+which may be a failure.
+
+## Scope and owner
+
+You own the frozen `receipt-report` repository for this task. Change only
+`src/report.mjs`.
+
+## Required behavior
+
+- `parseJsonLines(text)` ignores blank lines and returns records in input order.
+- Malformed JSON, non-object JSON, or a record missing non-empty `run_id`,
+  `cell`, or `result` fields throws a `TypeError` that identifies the input line.
+- `summarize(records)` returns one row per cell, sorted by cell name.
+- Each row is `{ cell, accepted_pass, last_run }`.
+- `accepted_pass` is the complete most recent `PASS` record for the cell, or
+  `null` when the cell has no pass.
+- `last_run` is the complete last record for the cell.
+- Neither exported function mutates caller-owned records.
+
+## Invariants and allowed actions
+
+Use only the repository's existing Node, Bash, and Git tools. Do not access the
+network or add dependencies. Do not weaken or replace tests. No action outside
+the frozen repository is authorized.
+
+## Evidence and done condition
+
+Run `npm test`. Completion additionally requires the evaluator's held-out tests
+to pass and a diff containing only `src/report.mjs`.