From 80f76a70e9ece42b38e938d8482ed28c4f024b95 Mon Sep 17 00:00:00 2001 From: screenleon Date: Fri, 31 Jul 2026 17:20:23 +0900 Subject: [PATCH 1/2] harden PR gate verification and lifecycle --- BACKLOG.md | 12 +- CHANGELOG.md | 42 +- DECISIONS.md | 117 +++ MILESTONES.md | 4 +- core/schema/gate-synthesis-result.schema.json | 357 ++++++++ .../script-variable-consumers.tsv | 3 + docs/review-model.md | 50 +- runtime/bin/gate-supervisor.sh | 26 +- runtime/bin/pr-gate.sh | 760 ++++++++++++++++-- runtime/hooks/guard-pm-bash.sh | 54 ++ runtime/lib/gate-result-verify.sh | 337 +++++++- runtime/lib/pmctl-gate.sh | 3 +- runtime/lib/pmctl-operation.sh | 37 +- skills/pr-gate-review/SKILL.md | 21 +- tests/bin/run-tests.sh | 2 +- tests/lib/test-pr-gate-fixture.sh | 125 +++ tests/shell/test-core-schemas.sh | 136 ++++ tests/shell/test-gate-lifecycle.sh | 35 + tests/shell/test-pmctl-gate.sh | 130 ++- tests/shell/test-pmctl-guard.sh | 71 ++ tests/shell/test-pmctl-operation.sh | 94 +++ tests/shell/test-pr-gate.sh | 524 +++++++++++- tests/shell/test-run-tests.sh | 17 + .../generate-gate-result-verifier-fallback.sh | 2 + 24 files changed, 2827 insertions(+), 132 deletions(-) create mode 100644 core/schema/gate-synthesis-result.schema.json diff --git a/BACKLOG.md b/BACKLOG.md index 973c589b..29d287b2 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -38,7 +38,7 @@ CC-001/CC-002 were consumed by PR #24 fix bundle inline, with no standalone entr | CC-516 | ⏸ deferred | evidence-gated thin delivery wrapper 評估；只組合既有 primitives，不建立 workflow engine/FSM | ux/process | 2026-07-23 | — | P3 | spike | | CC-517 | 🔵 active | maintainer `/ship`：primary review、structured remediation closure 與 conditional targeted confirmation | process/gate | 2026-07-23 | — | P1 | design | | CC-518 | ✅ closed 2026-07-29 | gate scope manifest v1：immutable subject、changed paths、paired tests、signals 與 bounded expansion | ops/gate | 2026-07-23 | pr:#455 | P1 | design | -| CC-519 | 🔵 active | selected-reviewer coverage／finding contract：declared coverage、stable IDs 與 actionable fix boundary | ops/gate | 2026-07-23 | — | P1 | design | +| CC-519 | ✅ closed 2026-07-30 | selected-reviewer coverage／finding contract：declared coverage、stable IDs 與 actionable fix boundary | ops/gate | 2026-07-23 | pr:#456 | P1 | design | | CC-520 | 🔵 active | synthesis parity 與 remediation seed：findings union、root-cause grouping、coverage matrix 與 no-silent-drop | ops/gate | 2026-07-23 | — | P1 | design | | CC-521 | 🔵 active | test-gap matrix、protocol recovery 與 live recall evaluation 分層 | ops/test | 2026-07-23 | — | P2 | design | | CC-522 | 🔵 active | 任意 `--test-cmd` 的 opaque／structured capability negotiation、執行失敗分類與外部 evidence recovery | ops/test | 2026-07-27 | feedback:2026-07-27 | P1 | design | @@ -47,7 +47,7 @@ CC-001/CC-002 were consumed by PR #24 fix bundle inline, with no standalone entr | CC-525 | 🔵 active | copy-mode verifier fallback 的 generated provenance 必須指向實際 generator，並由 parity ratchet 防止再次漂移 | ops/test | 2026-07-28 | feedback:2026-07-28 | P3 | hygiene | | CC-526 | 🔵 active | reviewer override file 的 symlink trust-boundary hardening 與相容性契約 | security/gate | 2026-07-28 | feedback:2026-07-28 | P2 | hygiene | | CC-527 | 🔵 active | targeted gate CLI 拆分 pass、reviewer coverage 與 tier，避免 full targeted 語意重疊 | ux/gate | 2026-07-28 | feedback:2026-07-28 | P2 | design | -| CC-528 | 🔵 active | publish policy compatibility：generic 為可接受 baseline、maintainer 為 preferred，並允許 ship 驗證既有 current-tree Gate artifact | release/gate | 2026-07-30 | feedback:2026-07-30 | P1 | design | +| CC-528 | ✅ closed 2026-07-30 | publish policy compatibility：generic 為可接受 baseline、maintainer 為 preferred，並允許 ship 驗證既有 current-tree Gate artifact | release/gate | 2026-07-30 | pr:#457 | P1 | design | | CC-529 | 🔵 active | publish assurance observability：在 ship 成功輸出、PR body 與 finish marker 保留 embedded policy 與 baseline/preferred satisfaction | release/gate | 2026-07-30 | feedback:2026-07-30 | P2 | hygiene | | CC-530 | 🔵 active | source-safe runtime library contract + centralized domain identifier policy | arch/reuse | 2026-07-30 | feedback:2026-07-30 | P1 | hygiene | | CC-531 | 🔵 active | Adapter manifest contract closure：dispatch entrypoint 成為唯一 runtime authority | arch/schema | 2026-07-30 | feedback:2026-07-30 | P1 | design | @@ -2076,7 +2076,9 @@ expansion 與 truncation 有 deterministic fixtures。 --- -## CC-519 — selected-reviewer coverage／finding contract 🔵 active +## CC-519 — selected-reviewer coverage／finding contract ✅ 2026-07-30 + +**See**: pr:#456 **Problem**: reviewer prose 沒有一致的 coverage declaration；找到 blocker 後可能 early stop，finding 也常缺少受影響 behavior、fix boundary 與 verification expectation。 @@ -2549,7 +2551,9 @@ inheritance 依賴 [[CC-515]]；maintainer consumer 接線由 [[CC-517]] 使用 --- -## CC-528 — publish policy compatibility：generic baseline + maintainer preferred 🔵 active +## CC-528 — publish policy compatibility：generic baseline + maintainer preferred ✅ 2026-07-30 + +**See**: pr:#457 **Problem**: `generic` 與 `maintainer` 是 Gate consumer policy，不是權限或身分； 但 shared verifier 目前以 policy 名稱完全相等判斷 applicability，並把 `publish` diff --git a/CHANGELOG.md b/CHANGELOG.md index 72f46a74..52986a56 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,38 @@ Versions follow [Semantic Versioning](https://semver.org/). ### Added +- **Deterministic synthesis parity and remediation seed (CC-520).** Completed + selected-reviewer routes now emit `pr_gate_result_v4` with exactly one + `gate_synthesis_result_v1` block. The verifier mechanically reconciles the + selected/not-reviewed dimensions, reviewer-by-surface coverage matrix, + complete stable-ID inventory and findings union, uncertainty/caution sets, + root-cause membership, and a pending `remediation_closure_v1` seed against + the authoritative reviewer JSON. Silent drops, duplicate IDs, coverage + drift, missing verification expectations, malformed uncertainty objects, or + malformed seeds stop as synthesis protocol `INCOMPLETE`. Sequential and + parallel modes share the same contract and fixed human sections for must-fix + order, advisories/cautions, coverage gaps, and recommended verification; + executor-authored frontmatter is normalized to an unbound v1 staging result, + so a model that anticipates v4 cannot race the shell-owned assurance sidecar + publication. Multiple model-authored assurance pointers fail closed, while + the shell alone binds the final bounded pointer and result version. + Legacy v1-v3 results remain readable under their historical contracts. + +- **Fail-closed gate artifact handoff hardening.** Model staging accepts an exact + `+---` patch-marker variant only long enough to canonicalize it back to a real + YAML fence; malformed or ambiguous staging still fails closed. Protocol + failures now emit `failure-result: ` so detached supervisors preserve + and surface the inspectable artifact without treating it as a verdict. + +- **Reviewer command guard false-positive hardening.** Quoted operands of + `rg`/`grep`/`egrep`/`fgrep` are treated as search data during denylist + matching, so reviewing source text containing destructive spellings no + longer blocks a reviewer; executable `sed`/`awk` programs remain subject to + denylist inspection. Command substitutions remain conservative and denied. + Actual destructive command forms remain denied. Supervisor EXIT handling also + publishes a failed terminal claim when normal dispatch exits before result + handoff, while parent reconciliation defers until the producer stops. + - **Selected-reviewer coverage and finding contract (CC-519).** Every selected reviewer now emits a scope-bound `gate_reviewer_result_v1` JSON report with an explicit eleven-surface checklist, evidence/reasons, stable finding IDs, @@ -36,8 +68,14 @@ Versions follow [Semantic Versioning](https://semver.org/). Sequential and parallel reviewers receive the same manifest digest. Budget omissions stop as `INCOMPLETE` unless explicitly accepted with `--accept-scope-truncation`; accepted truncation remains recorded with exact - omitted counts and reasons. Named v3 consumers now require verified linked - scope evidence; historical v3 envelopes with unavailable scope remain + omitted counts and reasons. Symbol expansion is language-aware, and shell + call-site hints are limited to direct source-path consumers, preventing + embedded foreign-language snippets and unrelated local functions with common + names from exhausting the search-match budget. Consumer scans read through + the full immutable snapshot so `pipefail` cannot turn an early grep match + into a nondeterministic omission. A compatible-language query that truly + exceeds the limit still fails closed. Named v3 consumers now require verified + linked scope evidence; historical v3 envelopes with unavailable scope remain readable only through non-authorizing artifact inspection. - **Immutable gate subject and shared three-axis verification (CC-515).** diff --git a/DECISIONS.md b/DECISIONS.md index ac6ea9e2..1fd87540 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -7,6 +7,123 @@ H2 標題格式：## YYYY-MM-DD: <短描述> 與 BACKLOG closure 對應的 entry，內文首行寫：Closes: BACKLOG.md#-NNN --> +## 2026-07-31: reviewer-search-commands-and-supervisor-failures-remain-observable + +**Context**: Reviewer dispatch can legitimately search source text containing +destructive command spellings. Matching the raw command string caused the PM +Bash guard to block such a search, while an early supervisor exit could leave +the operation apparently running with readiness evidence but no terminal claim. + +**Decision**: For known search tools, quoted operands without command +substitution are masked before the denylist is evaluated; `$()` and backtick +substitutions retain conservative denylist matching. Actual destructive command +syntax outside quoted search data remains covered by the existing denylist. The +gate supervisor installs an EXIT fallback that publishes a failed terminal +claim whenever ordinary shell exit occurs before the normal result handoff, and +parent reconciliation defers while the producer is still active. + +**Alternatives considered**: (a) Remove the destructive patterns—rejected +because that weakens the security boundary. (b) Allow all reviewer commands— +rejected because reviewers still execute in a protected PM context. (c) Infer a +verdict from readiness or child disappearance—rejected because protocol +failure must remain distinct from GO/NO-GO. + +**Constraints introduced**: Search-tool masking is intentionally limited to +quoted operands without executable substitution and does not become a shell +interpreter. Parent child claims cannot terminalize an operation while its +producer is pending/running/stopping. SIGKILL or host loss can still prevent an +EXIT trap; those cases remain indeterminate and require reconciliation rather +than synthetic verdict publication. + +## 2026-07-31: scope-search-budgets-follow-language-and-consumer-selection + +Relates: CC-518, CC-520 + +**Context**: A real CC-520 gate stopped before reviewer dispatch with 379 of +512 expansion entries but 142 omitted raw matches. The shell producer had +parsed an embedded jq `def flag(...)` as though it were a Python definition, +and treated the file-local `usage()` function like one repository-wide symbol. +The per-query limit therefore measured unrelated lexical collisions rather +than bounded adjacent review scope, forcing explicit truncation acceptance for +routine shell changes. + +**Decision**: Scope symbol extraction follows the changed source's language, +and symbol searches only retain compatible-language paths. Shell functions are +treated as file-local; a shell call-site hint is emitted only when another +shell file directly references the defining script and also uses the symbol. +Direct shared-helper consumer evidence remains independent. The existing +per-symbol budget is applied after this deterministic language/consumer +selection, so a real compatible-language overflow remains a truthful +`search-match-budget` omission and fails closed. + +**Alternatives considered**: (a) Increase the 64-match limit—rejected because +common names would hit the next fixed threshold and add noisy references. +(b) Automatically accept `search-match-budget` truncation—rejected because a +widely used exported symbol can represent real omitted review scope. (c) Treat +all raw matches as complete after global deduplication—rejected because omitted +provenance can still identify unique relevant paths. + +**Constraints introduced**: Embedded languages must not leak symbols into the +host file's query set. Shell call-site expansion must not search unrelated +same-name scripts. Snapshot-content predicates in a `pipefail` producer must +consume the complete input rather than use an early-exiting pipeline whose +upstream SIGPIPE can nondeterministically suppress a match. Language-compatible +overflow, diff-hunk/source/symbol overflow, and global expansion-entry overflow +keep the existing explicit acceptance contract; this refinement does not claim +a complete call graph or change immutable-subject, reviewer, synthesis, or +publication authorization semantics. + +--- + +## 2026-07-31: synthesis-preserves-reviewer-evidence-without-claiming-defect-completeness + +Relates: CC-517, CC-519, CC-520, CC-521 + +**Context**: Reviewer JSON 已能提供 declared coverage、stable finding IDs 與 +verification expectation，但 consolidated synthesis 仍由模型自由摘要。只檢查最高 +verdict 與 cross-reviewer overlap，無法證明 lower-severity finding、caution、 +uncertainty、disagreement 或 test expectation 沒有在 dedup 時消失；也無法直接產生 +後續 remediation 可沿用的完整 seed。把 synthesis prose 當 authority 會重新引入 +已由 reviewer protocol 排除的格式與遺漏風險。 + +**Decision**: Completed selected-reviewer routes emit one +`gate_synthesis_result_v1` and publish `pr_gate_result_v4`. Raw +`gate_reviewer_result_v1` documents remain authoritative inputs. The synthesis +copies a deterministic reviewer finding inventory, findings union and +reviewer-by-surface coverage matrix; root-cause grouping and disagreement +summaries remain synthesis judgments, but every original finding ID must appear +in exactly one group and one pending `remediation_closure_v1` seed entry. +Uncertainty and caution collections are mechanically derived from reviewer +origins and coverage statuses. The shell verifies all parity before binding the +result to assurance. Executor frontmatter is untrusted staging input: the shell +accepts a supported result version, rejects duplicate assurance pointer fields, +normalizes the document to unbound v1 for intermediate verification, then alone +publishes the sidecar and upgrades the final bounded result to v4. Human output +固定呈現 must-fix、advisory/caution、coverage gap/uncertainty 與 recommended +verification sections。 + +**Alternatives considered**: (a) 只加強 synthesis prompt——否決，prompt 不能證明 +沒有 silent drop。(b) 由 shell 自動合併所有 human prose——否決，root-cause 與 +disagreement 仍需要語意判斷，且會產生另一套摘要器。(c) 直接把 seed 宣告為 +remediation closure——否決，seed 只保存待處理 finding 與驗證期待，不證明修正、 +final-tree freshness 或 targeted confirmation。 + +**Constraints introduced**: `uncertainties` 必須是單一 +`{finding_ids,coverage_cells}` object，不得多包 array。Finding/coverage parity、 +stable-ID uniqueness、group membership、caution/uncertainty derivation與 seed parity +任一失敗都使 protocol `INCOMPLETE`，不能產生 authorization。Result v1-v3 保持歷史 +可讀；v4 證明 union/parity，不宣稱 reviewer recall 或 defect completeness，也不 +實作 CC-521 recovery／test-gap matrix 或 CC-517 closure lifecycle。模型不得決定 +assurance publication 時序；即使模型預先輸出 v4，intermediate verification 也只 +消費 shell-normalized v1 staging，避免在 sidecar 發佈前錯誤 dereference。 +Protocol failure 仍不得轉化為 GO/NO-GO；若結果檔已產生，producer 只透過 +`failure-result` 傳遞可供 post-mortem 的路徑，detached wait 仍以 terminal +sentinel 與完整 verifier 結果為準。模型意外留下的精確 `+---` patch marker +只可被 staging normalization canonicalize，其他 frontmatter 變形維持 +fail-closed。 + +--- + ## 2026-07-30: pre-v1-roadmap-is-contiguous-and-preserves-milestone-history Relates: CC-032, CC-033, CC-358, CC-446, CC-447, CC-511, CC-514, CC-517, diff --git a/MILESTONES.md b/MILESTONES.md index 934cf904..ec4d7314 100644 --- a/MILESTONES.md +++ b/MILESTONES.md @@ -130,7 +130,7 @@ Gate correctness 與 release evidence 仍必須先在 v0.11.0 關閉。 | 票 | 摘要 | 狀態 | |----|------|------| | CC-518 | `gate_scope_manifest_v1`：immutable subject、changed/renamed/untracked、paired tests、signals、bounded expansion/truncation | ✅ pr:#455 | -| CC-519 | selected-reviewer coverage/finding contract；sequential logical sections 與 parallel session isolation 分開 | 🔵 | +| CC-519 | selected-reviewer coverage/finding contract；sequential logical sections 與 parallel session isolation 分開 | ✅ pr:#456 | | CC-520 | synthesis findings-union parity、root-cause grouping、coverage matrix、remediation seed、no silent drop | 🔵 | | CC-521 | actionable test-gap matrix + bounded protocol recovery；seeded live recall 僅作 quality evaluation | 🔵 | | CC-522 | arbitrary `--test-cmd` opaque／structured negotiation；test failure 與 timeout／environment INCOMPLETE 分流 | 🔵 | @@ -139,7 +139,7 @@ Gate correctness 與 release evidence 仍必須先在 v0.11.0 關閉。 | 票 | 摘要 | 狀態 | |----|------|------| -| CC-528 | publish policy compatibility：generic current-tree initial GO 為 baseline、maintainer 為 preferred；ship 可驗證明確 supplied result | 🔵 | +| CC-528 | publish policy compatibility：generic current-tree initial GO 為 baseline、maintainer 為 preferred；ship 可驗證明確 supplied result | ✅ pr:#457 | | CC-529 | publish assurance observability：ship stdout、PR body、finish marker 保留 producer policy 與 baseline/preferred satisfaction | 🔵 | | CC-517 | `/ship` primary review→local/targeted/split remediation closure→final affected/full tests；不虛稱 final-tree GO | 🔵 | | CC-511 Phase B | final-tree review或 primary-review closure authorization + current-tree full PASS → publish | 🔵 | diff --git a/core/schema/gate-synthesis-result.schema.json b/core/schema/gate-synthesis-result.schema.json new file mode 100644 index 00000000..b8013976 --- /dev/null +++ b/core/schema/gate-synthesis-result.schema.json @@ -0,0 +1,357 @@ +{ + "title": "Gate synthesis result", + "description": "Deterministic synthesis parity contract for selected-reviewer findings, declared coverage, and the initial remediation closure seed.", + "type": "object", + "required": [ + "kind", + "schema_version", + "scope_manifest_sha256", + "selected_reviewers", + "not_reviewed_dimensions", + "coverage_matrix", + "reviewer_finding_inventory", + "findings_union", + "root_cause_groups", + "disagreements", + "uncertainties", + "cautions", + "remediation_seed" + ], + "properties": { + "kind": { "const": "gate_synthesis_result_v1" }, + "schema_version": { "const": 1 }, + "scope_manifest_sha256": { + "type": "string", + "pattern": "^[a-f0-9]{64}$" + }, + "selected_reviewers": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { "$ref": "#/definitions/reviewer" } + }, + "not_reviewed_dimensions": { + "type": "array", + "uniqueItems": true, + "items": { "$ref": "#/definitions/reviewer" } + }, + "coverage_matrix": { + "type": "array", + "items": { "$ref": "#/definitions/coverageCell" } + }, + "reviewer_finding_inventory": { + "type": "array", + "items": { "$ref": "#/definitions/findingInventoryEntry" } + }, + "findings_union": { + "type": "array", + "items": { "$ref": "#/definitions/findingUnionEntry" } + }, + "root_cause_groups": { + "type": "array", + "items": { "$ref": "#/definitions/rootCauseGroup" } + }, + "disagreements": { + "type": "array", + "items": { "$ref": "#/definitions/disagreement" } + }, + "uncertainties": { "$ref": "#/definitions/uncertainties" }, + "cautions": { + "type": "array", + "uniqueItems": true, + "items": { "$ref": "#/definitions/findingId" } + }, + "remediation_seed": { "$ref": "#/definitions/remediationSeed" } + }, + "definitions": { + "reviewer": { + "enum": [ + "critic", + "qa-tester", + "architecture-reviewer", + "security-reviewer", + "risk-reviewer" + ] + }, + "surface": { + "enum": [ + "changed_files", + "paired_tests", + "sensitive_signals", + "public_interface", + "schema", + "config", + "install", + "ci", + "release", + "migration", + "bounded_expansion" + ] + }, + "findingId": { + "type": "string", + "pattern": "^(critic|qa-tester|architecture-reviewer|security-reviewer|risk-reviewer)-F[0-9]{3,}$" + }, + "relativePath": { + "type": "string", + "minLength": 1, + "pattern": "^(?!/)", + "not": { + "pattern": "(^|/)\\.\\.(/|$)" + } + }, + "evidenceRef": { + "type": "object", + "required": ["path"], + "properties": { + "path": { "$ref": "#/definitions/relativePath" }, + "line": { "type": ["integer", "null"], "minimum": 1 }, + "symbol": { "type": ["string", "null"], "minLength": 1 } + }, + "anyOf": [ + { + "required": ["line"], + "properties": { + "line": { "type": "integer", "minimum": 1 } + } + }, + { + "required": ["symbol"], + "properties": { + "symbol": { "type": "string", "minLength": 1 } + } + } + ], + "additionalProperties": false + }, + "sourceRef": { + "type": "object", + "required": ["path"], + "properties": { + "path": { "$ref": "#/definitions/relativePath" }, + "line": { "type": ["integer", "null"], "minimum": 1 }, + "symbol": { "type": ["string", "null"], "minLength": 1 } + }, + "anyOf": [ + { + "required": ["line"], + "properties": { + "line": { "type": "integer", "minimum": 1 } + } + }, + { + "required": ["symbol"], + "properties": { + "symbol": { "type": "string", "minLength": 1 } + } + } + ], + "additionalProperties": false + }, + "coverageCell": { + "type": "object", + "required": [ + "reviewer", + "surface", + "status", + "evidence_refs", + "reason" + ], + "properties": { + "reviewer": { "$ref": "#/definitions/reviewer" }, + "surface": { "$ref": "#/definitions/surface" }, + "status": { + "enum": ["examined", "not_applicable", "uncertain"] + }, + "evidence_refs": { + "type": "array", + "items": { "$ref": "#/definitions/evidenceRef" } + }, + "reason": { "type": "string", "minLength": 1 } + }, + "additionalProperties": false + }, + "findingInventoryEntry": { + "type": "object", + "required": [ + "id", + "reviewer", + "severity", + "hard_gate_class", + "origin", + "verification_expectation" + ], + "properties": { + "id": { "$ref": "#/definitions/findingId" }, + "reviewer": { "$ref": "#/definitions/reviewer" }, + "severity": { "enum": ["critical", "high", "medium", "low"] }, + "hard_gate_class": { + "enum": ["none", "soft_block", "hard_block"] + }, + "origin": { + "enum": ["diff_caused", "pre_existing", "uncertain", "caution"] + }, + "verification_expectation": { + "type": "string", + "minLength": 1 + } + }, + "additionalProperties": false + }, + "findingUnionEntry": { + "type": "object", + "required": [ + "id", + "reviewer", + "severity", + "hard_gate_class", + "origin", + "source", + "affected_behavior", + "why_it_matters", + "failure_mode", + "minimum_fix_boundary", + "verification_expectation", + "root_cause_group_id", + "disposition" + ], + "properties": { + "id": { "$ref": "#/definitions/findingId" }, + "reviewer": { "$ref": "#/definitions/reviewer" }, + "severity": { "enum": ["critical", "high", "medium", "low"] }, + "hard_gate_class": { + "enum": ["none", "soft_block", "hard_block"] + }, + "origin": { + "enum": ["diff_caused", "pre_existing", "uncertain", "caution"] + }, + "source": { "$ref": "#/definitions/sourceRef" }, + "affected_behavior": { "type": "string", "minLength": 1 }, + "why_it_matters": { "type": "string", "minLength": 1 }, + "failure_mode": { "type": "string", "minLength": 1 }, + "minimum_fix_boundary": { "type": "string", "minLength": 1 }, + "verification_expectation": { + "type": "string", + "minLength": 1 + }, + "root_cause_group_id": { + "type": "string", + "pattern": "^RCG-[0-9]{3,}$" + }, + "disposition": { "const": "pending" } + }, + "additionalProperties": false + }, + "rootCauseGroup": { + "type": "object", + "required": ["id", "summary", "finding_ids"], + "properties": { + "id": { + "type": "string", + "pattern": "^RCG-[0-9]{3,}$" + }, + "summary": { "type": "string", "minLength": 1 }, + "finding_ids": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { "$ref": "#/definitions/findingId" } + } + }, + "additionalProperties": false + }, + "disagreement": { + "type": "object", + "required": ["id", "summary", "finding_ids"], + "properties": { + "id": { + "type": "string", + "pattern": "^D-[0-9]{3,}$" + }, + "summary": { "type": "string", "minLength": 1 }, + "finding_ids": { + "type": "array", + "minItems": 2, + "uniqueItems": true, + "items": { "$ref": "#/definitions/findingId" } + } + }, + "additionalProperties": false + }, + "uncertainCoverageCell": { + "type": "object", + "required": ["reviewer", "surface", "reason"], + "properties": { + "reviewer": { "$ref": "#/definitions/reviewer" }, + "surface": { "$ref": "#/definitions/surface" }, + "reason": { "type": "string", "minLength": 1 } + }, + "additionalProperties": false + }, + "uncertainties": { + "type": "object", + "required": ["finding_ids", "coverage_cells"], + "properties": { + "finding_ids": { + "type": "array", + "uniqueItems": true, + "items": { "$ref": "#/definitions/findingId" } + }, + "coverage_cells": { + "type": "array", + "items": { "$ref": "#/definitions/uncertainCoverageCell" } + } + }, + "additionalProperties": false + }, + "remediationSeedEntry": { + "type": "object", + "required": [ + "finding_id", + "reviewer", + "root_cause_group_id", + "disposition", + "verification_expectation" + ], + "properties": { + "finding_id": { "$ref": "#/definitions/findingId" }, + "reviewer": { "$ref": "#/definitions/reviewer" }, + "root_cause_group_id": { + "type": "string", + "pattern": "^RCG-[0-9]{3,}$" + }, + "disposition": { "const": "pending" }, + "verification_expectation": { + "type": "string", + "minLength": 1 + } + }, + "additionalProperties": false + }, + "remediationSeed": { + "type": "object", + "required": [ + "kind", + "schema_version", + "state", + "scope_manifest_sha256", + "entries" + ], + "properties": { + "kind": { "const": "remediation_closure_v1" }, + "schema_version": { "const": 1 }, + "state": { "const": "seed" }, + "scope_manifest_sha256": { + "type": "string", + "pattern": "^[a-f0-9]{64}$" + }, + "entries": { + "type": "array", + "items": { "$ref": "#/definitions/remediationSeedEntry" } + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false +} diff --git a/docs/architecture/script-variable-consumers.tsv b/docs/architecture/script-variable-consumers.tsv index 5e4fe3f1..2e526a12 100644 --- a/docs/architecture/script-variable-consumers.tsv +++ b/docs/architecture/script-variable-consumers.tsv @@ -39,6 +39,7 @@ CLAUDE_HOME CLAUDE_HOME uninstall.sh production CODEX_DISPATCH_TIMEOUT CODEX_DISPATCH_TIMEOUT adapters/codex/dispatch.sh production CODEX_DISPATCH_TIMEOUT CODEX_DISPATCH_TIMEOUT runtime/lib/pmctl-dispatch.sh production CODEX_GATE_STUB_* CODEX_GATE_STUB_ tests/shell/test-pr-gate.sh test +CODEX_GATE_STUB_* CODEX_GATE_STUB_ASSURANCE_FRONTMATTER tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_BOLD_FINAL tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_CALLED_MARKER tests/lib/test-pr-gate-fixture.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_CALLED_MARKER tests/shell/test-pr-gate-profile.sh test @@ -47,6 +48,7 @@ CODEX_GATE_STUB_* CODEX_GATE_STUB_CROSS_TAMPER_REVIEWER tests/shell/test-pr-gate CODEX_GATE_STUB_* CODEX_GATE_STUB_CROSS_TAMPER_VICTIM tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_DUPLICATE_HEADING tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_FRONTMATTER_FINAL tests/shell/test-pr-gate.sh test +CODEX_GATE_STUB_* CODEX_GATE_STUB_FRONTMATTER_OPENING tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_HEADER_ONLY_VERDICT tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_INJECT_FILE tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_MODE tests/lib/test-pr-gate-fixture.sh test @@ -54,6 +56,7 @@ CODEX_GATE_STUB_* CODEX_GATE_STUB_MODE tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_MULTIPLE_VERDICTS tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_PROTOCOL_MUTATION tests/lib/test-pr-gate-fixture.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_PROTOCOL_MUTATION tests/shell/test-pr-gate.sh test +CODEX_GATE_STUB_* CODEX_GATE_STUB_RESULT_VERSION tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_SYNTHESIS_EXTRA_FINAL tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_SYNTHESIS_FINAL tests/shell/test-pr-gate.sh test CODEX_GATE_STUB_* CODEX_GATE_STUB_SYNTHESIS_INJECT_FILE tests/shell/test-pr-gate.sh test diff --git a/docs/review-model.md b/docs/review-model.md index bfbc9b41..7805a891 100644 --- a/docs/review-model.md +++ b/docs/review-model.md @@ -221,7 +221,7 @@ architecture, security, and risk coverage, so the governance tables cannot quietly lower their own future review floor through a small edit. After reviewer dispatch completes, the final producer writes -`pr_gate_result_v3` Markdown plus a sibling `gate_assurance_v3` JSON envelope. +`pr_gate_result_v4` Markdown plus a sibling `gate_assurance_v3` JSON envelope. A pre-dispatch fail-fast route has no reviewer protocol and intentionally remains `pr_gate_result_v2`. The Markdown contains human findings and a bounded relative `gate_assurance` pointer; the shell-owned envelope records @@ -250,6 +250,18 @@ bounded same-stem peers, symbol call-site hints, and direct shared-helper consumers. Every expansion entry states its reason, source, evidence kind, and limit; the manifest explicitly says this is not a complete call graph. +Symbol hints are selected by the changed file's language before any search +budget is applied. Candidate call sites must use a compatible source language; +foreign-language snippets embedded in a file cannot become symbols for that +file. Shell functions are file-local unless another shell script directly +references the defining script, so shell call-site hints are limited to those +direct consumers. This keeps generic names such as `usage` from expanding to +unrelated scripts while preserving fail-closed behavior when a semantically +eligible query really exceeds its declared budget. Consumer content checks read +the complete snapshot before deciding a match; they do not use an +early-terminating pipeline that could turn an upstream SIGPIPE into a +subject-identical but narrower manifest. + The manifest publishes its budgets, omitted counts/reasons, and a canonical content digest. Any omission makes the gate `INCOMPLETE` before reviewer dispatch unless the operator explicitly supplies `--accept-scope-truncation`; @@ -294,6 +306,25 @@ invalid layer as JSON, top-level/binding, coverage, finding, evidence-reference, or verdict so a format error is not misdiagnosed as missing coverage. +Completed reviewer routes also carry one `gate_synthesis_result_v1`. Its +reviewer-by-surface matrix and stable-ID inventory are copied from the raw +reviewer documents, and its findings union preserves every source field and +verification expectation. Root-cause groups may consolidate presentation, but +they partition immutable finding IDs rather than replacing them. +Disagreements remain explicit; uncertainties and cautions are derived from the +original coverage statuses and finding origins. The nested +`remediation_closure_v1` document is only a pending seed, not proof that any +finding was fixed or that the final tree was re-reviewed. + +The shell verifies selected/not-reviewed dimensions, coverage and finding +parity, unique IDs, exact group membership, uncertainty/caution sets, and seed +parity. Any silent drop, duplicate, malformed object, or missing verification +expectation makes synthesis `INCOMPLETE`. The fixed human sections summarize +must-fix order, advisories/cautions, coverage gaps/uncertainties, and +recommended verification without replacing the machine evidence. This proves +union completeness relative to the emitted reviewer documents; it does not +prove model recall or defect completeness. + `pmctl gate verify [--cd ] [--consumer ] [--json]` returns three independent axes: @@ -333,7 +364,8 @@ Do not mutate the reviewed tree between gate finalization and `gate wait` or correctly changes `subject_current` to `fail`; the consumer refuses the result and reports the drift reason instead of silently authorizing a different tree. Legacy result v1/v2 artifacts remain readable under their historical -contracts. Result v3 is required for selected-reviewer protocol evidence. +contracts. Result v3 proves selected-reviewer protocol evidence; current result +v4 additionally proves synthesis union and coverage parity. No gate result version is publication authorization by itself; `ship finish` requires a current, applicable gate-assurance v3 assessment plus the authoritative current-tree full-suite evidence and publication guards. @@ -344,8 +376,9 @@ publish-consumer verification; it never guesses a latest artifact. The producer publishes the sidecar before atomically replacing the self-contained staging v1 result with the bound result that references it, so interruption cannot strand a result with a missing sidecar. A completed -selected-reviewer route becomes result v3; a pre-dispatch fail-fast result with -no reviewer protocol remains result v2. The protected attestation is published +selected-reviewer route becomes result v4; historical reviewer-only result v3 +remains readable, and a pre-dispatch fail-fast result with no reviewer protocol +remains result v2. The protected attestation is published afterward; verification uses a bounded retry when it observes that in-flight canonical finalization. Legacy `pr_gate_result_v1` and unbound `gate_assurance_v1` artifacts remain @@ -355,6 +388,15 @@ consumers must not infer mode, coverage, or independence from them. Earlier readable for artifact inspection, but they cannot supply immutable-subject or consumer-applicability evidence. +Executor-authored frontmatter is not allowed to choose that publication +lifecycle. Before intermediate verdict verification, the producer requires one +supported version field, rejects multiple model-authored assurance pointers, +then rewrites the document to an unbound v1 staging form and removes at most one +model-authored pointer. This means an executor that anticipates the final v4 +shape cannot cause a false protocol failure merely because the shell-owned +sidecar does not exist yet. Only finalization inserts the bounded sibling +pointer and upgrades the version. + --- ## When line-by-line review is appropriate diff --git a/runtime/bin/gate-supervisor.sh b/runtime/bin/gate-supervisor.sh index f5b20565..15afea42 100755 --- a/runtime/bin/gate-supervisor.sh +++ b/runtime/bin/gate-supervisor.sh @@ -37,6 +37,7 @@ unset _self _dir # sentinel path. _sentinel_nonce="${PM_GATE_SUPERVISOR_NONCE:-}" unset PM_GATE_SUPERVISOR_NONCE +_terminal_written=false gate_id="" @@ -52,9 +53,25 @@ _write_sentinel() { [[ -n "$_result" ]] && _pairs+=("result_file=$_result") [[ -n "${PM_GATE_PARENT_OPERATION:-}" ]] && _pairs+=("parent_operation=$PM_GATE_PARENT_OPERATION") detached_launch_write_sentinel "$_sentinel_path" "${_pairs[@]}" + _terminal_written=true fi } +# A child dispatch can terminate the supervisor before pr-gate.sh reaches its +# normal result handoff. Always publish a failed terminal claim on ordinary +# shell exits so wait/reconcile never leave the parent operation apparently +# running with only readiness evidence. SIGKILL remains inherently +# uncatchable; its unresolved state is still reported as indeterminate. +# shellcheck disable=SC2317 # invoked indirectly by the EXIT trap. +_supervisor_exit() { + local rc=$? + if [[ "$_terminal_written" != true ]]; then + _write_sentinel "failed" "${rc:-2}" "" + fi + exit "$rc" +} +trap _supervisor_exit EXIT + # Publish startup evidence only after the supervisor has parsed its arguments, # validated the run directory, and can prove its own PID identity. The launcher # authenticates this nonce-derived path before it returns a detached gate ID; @@ -146,10 +163,11 @@ _rc=0 "$REPO_ROOT/runtime/bin/pr-gate.sh" --run-dir "$run_dir" --cd "$cd_arg" ${native[@]+"${native[@]}"} \ > "$_log" 2>&1 || _rc=$? -# pr-gate.sh prints `result: ` on both the GO and integrity-checked -# NO-GO paths; extract it for the sentinel so `pmctl gate wait` can surface it -# without re-deriving OUTPUT_FILE naming. -_result_file="$(grep -m1 '^result: ' "$_log" 2>/dev/null | sed 's/^result: //')" || _result_file="" +# pr-gate.sh prints `result: ` for publishable outcomes and +# `failure-result: ` for protocol failures; extract either for the +# sentinel so `pmctl gate wait` can surface an inspectable artifact without +# re-deriving OUTPUT_FILE naming. +_result_file="$(grep -m1 -E '^(result|failure-result): ' "$_log" 2>/dev/null | sed -E 's/^[^:]+: //')" || _result_file="" _terminal_rc="$_rc" if [[ -z "$_result_file" && ( "$_rc" -eq 0 || "$_rc" -eq 1 ) ]]; then diff --git a/runtime/bin/pr-gate.sh b/runtime/bin/pr-gate.sh index f1f25f48..378de50e 100755 --- a/runtime/bin/pr-gate.sh +++ b/runtime/bin/pr-gate.sh @@ -819,20 +819,97 @@ _gate_scope_paired_tests_collect() { return "$rc" } +_gate_scope_symbols_collect() { + local source="$1" + case "$source" in + *.sh|*.bash) + _gate_scope_path_content "$source" 2>/dev/null | + sed -nE \ + -e 's/^[[:space:]]*([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(\)[[:space:]]*(\{|$).*/\1/p' + ;; + *.go) + _gate_scope_path_content "$source" 2>/dev/null | + sed -nE \ + -e 's/^[[:space:]]*func[[:space:]]+\([^)]*\)[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\1/p' \ + -e 's/^[[:space:]]*func[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\1/p' + ;; + *.js|*.jsx|*.ts|*.tsx) + _gate_scope_path_content "$source" 2>/dev/null | + sed -nE \ + -e 's/^[[:space:]]*(export[[:space:]]+)?(async[[:space:]]+)?function[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\3/p' \ + -e 's/^[[:space:]]*(export[[:space:]]+)?class[[:space:]]+([A-Za-z_][A-Za-z0-9_]*).*/\2/p' \ + -e 's/^[[:space:]]*(export[[:space:]]+)?(const|let|var)[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*=.*/\3/p' + ;; + *.py) + _gate_scope_path_content "$source" 2>/dev/null | + sed -nE \ + -e 's/^[[:space:]]*(async[[:space:]]+)?def[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\2/p' \ + -e 's/^[[:space:]]*class[[:space:]]+([A-Za-z_][A-Za-z0-9_]*).*/\1/p' + ;; + *.java) + _gate_scope_path_content "$source" 2>/dev/null | + sed -nE \ + -e 's/^[[:space:]]*(public|protected|private)?[[:space:]]*(abstract[[:space:]]+|final[[:space:]]+)?class[[:space:]]+([A-Za-z_][A-Za-z0-9_]*).*/\3/p' + ;; + *.kt) + _gate_scope_path_content "$source" 2>/dev/null | + sed -nE \ + -e 's/^[[:space:]]*(public|protected|private|internal)?[[:space:]]*(data[[:space:]]+|sealed[[:space:]]+)?class[[:space:]]+([A-Za-z_][A-Za-z0-9_]*).*/\3/p' \ + -e 's/^[[:space:]]*(public|protected|private|internal)?[[:space:]]*(suspend[[:space:]]+)?fun[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\3/p' + ;; + *.rs) + _gate_scope_path_content "$source" 2>/dev/null | + sed -nE \ + -e 's/^[[:space:]]*(pub([[:space:]]*\([^)]*\))?[[:space:]]+)?(async[[:space:]]+)?fn[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\4/p' + ;; + esac | + awk 'length($0) >= 3 && !seen[$0]++' | + LC_ALL=C sort +} + +_gate_scope_symbol_path_compatible() { + local source="$1" candidate="$2" + case "$source" in + *.sh|*.bash) [[ "$candidate" == *.sh || "$candidate" == *.bash ]] ;; + *.go) [[ "$candidate" == *.go ]] ;; + *.js|*.jsx|*.ts|*.tsx) + [[ "$candidate" == *.js || "$candidate" == *.jsx \ + || "$candidate" == *.ts || "$candidate" == *.tsx ]] + ;; + *.py) [[ "$candidate" == *.py ]] ;; + *.java|*.kt) [[ "$candidate" == *.java || "$candidate" == *.kt ]] ;; + *.rs) [[ "$candidate" == *.rs ]] ;; + *) return 1 ;; + esac +} + _gate_scope_search_paths() { - local query="$1" search_kind="$2" result + local query="$1" search_kind="$2" source="${3-}" result local -a options=(-l -z -F) [[ "$search_kind" == symbol ]] && options+=(-w) if [[ "$POLICY_DIFF_KIND" == fixed-head ]]; then while IFS= read -r -d '' result; do - printf '%s\0' "${result#*:}" + result="${result#*:}" + if [[ "$search_kind" != symbol ]] \ + || _gate_scope_symbol_path_compatible "$source" "$result"; then + printf '%s\0' "$result" + fi done < <(git grep "${options[@]}" "$query" "$GATE_BINDING_HEAD_COMMIT" -- \ 2>/dev/null || true) else - git grep "${options[@]}" "$query" -- 2>/dev/null || true + while IFS= read -r -d '' result; do + if [[ "$search_kind" != symbol ]] \ + || _gate_scope_symbol_path_compatible "$source" "$result"; then + printf '%s\0' "$result" + fi + done < <(git grep "${options[@]}" "$query" -- 2>/dev/null || true) if [[ "$POLICY_SCOPE_INCLUDE_UNTRACKED" == true ]]; then while IFS= read -r -d '' result; do [[ -f "$WORK_DIR/$result" && ! -L "$WORK_DIR/$result" ]] || continue + if [[ "$search_kind" == symbol ]] \ + && ! _gate_scope_symbol_path_compatible "$source" "$result"; then + continue + fi if [[ "$search_kind" == symbol ]]; then grep -IqlwF -- "$query" "$WORK_DIR/$result" 2>/dev/null \ && printf '%s\0' "$result" @@ -866,12 +943,13 @@ _gate_scope_expansions_collect() { local changed_paths_json="$1" output="$2" local candidates sources source_count=0 source path base stem dir ext candidate local query match eligible_count symbol_count + local source_is_shared=false source_is_shell=false local symbol_limit="$GATE_SCOPE_MAX_SYMBOLS_PER_SOURCE" local match_limit="$GATE_SCOPE_MAX_MATCHES_PER_QUERY" local source_limit="$GATE_SCOPE_MAX_EXPANSION_SOURCES" local expansion_limit="$GATE_SCOPE_MAX_EXPANSION_ENTRIES" local omitted_sources=0 omitted_symbols=0 omitted_matches=0 omitted_entries=0 - local -a symbols=() + local -a symbols=() shell_consumers=() local -A query_seen=() candidates="$(mktemp "${TMPDIR:-/tmp}/gate-scope-expansions.XXXXXX")" || return 2 sources="$(mktemp "${TMPDIR:-/tmp}/gate-scope-sources.XXXXXX")" || { @@ -899,6 +977,15 @@ _gate_scope_expansions_collect() { base="$(basename "$source")" stem="${base%.*}" dir="$(dirname "$source")" + source_is_shared=false + source_is_shell=false + shell_consumers=() + case "$source" in + */lib/*|lib/*|*/shared/*|shared/*) source_is_shared=true ;; + esac + case "$source" in + *.sh|*.bash) source_is_shell=true ;; + esac for ext in sh bash go py js jsx ts tsx java kt rs md; do candidate="$dir/$stem.$ext" @@ -912,8 +999,7 @@ _gate_scope_expansions_collect() { fi done - if [[ "$source" == */lib/* || "$source" == lib/* \ - || "$source" == */shared/* || "$source" == shared/* ]]; then + if [[ "$source_is_shared" == true || "$source_is_shell" == true ]]; then eligible_count=0 query_seen=() while IFS= read -r -d '' match; do @@ -922,30 +1008,24 @@ _gate_scope_expansions_collect() { query_seen["$match"]=1 jq -e --arg path "$match" 'index($path) != null' \ <<<"$changed_paths_json" >/dev/null && continue - eligible_count=$((eligible_count + 1)) - if [[ "$eligible_count" -le "$match_limit" ]]; then - _gate_scope_expansion_append "$candidates" "$match" \ - shared-helper-consumer "$source" path-reference per-source "$match_limit" \ - || return 2 - else - omitted_matches=$((omitted_matches + 1)) + if [[ "$source_is_shell" == true ]] \ + && _gate_scope_symbol_path_compatible "$source" "$match"; then + shell_consumers+=("$match") + fi + if [[ "$source_is_shared" == true ]]; then + eligible_count=$((eligible_count + 1)) + if [[ "$eligible_count" -le "$match_limit" ]]; then + _gate_scope_expansion_append "$candidates" "$match" \ + shared-helper-consumer "$source" path-reference per-source "$match_limit" \ + || return 2 + else + omitted_matches=$((omitted_matches + 1)) + fi fi done < <(_gate_scope_search_paths "$source" path) fi - mapfile -t symbols < <( - _gate_scope_path_content "$source" 2>/dev/null | - sed -nE \ - -e 's/^[[:space:]]*([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(\)[[:space:]]*(\{|$).*/\1/p' \ - -e 's/^[[:space:]]*func[[:space:]]+\([^)]*\)[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\1/p' \ - -e 's/^[[:space:]]*func[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\1/p' \ - -e 's/^[[:space:]]*(export[[:space:]]+)?(async[[:space:]]+)?function[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\3/p' \ - -e 's/^[[:space:]]*(export[[:space:]]+)?class[[:space:]]+([A-Za-z_][A-Za-z0-9_]*).*/\2/p' \ - -e 's/^[[:space:]]*(export[[:space:]]+)?(const|let|var)[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*=.*/\3/p' \ - -e 's/^[[:space:]]*(async[[:space:]]+)?def[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\(.*/\2/p' | - awk 'length($0) >= 3 && !seen[$0]++' | - LC_ALL=C sort - ) + mapfile -t symbols < <(_gate_scope_symbols_collect "$source") symbol_count="${#symbols[@]}" if [[ "$symbol_count" -gt "$symbol_limit" ]]; then omitted_symbols=$((omitted_symbols + symbol_count - symbol_limit)) @@ -954,21 +1034,36 @@ _gate_scope_expansions_collect() { for query in "${symbols[@]}"; do eligible_count=0 query_seen=() - while IFS= read -r -d '' match; do - [[ "$match" != "$source" ]] || continue - [[ -z "${query_seen[$match]:-}" ]] || continue - query_seen["$match"]=1 - jq -e --arg path "$match" 'index($path) != null' \ - <<<"$changed_paths_json" >/dev/null && continue - eligible_count=$((eligible_count + 1)) - if [[ "$eligible_count" -le "$match_limit" ]]; then - _gate_scope_expansion_append "$candidates" "$match" \ - call-site-hint "$source#$query" symbol-reference per-symbol "$match_limit" \ - || return 2 - else - omitted_matches=$((omitted_matches + 1)) - fi - done < <(_gate_scope_search_paths "$query" symbol) + if [[ "$source_is_shell" == true ]]; then + for match in "${shell_consumers[@]}"; do + _gate_scope_path_content "$match" 2>/dev/null | + grep -IwF -- "$query" >/dev/null || continue + eligible_count=$((eligible_count + 1)) + if [[ "$eligible_count" -le "$match_limit" ]]; then + _gate_scope_expansion_append "$candidates" "$match" \ + call-site-hint "$source#$query" symbol-reference per-symbol "$match_limit" \ + || return 2 + else + omitted_matches=$((omitted_matches + 1)) + fi + done + else + while IFS= read -r -d '' match; do + [[ "$match" != "$source" ]] || continue + [[ -z "${query_seen[$match]:-}" ]] || continue + query_seen["$match"]=1 + jq -e --arg path "$match" 'index($path) != null' \ + <<<"$changed_paths_json" >/dev/null && continue + eligible_count=$((eligible_count + 1)) + if [[ "$eligible_count" -le "$match_limit" ]]; then + _gate_scope_expansion_append "$candidates" "$match" \ + call-site-hint "$source#$query" symbol-reference per-symbol "$match_limit" \ + || return 2 + else + omitted_matches=$((omitted_matches + 1)) + fi + done < <(_gate_scope_search_paths "$query" symbol "$source") + fi done done < "$sources" @@ -2420,6 +2515,311 @@ else rm -rf -- "$tmp_dir" } + _gate_synthesis_protocol_documents() { + local artifact="$1" + awk ' + $0 == "```synthesis_result_v1" { + if (inside) exit 2 + inside=1 + next + } + inside && $0 == "```" { + inside=0 + print "" + next + } + inside { print } + END { if (inside) exit 2 } + ' "$artifact" + } + + gate_synthesis_protocol_verify() { + local artifact=${1-} selected=${2-} skipped=${3-} scope_sha=${4-} + local tmp_dir synthesis_documents reviewer_documents synthesis_count validation + local heading heading_count + [[ $# -eq 4 && -s "$artifact" && -n "$selected" \ + && "$scope_sha" =~ ^[a-f0-9]{64}$ ]] || { + printf 'Error: synthesis protocol INCOMPLETE: invalid verifier inputs\n' >&2 + return 2 + } + tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/gate-synthesis-protocol.XXXXXX")" \ + || return 2 + synthesis_documents="$tmp_dir/synthesis.jsonl" + reviewer_documents="$tmp_dir/reviewers.jsonl" + if ! _gate_synthesis_protocol_documents "$artifact" \ + > "$synthesis_documents"; then + printf 'Error: synthesis protocol INCOMPLETE: malformed synthesis_result_v1 fence in %s\n' \ + "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + synthesis_count="$(jq -s 'length' "$synthesis_documents" 2>/dev/null)" || { + printf 'Error: synthesis protocol INCOMPLETE: invalid synthesis JSON in %s\n' \ + "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + } + if [[ "$synthesis_count" -ne 1 ]]; then + printf 'Error: synthesis protocol INCOMPLETE: expected one synthesis_result_v1 block, found %d in %s\n' \ + "$synthesis_count" "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + if ! _gate_reviewer_protocol_documents "$artifact" \ + > "$reviewer_documents" \ + || ! jq -s -e 'length > 0' "$reviewer_documents" >/dev/null 2>&1; then + printf 'Error: synthesis protocol INCOMPLETE: reviewer documents unavailable in %s\n' \ + "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + + validation="$( + jq -nr \ + --arg selected "$selected" --arg skipped "$skipped" \ + --arg scope_sha "$scope_sha" \ + --slurpfile synthesis "$synthesis_documents" \ + --slurpfile reviewers "$reviewer_documents" ' + def only_keys($allowed): + type == "object" and ((keys_unsorted - $allowed) | length) == 0; + def nonempty: type == "string" and length > 0; + def reviewer: + IN("critic","qa-tester","architecture-reviewer", + "security-reviewer","risk-reviewer"); + def surface: + IN("changed_files","paired_tests","sensitive_signals", + "public_interface","schema","config","install","ci","release", + "migration","bounded_expansion"); + def finding_id: + type == "string" and + test("^(critic|qa-tester|architecture-reviewer|security-reviewer|risk-reviewer)-F[0-9]{3,}$"); + def reference: + only_keys(["path","line","symbol"]) and + (.path | nonempty) and + ((.line | type == "number" and . >= 1 and floor == .) or + (.symbol | nonempty)); + def coverage_cell: + only_keys(["reviewer","surface","status","evidence_refs","reason"]) and + (.reviewer | reviewer) and (.surface | surface) and + (.status | IN("examined","not_applicable","uncertain")) and + (.evidence_refs | type == "array" and all(.[]; reference)) and + (.reason | nonempty); + def finding_inventory: + only_keys(["id","reviewer","severity","hard_gate_class","origin", + "verification_expectation"]) and + (.id | finding_id) and (.reviewer | reviewer) and + (.severity | IN("critical","high","medium","low")) and + (.hard_gate_class | IN("none","soft_block","hard_block")) and + (.origin | IN("diff_caused","pre_existing","uncertain","caution")) and + (.verification_expectation | nonempty); + def finding_union: + only_keys(["id","reviewer","severity","hard_gate_class","origin", + "source","affected_behavior","why_it_matters","failure_mode", + "minimum_fix_boundary","verification_expectation", + "root_cause_group_id","disposition"]) and + (.id | finding_id) and (.reviewer | reviewer) and + (.severity | IN("critical","high","medium","low")) and + (.hard_gate_class | IN("none","soft_block","hard_block")) and + (.origin | IN("diff_caused","pre_existing","uncertain","caution")) and + (.source | reference) and + (.affected_behavior | nonempty) and (.why_it_matters | nonempty) and + (.failure_mode | nonempty) and (.minimum_fix_boundary | nonempty) and + (.verification_expectation | nonempty) and + (.root_cause_group_id | + type == "string" and test("^RCG-[0-9]{3,}$")) and + .disposition == "pending"; + def root_group: + only_keys(["id","summary","finding_ids"]) and + (.id | type == "string" and test("^RCG-[0-9]{3,}$")) and + (.summary | nonempty) and + (.finding_ids | type == "array" and length > 0 and + length == (unique | length) and all(.[]; finding_id)); + def disagreement: + only_keys(["id","summary","finding_ids"]) and + (.id | type == "string" and test("^D-[0-9]{3,}$")) and + (.summary | nonempty) and + (.finding_ids | type == "array" and length >= 2 and + length == (unique | length) and all(.[]; finding_id)); + def uncertain_cell: + only_keys(["reviewer","surface","reason"]) and + (.reviewer | reviewer) and (.surface | surface) and + (.reason | nonempty); + def seed_entry: + only_keys(["finding_id","reviewer","root_cause_group_id", + "disposition","verification_expectation"]) and + (.finding_id | finding_id) and (.reviewer | reviewer) and + (.root_cause_group_id | + type == "string" and test("^RCG-[0-9]{3,}$")) and + .disposition == "pending" and + (.verification_expectation | nonempty); + ($selected | split(" ") | map(select(length > 0))) as $selected_reviewers | + ($skipped | split(" ") | map(select(length > 0))) as $skipped_reviewers | + $synthesis[0] as $s | + ([$reviewers[] as $review | + $review.coverage[] | + { + reviewer:$review.reviewer, + surface:.surface, + status:.status, + evidence_refs:.evidence_refs, + reason:.reason + } + ] | sort_by(.reviewer,.surface)) as $expected_coverage | + ([$reviewers[] | .findings[]] | sort_by(.id)) as $expected_findings | + ($expected_findings | map({ + id,reviewer,severity,hard_gate_class,origin,verification_expectation + })) as $expected_inventory | + ($expected_findings | map({ + id,reviewer,severity,hard_gate_class,origin,source,affected_behavior, + why_it_matters,failure_mode,minimum_fix_boundary, + verification_expectation + })) as $expected_union | + ($expected_findings | + map(select(.origin == "uncertain") | .id) | sort) as $expected_uncertain_ids | + ([$reviewers[] as $review | + $review.coverage[] | + select(.status == "uncertain") | + {reviewer:$review.reviewer,surface:.surface,reason:.reason} + ] | sort_by(.reviewer,.surface)) as $expected_uncertain_coverage | + ($expected_findings | + map(select(.origin == "caution") | .id) | sort) as $expected_cautions | + ($expected_findings | map(.id) | sort) as $expected_ids | + if + ($s | only_keys([ + "kind","schema_version","scope_manifest_sha256", + "selected_reviewers","not_reviewed_dimensions","coverage_matrix", + "reviewer_finding_inventory","findings_union","root_cause_groups", + "disagreements","uncertainties","cautions","remediation_seed" + ]) | not) or + $s.kind != "gate_synthesis_result_v1" or + $s.schema_version != 1 or + ($s.scope_manifest_sha256 | + type != "string" or test("^[a-f0-9]{64}$") | not) or + ($s.selected_reviewers | type) != "array" or + ($s.not_reviewed_dimensions | type) != "array" or + ($s.coverage_matrix | type) != "array" or + ($s.reviewer_finding_inventory | type) != "array" or + ($s.findings_union | type) != "array" or + ($s.root_cause_groups | type) != "array" or + ($s.disagreements | type) != "array" or + ($s.cautions | type) != "array" or + ($s.remediation_seed | type) != "object" + then "invalid top-level contract" + elif + $s.scope_manifest_sha256 != $scope_sha or + $s.selected_reviewers != $selected_reviewers or + $s.not_reviewed_dimensions != $skipped_reviewers + then "selected/not-reviewed dimensions mismatch" + elif + (all($s.coverage_matrix[]; coverage_cell) | not) + then "invalid coverage matrix" + elif + ($s.coverage_matrix | sort_by(.reviewer,.surface)) != $expected_coverage + then "coverage matrix parity mismatch" + elif + (all($s.reviewer_finding_inventory[]; finding_inventory) | not) or + (all($s.findings_union[]; finding_union) | not) + then "invalid finding inventory or union" + elif + (($s.reviewer_finding_inventory | map(.id)) | + length != (unique | length)) or + (($s.findings_union | map(.id)) | + length != (unique | length)) + then "duplicate finding ID collision" + elif + ($s.reviewer_finding_inventory | sort_by(.id)) != $expected_inventory + then "reviewer finding inventory parity mismatch" + elif + ($s.findings_union | + map(del(.root_cause_group_id,.disposition)) | sort_by(.id)) != + $expected_union + then "findings union parity mismatch" + elif + (all($s.root_cause_groups[]; root_group) | not) or + (($s.root_cause_groups | map(.id)) | + length != (unique | length)) or + ([$s.root_cause_groups[].finding_ids[]] | sort) != $expected_ids or + (([$s.root_cause_groups[].finding_ids[]] | length) != + ([$s.root_cause_groups[].finding_ids[]] | unique | length)) or + ([$s.findings_union[] as $finding | + any($s.root_cause_groups[]; + .id == $finding.root_cause_group_id and + ((.finding_ids | index($finding.id)) != null)) + ] | all | not) + then "root-cause grouping parity mismatch" + elif + (all($s.disagreements[]; disagreement) | not) or + (($s.disagreements | map(.id)) | + length != (unique | length)) or + ([$s.disagreements[].finding_ids[] as $finding_id | + ($expected_ids | index($finding_id)) != null + ] | all | not) + then "invalid disagreement references" + elif + ($s.uncertainties | type) != "object" or + ($s.uncertainties | + only_keys(["finding_ids","coverage_cells"]) | not) or + ($s.uncertainties.finding_ids | type) != "array" or + ($s.uncertainties.coverage_cells | type) != "array" or + (all($s.uncertainties.finding_ids[]; finding_id) | not) or + (all($s.uncertainties.coverage_cells[]; uncertain_cell) | not) or + ($s.uncertainties.finding_ids | sort) != $expected_uncertain_ids or + ($s.uncertainties.coverage_cells | + sort_by(.reviewer,.surface)) != $expected_uncertain_coverage + then "malformed uncertainties contract or parity mismatch" + elif + (all($s.cautions[]; finding_id) | not) or + ($s.cautions | sort) != $expected_cautions + then "caution parity mismatch" + elif + ($s.remediation_seed | + only_keys(["kind","schema_version","state", + "scope_manifest_sha256","entries"]) | not) or + $s.remediation_seed.kind != "remediation_closure_v1" or + $s.remediation_seed.schema_version != 1 or + $s.remediation_seed.state != "seed" or + $s.remediation_seed.scope_manifest_sha256 != $scope_sha or + ($s.remediation_seed.entries | type) != "array" or + (all($s.remediation_seed.entries[]; seed_entry) | not) + then "malformed remediation seed" + elif + ($s.remediation_seed.entries | sort_by(.finding_id)) != + ($s.findings_union | map({ + finding_id:.id, + reviewer, + root_cause_group_id, + disposition, + verification_expectation + }) | sort_by(.finding_id)) + then "remediation seed parity mismatch" + else "ok" + end + ' + )" || validation="invalid synthesis JSON document" + if [[ "$validation" != ok ]]; then + printf 'Error: synthesis protocol INCOMPLETE: %s in %s\n' \ + "$validation" "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + + for heading in \ + '## Must-Fix Order' \ + '## Advisory and Cautions' \ + '## Coverage Gaps and Uncertainties' \ + '## Recommended Verification' + do + heading_count="$(grep -Fxc -- "$heading" "$artifact" || true)" + if [[ "$heading_count" -ne 1 ]]; then + printf 'Error: synthesis protocol INCOMPLETE: required human section %s appears %d time(s) in %s\n' \ + "$heading" "$heading_count" "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + done + rm -rf -- "$tmp_dir" + } + _gate_result_sha256_stream() { if command -v sha256sum >/dev/null 2>&1; then sha256sum | awk '{print $1}' @@ -3352,7 +3752,7 @@ else gate_result_verify() { local result_file=${1-} expected_final=${2-} route_label=${3-gate} local version pointer result_parent assurance_file body_final - local selected_reviewers scope_sha scope_artifact scope_manifest + local selected_reviewers skipped_reviewers scope_sha scope_artifact scope_manifest local assurance_kind protocol_final [[ $# -ge 1 && $# -le 3 ]] || { printf 'gate-result-verify: gate_result_verify expects [expected_final] [route_label]\n' >&2 @@ -3367,7 +3767,7 @@ else export GATE_RESULT_ASSURANCE return 0 ;; - pr_gate_result_v2 | pr_gate_result_v3) + pr_gate_result_v2 | pr_gate_result_v3 | pr_gate_result_v4) pointer="$(_gate_result_frontmatter_value "$result_file" gate_assurance)" if [[ -z "$pointer" || "$pointer" == */* || "$pointer" == "." || "$pointer" == ".." \ || ! "$pointer" =~ ^[A-Za-z0-9._-]+\.json$ ]]; then @@ -3379,11 +3779,15 @@ else assurance_file="$result_parent/$pointer" body_final=$(grep -E '^Final: (GO|NO-GO)$' "$result_file" | awk '{print $2}') gate_assurance_verify "$result_file" "$assurance_file" "$body_final" || return $? - if [[ "$version" == pr_gate_result_v3 ]]; then + if [[ "$version" == pr_gate_result_v3 \ + || "$version" == pr_gate_result_v4 ]]; then assurance_kind="$(jq -r '.kind // empty' "$assurance_file" 2>/dev/null)" selected_reviewers="$(jq -r \ '.coordinates.coverage.selected // [] | join(" ")' \ "$assurance_file" 2>/dev/null)" + skipped_reviewers="$(jq -r \ + '.coordinates.coverage.skipped // [] | join(" ")' \ + "$assurance_file" 2>/dev/null)" scope_sha="$(jq -r \ '.evidence.scope_manifest.sha256 // empty' \ "$assurance_file" 2>/dev/null)" @@ -3394,8 +3798,8 @@ else if [[ "$assurance_kind" != gate_assurance_v3 \ || -z "$selected_reviewers" \ || ! "$scope_sha" =~ ^[a-f0-9]{64}$ ]]; then - printf 'Error: pr_gate_result_v3 requires verified selected-reviewer scope evidence: %s\n' \ - "$assurance_file" >&2 + printf 'Error: %s requires verified selected-reviewer scope evidence: %s\n' \ + "$version" "$assurance_file" >&2 return 1 fi if jq -e '.reference_index != null' "$scope_manifest" >/dev/null 2>&1; then @@ -3414,6 +3818,11 @@ else "$protocol_final" "$body_final" "$result_file" >&2 return 1 fi + if [[ "$version" == pr_gate_result_v4 ]]; then + gate_synthesis_protocol_verify \ + "$result_file" "$selected_reviewers" "$skipped_reviewers" \ + "$scope_sha" || return $? + fi fi if [[ "${GATE_ASSURANCE_BOUND:-false}" == true ]]; then GATE_RESULT_ASSURANCE=verified @@ -4450,6 +4859,14 @@ for r in $ALL_REVIEWERS; do fi done SKIPPED_DISPLAY="${SKIPPED:-none}" +SYNTHESIS_SELECTED_JSON="$( + jq -cn --arg reviewers "$REVIEWERS" \ + '$reviewers | split(" ") | map(select(length > 0))' +)" || exit 2 +SYNTHESIS_SKIPPED_JSON="$( + jq -cn --arg reviewers "$SKIPPED_WORDS" \ + '$reviewers | split(" ") | map(select(length > 0))' +)" || exit 2 COVERAGE_SELECTED_DISPLAY="$REVIEWER_DISPLAY" COVERAGE_SKIPPED_DISPLAY="$SKIPPED_DISPLAY" @@ -4590,6 +5007,7 @@ GATE_OUTPUT_EXISTED=false [[ -e "$OUTPUT_FILE" ]] && GATE_OUTPUT_EXISTED=true touch "$OUTPUT_FILE" REVIEWER_PROTOCOL_COMPLETE=false +SYNTHESIS_PROTOCOL_COMPLETE=false # Track all brief files for EXIT cleanup BRIEF_FILES=() @@ -4639,6 +5057,7 @@ relocate_gate_artifacts() { } gate_exit_cleanup() { + local _gate_exit_status=$? # Relocate first (preserves the result artifact out-of-repo for post-mortem on failure # paths), then drop transient briefs. Both are idempotent / no-ops on the success path # where relocation already ran inline. @@ -4654,10 +5073,89 @@ gate_exit_cleanup() { else relocate_gate_artifacts fi + # Preserve the post-mortem artifact path even when protocol validation fails + # before the normal `result:` handoff. Detached gate wait can then surface a + # failed, inspectable artifact instead of leaving callers with only an exit 2. + if [[ "$_gate_exit_status" -ne 0 && -n "${OUTPUT_FILE:-}" && -e "${OUTPUT_FILE:-}" ]]; then + printf 'failure-result: %s\n' "$OUTPUT_FILE" + fi cleanup_briefs + return "$_gate_exit_status" } trap gate_exit_cleanup EXIT +gate_result_staging_normalize() { + local result_file="$1" route_label="${2:-gate}" + local version version_count pointer_count result_tmp + + [[ -s "$result_file" ]] || { + printf 'Error: %s did not produce a staging gate result: %s\n' \ + "$route_label" "$result_file" >&2 + return 1 + } + version_count="$(awk ' + /^\+?---$/ { + if (fence == 0) { fence=1; next } + if (fence == 1) { fence=2; next } + } + fence == 1 && $1 == "gate_result_version:" { count++ } + END { print count+0 } + ' "$result_file")" + pointer_count="$(awk ' + /^\+?---$/ { + if (fence == 0) { fence=1; next } + if (fence == 1) { fence=2; next } + } + fence == 1 && $1 == "gate_assurance:" { count++ } + END { print count+0 } + ' "$result_file")" + if [[ "$version_count" -ne 1 ]]; then + printf 'Error: %s staging frontmatter must contain exactly one gate_result_version (found %d): %s\n' \ + "$route_label" "$version_count" "$result_file" >&2 + return 1 + fi + if [[ "$pointer_count" -gt 1 ]]; then + printf 'Error: %s staging frontmatter contains multiple model-authored gate_assurance pointers: %s\n' \ + "$route_label" "$result_file" >&2 + return 1 + fi + version="$(awk ' + /^\+?---$/ { if (fence == 0) { fence=1; next } if (fence == 1) exit } + fence == 1 && $1 == "gate_result_version:" { print $2; exit } + ' "$result_file")" + case "$version" in + pr_gate_result_v1 | pr_gate_result_v2 | pr_gate_result_v3 | pr_gate_result_v4) ;; + *) + printf 'Error: unsupported model-authored staging gate_result_version (%s): %s\n' \ + "${version:-missing}" "$result_file" >&2 + return 1 + ;; + esac + + result_tmp="$(mktemp "${result_file}.staging-tmp.XXXXXX")" || return 1 + if ! awk ' + /^\+?---$/ { + if (fence < 2) { + fence++ + print "---" + } else { + print + } + next + } + fence == 1 && $1 == "gate_result_version:" { + print "gate_result_version: pr_gate_result_v1" + next + } + fence == 1 && $1 == "gate_assurance:" { next } + { print } + ' "$result_file" > "$result_tmp"; then + rm -f -- "$result_tmp" + return 1 + fi + mv -- "$result_tmp" "$result_file" +} + gate_finalize_assurance() { local result_file="$1" assurance_file="$2" local final requested_json outcomes_json independence_status implementation_isolated @@ -4841,8 +5339,12 @@ gate_finalize_assurance() { return 1 } local result_version=pr_gate_result_v2 - [[ "$REVIEWER_PROTOCOL_COMPLETE" == true ]] \ - && result_version=pr_gate_result_v3 + if [[ "$REVIEWER_PROTOCOL_COMPLETE" == true \ + && "$SYNTHESIS_PROTOCOL_COMPLETE" == true ]]; then + result_version=pr_gate_result_v4 + elif [[ "$REVIEWER_PROTOCOL_COMPLETE" == true ]]; then + result_version=pr_gate_result_v3 + fi awk -v pointer="$ASSURANCE_POINTER" -v result_version="$result_version" ' /^---$/ { fence++ @@ -5288,6 +5790,13 @@ REVIEWER_PROTOCOL_SURFACES="$( END { print out } ' )" || exit 2 +REVIEWER_REFERENCE_LINE_BOUNDS="$({ + jq -r '.reference_index.entries[] | " \(.path): max-line=\(.line_count)"' \ + "$SCOPE_MANIFEST_PATH" + printf ' .gate-results/%s: max-line=%s\n' \ + "$(basename "$SCOPE_MANIFEST_PATH")" \ + "$(awk 'END { print NR+0 }' "$SCOPE_MANIFEST_PATH")" +} 2>/dev/null)" || exit 2 # shellcheck disable=SC2016 # Literal Markdown fence delimiters in reviewer prose. printf -v REVIEWER_PROTOCOL_INSTRUCTIONS \ '%s\n' \ @@ -5303,6 +5812,8 @@ printf -v REVIEWER_PROTOCOL_INSTRUCTIONS \ " reference .gate-results/$(basename "$SCOPE_MANIFEST_PATH")." \ ' A line reference must not exceed that index entry line_count. Arbitrary,' \ ' nonexistent, or out-of-scope repository paths make the protocol INCOMPLETE.' \ + ' - Reference line bounds (do not cite beyond these immutable snapshot limits):' \ + "$REVIEWER_REFERENCE_LINE_BOUNDS" \ ' - coverage_claim=declared-scope-checklist-not-review-completeness.' \ " - coverage contains each surface exactly once: ${REVIEWER_PROTOCOL_SURFACES}." \ ' - Every coverage entry has surface, status=examined|not_applicable|uncertain,' \ @@ -5330,6 +5841,45 @@ printf -v REVIEWER_PROTOCOL_INSTRUCTIONS \ ' missed, alignment, reversibility, and override_path must not appear at top level.' \ ' - Do not claim semantic completeness or coverage for an unselected reviewer.' +# shellcheck disable=SC2016 # Literal Markdown fence delimiters in synthesis prose. +printf -v SYNTHESIS_PROTOCOL_INSTRUCTIONS \ + '%s\n' \ + ' Synthesis protocol (mandatory; parity with reviewer JSON is machine-validated):' \ + ' - Emit exactly one JSON block opened by ```synthesis_result_v1 and closed by ```.' \ + ' - The JSON object has exactly these thirteen top-level keys: kind,' \ + ' schema_version, scope_manifest_sha256, selected_reviewers,' \ + ' not_reviewed_dimensions, coverage_matrix, reviewer_finding_inventory,' \ + ' findings_union, root_cause_groups, disagreements, uncertainties, cautions,' \ + ' remediation_seed. Do not add wrapper objects or arrays.' \ + ' - kind=gate_synthesis_result_v1, schema_version=1, and' \ + " scope_manifest_sha256=${SCOPE_MANIFEST_DIGEST}." \ + " - selected_reviewers is exactly ${SYNTHESIS_SELECTED_JSON} in that order." \ + " - not_reviewed_dimensions is exactly ${SYNTHESIS_SKIPPED_JSON} in that order." \ + ' - coverage_matrix copies every reviewer coverage cell without changing reviewer,' \ + ' surface, status, evidence_refs, or reason.' \ + ' - reviewer_finding_inventory copies every stable ID, reviewer, severity,' \ + ' hard_gate_class, origin, and verification_expectation.' \ + ' - findings_union preserves every original finding field and adds only' \ + ' root_cause_group_id=RCG-NNN plus disposition=pending. Never drop a lower' \ + ' severity, caution, uncertainty, disagreement input, or test expectation.' \ + ' - root_cause_groups partitions every finding ID exactly once. Different reviewers' \ + ' may share a group only when they describe the same root cause; different issues' \ + ' in the same file remain distinct. With no findings, emit an empty group array.' \ + ' - disagreements is an array of {id:D-NNN,summary,finding_ids}; use [] when none.' \ + ' - uncertainties is exactly one object, never an array:' \ + ' {finding_ids:[...],coverage_cells:[{reviewer,surface,reason},...]}.' \ + ' Its two arrays exactly match uncertain findings and coverage statuses.' \ + ' - cautions is the complete stable-ID list whose origin is caution.' \ + ' - remediation_seed is {kind:remediation_closure_v1,schema_version:1,state:seed,' \ + ' scope_manifest_sha256,entries}. It contains one pending entry per finding with' \ + ' finding_id, reviewer, root_cause_group_id, disposition, and' \ + ' verification_expectation. This is a seed, never a closure or final-tree GO claim.' \ + ' - The consolidated human result contains exactly one section each named:' \ + ' ## Must-Fix Order; ## Advisory and Cautions;' \ + ' ## Coverage Gaps and Uncertainties; ## Recommended Verification.' \ + ' - Raw reviewer_result_v1 blocks remain authoritative and traceable. The synthesis' \ + ' contract proves union/parity only; it does not claim defect or model-recall completeness.' + if [[ "$SKIP_PREFLIGHT_TESTS" != "true" && -n "$TEST_CMD_OVERRIDE" ]]; then # pr-gate.sh is designed to be copied standalone into any repo (copy-mode -- # see the file header), so it must not hardcode any repo-specific test @@ -5690,10 +6240,18 @@ constraints: - Before your FIRST write to ${OUTPUT_FILE} in this session, call: ${GUARD_PMCTL_CMD} guard check --role reviewer --runtime ${EXECUTOR} --event pre-write --file ${OUTPUT_FILE} If that call exits nonzero, abort and report the guard denial -- do NOT write the file. You will write to this same file multiple times in this session (once per reviewer, then once for synthesis) -- that is expected. Do not create or write any other file. + - If you need to test the PM Bash denylist with a command string containing + destructive syntax, construct that string from shell variables before + passing it as data to pmctl. Never place a literal destructive invocation + in the outer Bash command, and never execute the probe itself. - Create parent directories for ${OUTPUT_FILE} if needed (mkdir -p). - Only cite files in the verified reference index or the diff list. Read a file before citing its sections; do not invent citations. - reviewer_result_v1.verdict is the only machine verdict. Markdown reviewer headings are presentation only; do not rely on their count or wording. + - Write a self-contained staging frontmatter with exactly + gate_result_version: pr_gate_result_v1 and no gate_assurance field. The gate + shell owns the final result version and bounded assurance pointer and + publishes them only after reviewer and synthesis verification. context: Tier: ${TIER} @@ -5705,6 +6263,7 @@ context: Date: $(date '+%Y-%m-%d') ${GATE_ASSURANCE_CONTEXT_BLOCK}${SCOPE_MANIFEST_CONTEXT_BLOCK}${GATE_OVERRIDES_CONTEXT_BLOCK}${TEST_EVIDENCE_CONTEXT_BLOCK} ${REVIEWER_PROTOCOL_INSTRUCTIONS} +${SYNTHESIS_PROTOCOL_INSTRUCTIONS} ${MEMORY_CONTEXT_BLOCK} Verified reference files (exist in working tree -- check before citing): ${REPO_REF_INDEX} @@ -5735,15 +6294,18 @@ task: heading is accidentally duplicated or omitted. After all reviewers, synthesize as project-pm would: - 5. Identify cross-reviewer overlaps (same issue raised by multiple reviewers) - 6. Overall verdict = most severe individual verdict - 7. State which dimensions were NOT covered (not-reviewed list above) - 8. Final GO (no blocks) / NO-GO (any block or block-soft) with rationale and override path if applicable + 5. Build the deterministic finding inventory and coverage matrix from every + reviewer_result_v1 block. Preserve all IDs and verification expectations. + 6. Group findings by root cause without dropping or merging stable IDs, and record + disagreements, uncertainties, cautions, and not-reviewed dimensions. + 7. Emit the complete synthesis_result_v1 JSON block and remediation seed. + 8. Overall verdict = most severe individual verdict. Final GO (no blocks) / + NO-GO (any block or block-soft), with rationale and override path if applicable. 9. Now that the final verdict is known: PREPEND the YAML frontmatter block to the very top of ${OUTPUT_FILE} (before the header already written in step 4), then APPEND the - synthesis sections (Cross-Reviewer Overlaps / Coverage Notes / Gate Conclusion / - Escalation) to the bottom. Do not rewrite the reviewer sections already written in - step 4 -- only prepend the frontmatter and append the synthesis sections. + synthesis protocol block and human sections to the bottom. The frontmatter is the + self-contained v1 staging form described above; do not add gate_assurance. Do not + rewrite the reviewer sections already written in step 4. output_format: | --- @@ -5776,11 +6338,26 @@ output_format: | (repeat for each reviewer in order) + \`\`\`synthesis_result_v1 + {one JSON object satisfying the synthesis protocol above} + \`\`\` + ## Cross-Reviewer Overlaps {list issues raised by >1 reviewer; "none" if clean} - ## Coverage Notes + ## Must-Fix Order + {ordered blocking findings by stable ID; "none" if clean} + + ## Advisory and Cautions + {all non-blocking findings and cautions by stable ID; "none" if clean} + + ## Coverage Gaps and Uncertainties **Dimensions not covered**: ${SKIPPED_DISPLAY} + {all uncertain coverage cells/findings; "none" if complete} + + ## Recommended Verification + {verification expectations grouped without dropping any stable finding ID; + "none" if there are no findings} ## Gate Conclusion **Overall verdict**: {most severe} @@ -5814,6 +6391,9 @@ self_verify: acceptance: - ${OUTPUT_FILE} exists with a verdict section for each of the ${NUM_REVIEWERS} reviewers + - exactly one synthesis_result_v1 block preserves reviewer finding and coverage parity + - Must-Fix Order / Advisory and Cautions / Coverage Gaps and Uncertainties / + Recommended Verification sections are present exactly once - "Final: GO" or "Final: NO-GO" is present in Gate Conclusion (plain text, no markdown emphasis) BRIEF_EOF @@ -5871,10 +6451,20 @@ BRIEF_EOF gate_reviewer_protocol_verify \ "$OUTPUT_FILE" "$REVIEWERS" "$SCOPE_MANIFEST_DIGEST" \ "$SCOPE_MANIFEST_PATH" || exit 1 + gate_synthesis_protocol_verify \ + "$OUTPUT_FILE" "$REVIEWERS" "$SKIPPED_WORDS" \ + "$SCOPE_MANIFEST_DIGEST" || exit 1 SEQ_PROTOCOL_FINAL="$( _gate_reviewer_protocol_final_extract "$OUTPUT_FILE" )" || exit 1 REVIEWER_PROTOCOL_COMPLETE=true + SYNTHESIS_PROTOCOL_COMPLETE=true + # Executors author an unbound staging document only. Normalize a model that + # anticipated the final v4 contract back to v1 before the verdict verifier + # could dereference an assurance sidecar that the shell has not published. + # Protocol verification intentionally precedes this rewrite so malformed or + # partial sequential output retains its precise protocol diagnostic. + gate_result_staging_normalize "$OUTPUT_FILE" "sequential gate" || exit 1 gate_result_verify \ "$OUTPUT_FILE" "$SEQ_PROTOCOL_FINAL" "sequential gate" || exit 1 @@ -5942,6 +6532,11 @@ constraints: - Only write ${REVIEWER_OUTPUT}. - Before writing ${REVIEWER_OUTPUT}, call: ${GUARD_PMCTL_CMD} guard check --role reviewer --runtime ${EXECUTOR} --event pre-write --file ${REVIEWER_OUTPUT} If that call exits nonzero, abort and report the guard denial -- do NOT write the file. + - If you need to test the PM Bash denylist with a command string containing + destructive syntax, construct that string from shell variables (for + example, separate command and flag variables) before passing it as data to + pmctl. Never place a literal destructive invocation in the outer Bash + command, and never execute the probe itself. - Create parent directories if needed (mkdir -p). - Only cite files in the verified reference index or the diff list. Read a file before citing its sections; do not invent citations. - reviewer_result_v1.verdict is the only machine verdict. A Markdown heading @@ -6172,12 +6767,18 @@ files: constraints: - Do NOT modify any source file. - Only write ${OUTPUT_FILE}. + - Before your FIRST write to ${OUTPUT_FILE} in this session, call: ${GUARD_PMCTL_CMD} guard check --role reviewer --runtime ${EXECUTOR} --event pre-write --file ${OUTPUT_FILE} + If that call exits nonzero, abort and report the guard denial -- do NOT write the file. - Create parent directories if needed (mkdir -p). - The Gate Conclusion MUST contain exactly: Final: ${SHELL_FINAL} This is pre-computed from the reviewer verdicts and must not be overridden. - Only cite files in the verified reference index or reviewer findings; do not invent citations. - Do not emit or copy any reviewer_result_v1 fenced block. The gate shell validates and appends the original reviewer protocol blocks after synthesis. + - Write a self-contained staging frontmatter with exactly + gate_result_version: pr_gate_result_v1 and no gate_assurance field. The gate + shell owns the final result version and bounded assurance pointer and + publishes them only after reviewer and synthesis verification. context: Tier: ${TIER} @@ -6188,6 +6789,7 @@ context: Scope: ${SCOPE:-none} Date: $(date '+%Y-%m-%d') ${GATE_ASSURANCE_CONTEXT_BLOCK}${SCOPE_MANIFEST_CONTEXT_BLOCK}${GATE_OVERRIDES_CONTEXT_BLOCK}${TEST_EVIDENCE_CONTEXT_BLOCK} +${SYNTHESIS_PROTOCOL_INSTRUCTIONS} Verified reference files (exist in working tree -- check before citing): ${REPO_REF_INDEX} Reviewer findings (embedded -- do NOT attempt to read any external reviewer output file): @@ -6209,14 +6811,18 @@ SBRIEF_P1 task: 1. Use the reviewer findings embedded in the context above. - 2. Identify cross-reviewer overlaps: issues raised by more than one reviewer. - 3. Determine the overall verdict: most severe individual verdict across all reviewers + 2. Build the deterministic finding inventory and coverage matrix from every + reviewer_result_v1 block. Preserve all IDs and verification expectations. + 3. Group findings by root cause without dropping or merging stable IDs, and record + disagreements, uncertainties, cautions, and not-reviewed dimensions. + 4. Emit the complete synthesis_result_v1 JSON block and remediation seed. + 5. Determine the overall verdict: most severe individual verdict across all reviewers (approve < advise < block-soft < block). - 4. State Final: GO or NO-GO. + 6. State Final: GO or NO-GO. - GO: no reviewer returned block or block-soft. - NO-GO: any reviewer returned block or block-soft. List required fixes and any applicable override path. - 5. Write the complete consolidated result to ${OUTPUT_FILE}. + 7. Write the complete consolidated result to ${OUTPUT_FILE}. output_format: | --- @@ -6242,6 +6848,10 @@ output_format: | **Reviewers**: ${REVIEWER_DISPLAY} **Not reviewed**: ${SKIPPED_DISPLAY} + \`\`\`synthesis_result_v1 + {one JSON object satisfying the synthesis protocol above} + \`\`\` + ## {reviewer-name} -- {verdict} {Summarize findings from that reviewer, one bullet per finding with stable ID, severity, and file:line. Do not copy the reviewer_result_v1 fenced block.} @@ -6253,8 +6863,19 @@ output_format: | ## Cross-Reviewer Overlaps {list issues raised by more than one reviewer; "none" if clean} - ## Coverage Notes + ## Must-Fix Order + {ordered blocking findings by stable ID; "none" if clean} + + ## Advisory and Cautions + {all non-blocking findings and cautions by stable ID; "none" if clean} + + ## Coverage Gaps and Uncertainties **Dimensions not covered**: ${SKIPPED_DISPLAY} + {all uncertain coverage cells/findings; "none" if complete} + + ## Recommended Verification + {verification expectations grouped without dropping any stable finding ID; + "none" if there are no findings} ## Gate Conclusion **Overall verdict**: {most severe across all reviewers} @@ -6294,7 +6915,9 @@ self_verify: acceptance: - ${OUTPUT_FILE} exists with a section for each of the ${NUM_REVIEWERS} reviewers - - Cross-Reviewer Overlaps section is present + - exactly one synthesis_result_v1 block preserves reviewer finding and coverage parity + - Must-Fix Order / Advisory and Cautions / Coverage Gaps and Uncertainties / + Recommended Verification sections are present exactly once - "Final: GO" or "Final: NO-GO" is present in Gate Conclusion (plain text, no markdown emphasis) SBRIEF_P2 @@ -6322,6 +6945,10 @@ SBRIEF_P2 exit 1 fi + # Keep the parallel synthesis route on the same producer-owned staging + # lifecycle as sequential mode. + gate_result_staging_normalize "$OUTPUT_FILE" "PM synthesis" || exit 1 + # Validate synthesis output via the shared contract, pinned to the # shell-computed verdict: a synthesis that contradicts SHELL_FINAL (in either # the body Final: line or the frontmatter final: field) indicates a @@ -6360,7 +6987,11 @@ SBRIEF_P2 gate_reviewer_protocol_verify \ "$OUTPUT_FILE" "$REVIEWERS" "$SCOPE_MANIFEST_DIGEST" \ "$SCOPE_MANIFEST_PATH" || exit 1 + gate_synthesis_protocol_verify \ + "$OUTPUT_FILE" "$REVIEWERS" "$SKIPPED_WORDS" \ + "$SCOPE_MANIFEST_DIGEST" || exit 1 REVIEWER_PROTOCOL_COMPLETE=true + SYNTHESIS_PROTOCOL_COMPLETE=true fi fi @@ -6503,8 +7134,9 @@ fi # Replace the executor-authored staging frontmatter with a bound pointer and # write the machine-owned assurance sidecar only after every deterministic # rewrite and explicitly enabled post-gate hook is complete. Completed reviewer -# routes publish result v3; pre-dispatch fail-fast routes without reviewer -# protocol remain v2. The shared verifier then checks result/pointer/envelope +# routes publish result v4; historical reviewer-only routes remain readable as +# v3, while pre-dispatch fail-fast routes without reviewer protocol remain v2. +# The shared verifier then checks result/pointer/envelope # parity before publication or relocation. gate_finalize_assurance "$OUTPUT_FILE" "$ASSURANCE_FILE" || exit 2 diff --git a/runtime/hooks/guard-pm-bash.sh b/runtime/hooks/guard-pm-bash.sh index 421e893e..29681b12 100755 --- a/runtime/hooks/guard-pm-bash.sh +++ b/runtime/hooks/guard-pm-bash.sh @@ -160,6 +160,60 @@ fi # guard exists to stop evaluating). _normalize_for_denylist() { local s="$1" + local first_word + _has_shell_control_operator() { + awk ' + BEGIN { sq=0; dq=0; esc=0; found=0 } + { + for (i=1; i<=length($0); i++) { + c=substr($0, i, 1) + if (esc) { esc=0; continue } + if (c == "\\") { esc=1; continue } + if (sq) { if (c == "\047") sq=0; continue } + if (dq) { if (c == "\042") dq=0; continue } + if (c == "\047") { sq=1; continue } + if (c == "\042") { dq=1; continue } + if (c ~ /[;&|<>]/) found=1 + } + } + END { exit(found ? 0 : 1) } + ' <<<"$1" + } + first_word="$(awk '{print $1}' <<<"$s")" + case "$first_word" in + rg|grep|egrep|fgrep) + # Search expressions are data, not commands. Keep the denylist + # conservative for normal shell commands, but mask quoted search + # operands so a reviewer cannot be blocked merely for searching source + # text that contains `rm -rf` or another denylisted spelling. If the + # command contains command substitution, retain the original input: the + # substitution is executable shell syntax even when nested in a search + # operand, so the security guard must remain conservative. + if [[ "$s" != *\$\(* && "$s" != *\`* ]] \ + && ! _has_shell_control_operator "$s"; then + s="$(awk ' + BEGIN { sq=0; dq=0; esc=0 } + { + out="" + for (i=1; i<=length($0); i++) { + c=substr($0, i, 1) + if (sq) { + if (c == "\047") sq=0 + out=out " " + } else if (dq) { + if (esc) { esc=0; out=out " " } + else if (c == "\\") { esc=1; out=out " " } + else { if (c == "\042") dq=0; out=out " " } + } else if (c == "\047") { sq=1; out=out " " + } else if (c == "\042") { dq=1; out=out " " + } else out=out c + } + print out + } + ' <<<"$s")" + fi + ;; + esac s="${s//\$\{IFS\}/ }" s="${s//\$IFS/ }" # ANSI-C quoted whitespace: $'\x20' $'\x09' $'\x0a' $'\t' $'\n' $' ' diff --git a/runtime/lib/gate-result-verify.sh b/runtime/lib/gate-result-verify.sh index 9da9d8d8..25698527 100644 --- a/runtime/lib/gate-result-verify.sh +++ b/runtime/lib/gate-result-verify.sh @@ -10,6 +10,10 @@ # gate_assurance_v2 adds result bindings and protected dispatch attestation. # gate_assurance_v3 adds an immutable subject plus digest-bound evidence links. +# pr_gate_result_v4 additionally requires one synthesis parity document whose +# inventory and remediation seed are mechanically reconciled with the original +# reviewer protocol documents. +# # gate_result_verdict_verify [expected_final] [route_label] gate_result_verdict_verify() { local result_file=${1-} expected_final=${2-} route_label=${3-gate} @@ -395,6 +399,320 @@ gate_reviewer_protocol_verify() { rm -rf -- "$tmp_dir" } +_gate_synthesis_protocol_documents() { + local artifact="$1" + awk ' + $0 == "```synthesis_result_v1" { + if (inside) exit 2 + inside=1 + next + } + inside && $0 == "```" { + inside=0 + print "" + next + } + inside { print } + END { if (inside) exit 2 } + ' "$artifact" +} + +# gate_synthesis_protocol_verify +# +# +# Validates the synthesis-owned JSON shape and then derives the authoritative +# finding inventory, coverage matrix, uncertainties, cautions, and remediation +# entries from the original reviewer_result_v1 documents. Root-cause grouping +# and disagreement prose remain synthesis judgments, but every referenced ID +# must belong to that immutable inventory and every finding must be grouped +# exactly once. +gate_synthesis_protocol_verify() { + local artifact=${1-} selected=${2-} skipped=${3-} scope_sha=${4-} + local tmp_dir synthesis_documents reviewer_documents synthesis_count validation + local heading heading_count + [[ $# -eq 4 && -s "$artifact" && -n "$selected" \ + && "$scope_sha" =~ ^[a-f0-9]{64}$ ]] || { + printf 'Error: synthesis protocol INCOMPLETE: invalid verifier inputs\n' >&2 + return 2 + } + tmp_dir="$(mktemp -d "${TMPDIR:-/tmp}/gate-synthesis-protocol.XXXXXX")" \ + || return 2 + synthesis_documents="$tmp_dir/synthesis.jsonl" + reviewer_documents="$tmp_dir/reviewers.jsonl" + if ! _gate_synthesis_protocol_documents "$artifact" \ + > "$synthesis_documents"; then + printf 'Error: synthesis protocol INCOMPLETE: malformed synthesis_result_v1 fence in %s\n' \ + "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + synthesis_count="$(jq -s 'length' "$synthesis_documents" 2>/dev/null)" || { + printf 'Error: synthesis protocol INCOMPLETE: invalid synthesis JSON in %s\n' \ + "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + } + if [[ "$synthesis_count" -ne 1 ]]; then + printf 'Error: synthesis protocol INCOMPLETE: expected one synthesis_result_v1 block, found %d in %s\n' \ + "$synthesis_count" "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + if ! _gate_reviewer_protocol_documents "$artifact" \ + > "$reviewer_documents" \ + || ! jq -s -e 'length > 0' "$reviewer_documents" >/dev/null 2>&1; then + printf 'Error: synthesis protocol INCOMPLETE: reviewer documents unavailable in %s\n' \ + "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + + validation="$( + jq -nr \ + --arg selected "$selected" --arg skipped "$skipped" \ + --arg scope_sha "$scope_sha" \ + --slurpfile synthesis "$synthesis_documents" \ + --slurpfile reviewers "$reviewer_documents" ' + def only_keys($allowed): + type == "object" and ((keys_unsorted - $allowed) | length) == 0; + def nonempty: type == "string" and length > 0; + def reviewer: + IN("critic","qa-tester","architecture-reviewer", + "security-reviewer","risk-reviewer"); + def surface: + IN("changed_files","paired_tests","sensitive_signals", + "public_interface","schema","config","install","ci","release", + "migration","bounded_expansion"); + def finding_id: + type == "string" and + test("^(critic|qa-tester|architecture-reviewer|security-reviewer|risk-reviewer)-F[0-9]{3,}$"); + def reference: + only_keys(["path","line","symbol"]) and + (.path | nonempty) and + ((.line | type == "number" and . >= 1 and floor == .) or + (.symbol | nonempty)); + def coverage_cell: + only_keys(["reviewer","surface","status","evidence_refs","reason"]) and + (.reviewer | reviewer) and (.surface | surface) and + (.status | IN("examined","not_applicable","uncertain")) and + (.evidence_refs | type == "array" and all(.[]; reference)) and + (.reason | nonempty); + def finding_inventory: + only_keys(["id","reviewer","severity","hard_gate_class","origin", + "verification_expectation"]) and + (.id | finding_id) and (.reviewer | reviewer) and + (.severity | IN("critical","high","medium","low")) and + (.hard_gate_class | IN("none","soft_block","hard_block")) and + (.origin | IN("diff_caused","pre_existing","uncertain","caution")) and + (.verification_expectation | nonempty); + def finding_union: + only_keys(["id","reviewer","severity","hard_gate_class","origin", + "source","affected_behavior","why_it_matters","failure_mode", + "minimum_fix_boundary","verification_expectation", + "root_cause_group_id","disposition"]) and + (.id | finding_id) and (.reviewer | reviewer) and + (.severity | IN("critical","high","medium","low")) and + (.hard_gate_class | IN("none","soft_block","hard_block")) and + (.origin | IN("diff_caused","pre_existing","uncertain","caution")) and + (.source | reference) and + (.affected_behavior | nonempty) and (.why_it_matters | nonempty) and + (.failure_mode | nonempty) and (.minimum_fix_boundary | nonempty) and + (.verification_expectation | nonempty) and + (.root_cause_group_id | + type == "string" and test("^RCG-[0-9]{3,}$")) and + .disposition == "pending"; + def root_group: + only_keys(["id","summary","finding_ids"]) and + (.id | type == "string" and test("^RCG-[0-9]{3,}$")) and + (.summary | nonempty) and + (.finding_ids | type == "array" and length > 0 and + length == (unique | length) and all(.[]; finding_id)); + def disagreement: + only_keys(["id","summary","finding_ids"]) and + (.id | type == "string" and test("^D-[0-9]{3,}$")) and + (.summary | nonempty) and + (.finding_ids | type == "array" and length >= 2 and + length == (unique | length) and all(.[]; finding_id)); + def uncertain_cell: + only_keys(["reviewer","surface","reason"]) and + (.reviewer | reviewer) and (.surface | surface) and + (.reason | nonempty); + def seed_entry: + only_keys(["finding_id","reviewer","root_cause_group_id", + "disposition","verification_expectation"]) and + (.finding_id | finding_id) and (.reviewer | reviewer) and + (.root_cause_group_id | + type == "string" and test("^RCG-[0-9]{3,}$")) and + .disposition == "pending" and + (.verification_expectation | nonempty); + ($selected | split(" ") | map(select(length > 0))) as $selected_reviewers | + ($skipped | split(" ") | map(select(length > 0))) as $skipped_reviewers | + $synthesis[0] as $s | + ([$reviewers[] as $review | + $review.coverage[] | + { + reviewer:$review.reviewer, + surface:.surface, + status:.status, + evidence_refs:.evidence_refs, + reason:.reason + } + ] | sort_by(.reviewer,.surface)) as $expected_coverage | + ([$reviewers[] | .findings[]] | sort_by(.id)) as $expected_findings | + ($expected_findings | map({ + id,reviewer,severity,hard_gate_class,origin,verification_expectation + })) as $expected_inventory | + ($expected_findings | map({ + id,reviewer,severity,hard_gate_class,origin,source,affected_behavior, + why_it_matters,failure_mode,minimum_fix_boundary, + verification_expectation + })) as $expected_union | + ($expected_findings | + map(select(.origin == "uncertain") | .id) | sort) as $expected_uncertain_ids | + ([$reviewers[] as $review | + $review.coverage[] | + select(.status == "uncertain") | + {reviewer:$review.reviewer,surface:.surface,reason:.reason} + ] | sort_by(.reviewer,.surface)) as $expected_uncertain_coverage | + ($expected_findings | + map(select(.origin == "caution") | .id) | sort) as $expected_cautions | + ($expected_findings | map(.id) | sort) as $expected_ids | + if + ($s | only_keys([ + "kind","schema_version","scope_manifest_sha256", + "selected_reviewers","not_reviewed_dimensions","coverage_matrix", + "reviewer_finding_inventory","findings_union","root_cause_groups", + "disagreements","uncertainties","cautions","remediation_seed" + ]) | not) or + $s.kind != "gate_synthesis_result_v1" or + $s.schema_version != 1 or + ($s.scope_manifest_sha256 | + type != "string" or test("^[a-f0-9]{64}$") | not) or + ($s.selected_reviewers | type) != "array" or + ($s.not_reviewed_dimensions | type) != "array" or + ($s.coverage_matrix | type) != "array" or + ($s.reviewer_finding_inventory | type) != "array" or + ($s.findings_union | type) != "array" or + ($s.root_cause_groups | type) != "array" or + ($s.disagreements | type) != "array" or + ($s.cautions | type) != "array" or + ($s.remediation_seed | type) != "object" + then "invalid top-level contract" + elif + $s.scope_manifest_sha256 != $scope_sha or + $s.selected_reviewers != $selected_reviewers or + $s.not_reviewed_dimensions != $skipped_reviewers + then "selected/not-reviewed dimensions mismatch" + elif + (all($s.coverage_matrix[]; coverage_cell) | not) + then "invalid coverage matrix" + elif + ($s.coverage_matrix | sort_by(.reviewer,.surface)) != $expected_coverage + then "coverage matrix parity mismatch" + elif + (all($s.reviewer_finding_inventory[]; finding_inventory) | not) or + (all($s.findings_union[]; finding_union) | not) + then "invalid finding inventory or union" + elif + (($s.reviewer_finding_inventory | map(.id)) | + length != (unique | length)) or + (($s.findings_union | map(.id)) | + length != (unique | length)) + then "duplicate finding ID collision" + elif + ($s.reviewer_finding_inventory | sort_by(.id)) != $expected_inventory + then "reviewer finding inventory parity mismatch" + elif + ($s.findings_union | + map(del(.root_cause_group_id,.disposition)) | sort_by(.id)) != + $expected_union + then "findings union parity mismatch" + elif + (all($s.root_cause_groups[]; root_group) | not) or + (($s.root_cause_groups | map(.id)) | + length != (unique | length)) or + ([$s.root_cause_groups[].finding_ids[]] | sort) != $expected_ids or + (([$s.root_cause_groups[].finding_ids[]] | length) != + ([$s.root_cause_groups[].finding_ids[]] | unique | length)) or + ([$s.findings_union[] as $finding | + any($s.root_cause_groups[]; + .id == $finding.root_cause_group_id and + ((.finding_ids | index($finding.id)) != null)) + ] | all | not) + then "root-cause grouping parity mismatch" + elif + (all($s.disagreements[]; disagreement) | not) or + (($s.disagreements | map(.id)) | + length != (unique | length)) or + ([$s.disagreements[].finding_ids[] as $finding_id | + ($expected_ids | index($finding_id)) != null + ] | all | not) + then "invalid disagreement references" + elif + ($s.uncertainties | type) != "object" or + ($s.uncertainties | + only_keys(["finding_ids","coverage_cells"]) | not) or + ($s.uncertainties.finding_ids | type) != "array" or + ($s.uncertainties.coverage_cells | type) != "array" or + (all($s.uncertainties.finding_ids[]; finding_id) | not) or + (all($s.uncertainties.coverage_cells[]; uncertain_cell) | not) or + ($s.uncertainties.finding_ids | sort) != $expected_uncertain_ids or + ($s.uncertainties.coverage_cells | + sort_by(.reviewer,.surface)) != $expected_uncertain_coverage + then "malformed uncertainties contract or parity mismatch" + elif + (all($s.cautions[]; finding_id) | not) or + ($s.cautions | sort) != $expected_cautions + then "caution parity mismatch" + elif + ($s.remediation_seed | + only_keys(["kind","schema_version","state", + "scope_manifest_sha256","entries"]) | not) or + $s.remediation_seed.kind != "remediation_closure_v1" or + $s.remediation_seed.schema_version != 1 or + $s.remediation_seed.state != "seed" or + $s.remediation_seed.scope_manifest_sha256 != $scope_sha or + ($s.remediation_seed.entries | type) != "array" or + (all($s.remediation_seed.entries[]; seed_entry) | not) + then "malformed remediation seed" + elif + ($s.remediation_seed.entries | sort_by(.finding_id)) != + ($s.findings_union | map({ + finding_id:.id, + reviewer, + root_cause_group_id, + disposition, + verification_expectation + }) | sort_by(.finding_id)) + then "remediation seed parity mismatch" + else "ok" + end + ' + )" || validation="invalid synthesis JSON document" + if [[ "$validation" != ok ]]; then + printf 'Error: synthesis protocol INCOMPLETE: %s in %s\n' \ + "$validation" "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + + for heading in \ + '## Must-Fix Order' \ + '## Advisory and Cautions' \ + '## Coverage Gaps and Uncertainties' \ + '## Recommended Verification' + do + heading_count="$(grep -Fxc -- "$heading" "$artifact" || true)" + if [[ "$heading_count" -ne 1 ]]; then + printf 'Error: synthesis protocol INCOMPLETE: required human section %s appears %d time(s) in %s\n' \ + "$heading" "$heading_count" "$artifact" >&2 + rm -rf -- "$tmp_dir" + return 1 + fi + done + rm -rf -- "$tmp_dir" +} + _gate_result_sha256_stream() { if command -v sha256sum >/dev/null 2>&1; then sha256sum | awk '{print $1}' @@ -1547,7 +1865,7 @@ gate_assurance_authorization_verify() { gate_result_verify() { local result_file=${1-} expected_final=${2-} route_label=${3-gate} local version pointer result_parent assurance_file body_final - local selected_reviewers scope_sha scope_artifact scope_manifest + local selected_reviewers skipped_reviewers scope_sha scope_artifact scope_manifest local assurance_kind protocol_final [[ $# -ge 1 && $# -le 3 ]] || { printf 'gate-result-verify: gate_result_verify expects [expected_final] [route_label]\n' >&2 @@ -1562,7 +1880,7 @@ gate_result_verify() { export GATE_RESULT_ASSURANCE return 0 ;; - pr_gate_result_v2 | pr_gate_result_v3) + pr_gate_result_v2 | pr_gate_result_v3 | pr_gate_result_v4) pointer="$(_gate_result_frontmatter_value "$result_file" gate_assurance)" if [[ -z "$pointer" || "$pointer" == */* || "$pointer" == "." || "$pointer" == ".." \ || ! "$pointer" =~ ^[A-Za-z0-9._-]+\.json$ ]]; then @@ -1574,11 +1892,15 @@ gate_result_verify() { assurance_file="$result_parent/$pointer" body_final=$(grep -E '^Final: (GO|NO-GO)$' "$result_file" | awk '{print $2}') gate_assurance_verify "$result_file" "$assurance_file" "$body_final" || return $? - if [[ "$version" == pr_gate_result_v3 ]]; then + if [[ "$version" == pr_gate_result_v3 \ + || "$version" == pr_gate_result_v4 ]]; then assurance_kind="$(jq -r '.kind // empty' "$assurance_file" 2>/dev/null)" selected_reviewers="$(jq -r \ '.coordinates.coverage.selected // [] | join(" ")' \ "$assurance_file" 2>/dev/null)" + skipped_reviewers="$(jq -r \ + '.coordinates.coverage.skipped // [] | join(" ")' \ + "$assurance_file" 2>/dev/null)" scope_sha="$(jq -r \ '.evidence.scope_manifest.sha256 // empty' \ "$assurance_file" 2>/dev/null)" @@ -1589,8 +1911,8 @@ gate_result_verify() { if [[ "$assurance_kind" != gate_assurance_v3 \ || -z "$selected_reviewers" \ || ! "$scope_sha" =~ ^[a-f0-9]{64}$ ]]; then - printf 'Error: pr_gate_result_v3 requires verified selected-reviewer scope evidence: %s\n' \ - "$assurance_file" >&2 + printf 'Error: %s requires verified selected-reviewer scope evidence: %s\n' \ + "$version" "$assurance_file" >&2 return 1 fi if jq -e '.reference_index != null' "$scope_manifest" >/dev/null 2>&1; then @@ -1609,6 +1931,11 @@ gate_result_verify() { "$protocol_final" "$body_final" "$result_file" >&2 return 1 fi + if [[ "$version" == pr_gate_result_v4 ]]; then + gate_synthesis_protocol_verify \ + "$result_file" "$selected_reviewers" "$skipped_reviewers" \ + "$scope_sha" || return $? + fi fi if [[ "${GATE_ASSURANCE_BOUND:-false}" == true ]]; then GATE_RESULT_ASSURANCE=verified diff --git a/runtime/lib/pmctl-gate.sh b/runtime/lib/pmctl-gate.sh index 541f2e63..5ab4eb93 100644 --- a/runtime/lib/pmctl-gate.sh +++ b/runtime/lib/pmctl-gate.sh @@ -91,7 +91,8 @@ _pmctl_gate_wait_for_assurance_publication() { local attempt=0 max_attempts=20 version="$(_gate_result_frontmatter_value "$result_file" gate_result_version)" - [[ "$version" == pr_gate_result_v2 || "$version" == pr_gate_result_v3 ]] \ + [[ "$version" == pr_gate_result_v2 || "$version" == pr_gate_result_v3 \ + || "$version" == pr_gate_result_v4 ]] \ || return 0 result_parent="$(cd "$(dirname "$result_file")" 2>/dev/null && pwd -P)" || return 0 # Only canonical run-layout results have an asynchronous producer diff --git a/runtime/lib/pmctl-operation.sh b/runtime/lib/pmctl-operation.sh index a2da983b..df648438 100644 --- a/runtime/lib/pmctl-operation.sh +++ b/runtime/lib/pmctl-operation.sh @@ -418,7 +418,8 @@ pmctl_operation_cancellation_requested() { _pmctl_operation_reconcile_inner() { local repo_root="$1" expected_kind="$2" work_dir="$3" operation_id="$4" - local record op_dir line run_id child_dir missing=0 failed=0 cancelled=0 children=0 next ts updated + local record op_dir line run_id child_dir producer_status producer_identity producer_identity_file + local producer_verify=0 producer_invalid=false missing=0 failed=0 cancelled=0 children=0 next ts updated record="$(_pmctl_operation_record_path "$repo_root" "$work_dir" "$operation_id")" || return 2 _pmctl_operation_validate_record "$record" "$expected_kind" "$work_dir" || return $? case "$PMCTL_OPERATION_RECORD_STATE" in @@ -437,6 +438,39 @@ _pmctl_operation_reconcile_inner() { fi ;; esac + producer_status="$(jq -r '.producer.status // "none"' <<<"$PMCTL_OPERATION_RECORD_JSON")" + if [[ "$producer_status" == pending ]]; then + printf 'operation: %s state: %s reconciliation: producer-active (%s) deferred\n' \ + "$operation_id" "$PMCTL_OPERATION_RECORD_STATE" "$producer_status" + return 1 + fi + if [[ "$producer_status" =~ ^(running|stopping)$ ]]; then + _pmctl_operation_load_detached_launch "$repo_root" || return 2 + producer_identity="$(jq -c '.producer.identity // null' <<<"$PMCTL_OPERATION_RECORD_JSON")" + if [[ "$producer_identity" == null ]]; then + printf 'operation: %s state: %s reconciliation: producer-active (%s) deferred\n' \ + "$operation_id" "$PMCTL_OPERATION_RECORD_STATE" "$producer_status" + return 1 + fi + producer_identity_file="$(mktemp /tmp/pm-operation-reconcile-XXXXXX.identity)" || return 2 + if ! _pmctl_operation_identity_file_from_json "$producer_identity" "$producer_identity_file" \ + || ! detached_launch_load_identity_file "$producer_identity_file"; then + producer_invalid=true + producer_verify=3 + else + detached_launch_verify_identity "$DL_ID_PID" "$producer_identity_file" || producer_verify=$? + fi + rm -f "$producer_identity_file" + case "$producer_verify" in + 0) + printf 'operation: %s state: %s reconciliation: producer-active (%s) deferred\n' \ + "$operation_id" "$PMCTL_OPERATION_RECORD_STATE" "$producer_status" + return 1 + ;; + 1|2) ;; # producer is gone or its identity has been replaced + *) producer_invalid=true ;; + esac + fi op_dir="$(_pmctl_operation_dir "$repo_root" "$work_dir" "$operation_id")" || return 2 if [[ -f "$op_dir/children.jsonl" ]]; then while IFS= read -r line; do @@ -448,6 +482,7 @@ _pmctl_operation_reconcile_inner() { done < "$op_dir/children.jsonl" fi [[ "$children" -gt 0 ]] || missing=1 + [[ "$producer_invalid" == true ]] && missing=1 if [[ "$missing" -gt 0 ]]; then next="indeterminate"; elif [[ "$cancelled" -gt 0 ]]; then next="cancelled"; elif [[ "$failed" -gt 0 ]]; then next="failed"; else next="completed"; fi ts="$(_pmctl_operation_ts)" updated="$(jq -c --arg state "$next" --arg ts "$ts" \ diff --git a/skills/pr-gate-review/SKILL.md b/skills/pr-gate-review/SKILL.md index ec772b21..532ae591 100644 --- a/skills/pr-gate-review/SKILL.md +++ b/skills/pr-gate-review/SKILL.md @@ -37,23 +37,32 @@ implement → pr-gate → fix NO-GO → push → PR). ## Reading the result -The result file carries `pr_gate_result_v2` frontmatter with `final: GO|NO-GO`, -per-reviewer verdicts, and a bounded pointer to its sibling -`gate_assurance_v3` JSON envelope. The `Final: GO|NO-GO` line is +Completed selected-reviewer results carry `pr_gate_result_v4` frontmatter with +`final: GO|NO-GO`, per-reviewer verdicts, one synthesis parity block, and a +bounded pointer to the sibling `gate_assurance_v3` JSON envelope. The +`Final: GO|NO-GO` line is parser-significant (plain text, exact shape), but is not freshness or authorization evidence. Run `pmctl gate verify --cd --consumer embedded --json` before consuming assurance claims. Named-consumer success requires all three axes to pass: `artifact_valid`, `subject_current`, and `policy_applicable`. -Legacy `pr_gate_result_v1` and v1/v2 assurance artifacts remain readable for -historical inspection but cannot prove immutable-subject freshness or consumer -applicability. Repo-layout authorization is authoritative only when +Legacy result v1-v3 and v1/v2 assurance artifacts remain readable under their +historical contracts; result v3 proves reviewer protocol but not synthesis +parity. They cannot prove capabilities their versions did not record. +Repo-layout authorization is authoritative only when verification also confirms the protected producer attestation, the invoking repository's canonical state partition, and every claimed canonical run record. Current v3 envelopes also record the shell-owned policy classification, matched signals, resolved coordinates, linked evidence, and any scope-bound user override. +Treat `gate_synthesis_result_v1` as a parity-preserving view of the raw +reviewer documents. Its coverage matrix, finding inventory, findings union, +uncertainties, cautions, and pending remediation seed are machine-checked. +Root-cause groups organize findings without replacing stable IDs. A synthesis +protocol `INCOMPLETE` is not reviewer NO-GO and cannot authorize publication, +even when every reviewer verdict says approve. + - **NO-GO** (a reviewer returned `block`): fix the blocking finding. Per project convention, clear **every** finding (high/med/low/advise) on a NO-GO, not just the blocks, then re-gate. diff --git a/tests/bin/run-tests.sh b/tests/bin/run-tests.sh index 1f1c77f6..afa1610a 100755 --- a/tests/bin/run-tests.sh +++ b/tests/bin/run-tests.sh @@ -217,7 +217,7 @@ map_path() { add_suite test-pmctl-task; behavioral=1 ;; core/schema/preflight-evidence.schema.json) add_suite test-pr-gate; behavioral=1 ;; - core/schema/gate-assurance.schema.json|core/schema/gate-policy-override.schema.json|core/schema/gate-reviewer-result.schema.json|core/schema/gate-scope-manifest.schema.json|core/schema/gate-verification.schema.json) + core/schema/gate-assurance.schema.json|core/schema/gate-policy-override.schema.json|core/schema/gate-reviewer-result.schema.json|core/schema/gate-scope-manifest.schema.json|core/schema/gate-synthesis-result.schema.json|core/schema/gate-verification.schema.json) add_suite test-core-schemas; add_suite test-pr-gate add_suite test-pmctl-gate; behavioral=1 ;; runtime/bin/pr-gate.sh) diff --git a/tests/lib/test-pr-gate-fixture.sh b/tests/lib/test-pr-gate-fixture.sh index 9a5074f3..3bd2c915 100644 --- a/tests/lib/test-pr-gate-fixture.sh +++ b/tests/lib/test-pr-gate-fixture.sh @@ -43,6 +43,23 @@ pr_gate_fixture_write_reviewer_protocol() { verification_expectation:"Run the focused regression for the changed behavior." }]')" fi + if [[ "$mutation" == advisory-finding ]]; then + findings_json="$(jq -nc \ + --arg id "${reviewer}-F001" --arg reviewer "$reviewer" \ + --arg evidence_path "$evidence_path" '[{ + id:$id, + reviewer:$reviewer, + severity:"low", + hard_gate_class:"none", + origin:"caution", + source:{path:$evidence_path,line:1,symbol:null}, + affected_behavior:"The fixture preserves a lower-severity behavior note.", + why_it_matters:"Synthesis must not discard advisory findings.", + failure_mode:"The advisory disappears from remediation evidence.", + minimum_fix_boundary:"Retain the original finding without increasing scope.", + verification_expectation:"Run the focused advisory verification." + }]')" + fi if [[ "$mutation" == invalid-id ]]; then findings_json="$(jq -nc --arg reviewer "$reviewer" \ --arg evidence_path "$evidence_path" '[{ @@ -128,6 +145,112 @@ pr_gate_fixture_write_reviewer_protocol() { } >> "$output_path" } +pr_gate_fixture_write_synthesis_protocol() { + local brief_file="$1" output_path="$2" + local document_source reviewer_documents synthesis_document + reviewer_documents="$(mktemp "${TMPDIR:-/tmp}/gate-fixture-reviewers.XXXXXX")" + synthesis_document="$(mktemp "${TMPDIR:-/tmp}/gate-fixture-synthesis.XXXXXX")" + document_source="$output_path" + if ! grep -q '^```reviewer_result_v1$' "$document_source"; then + document_source="$brief_file" + fi + awk ' + $0 == "```reviewer_result_v1" { inside=1; next } + inside && $0 == "```" { inside=0; print ""; next } + inside { print } + ' "$document_source" > "$reviewer_documents" + jq -s ' + def rcg($number): + "RCG-" + + (if $number < 10 then "00" + elif $number < 100 then "0" + else "" + end) + ($number | tostring); + . as $reviewers | + [$reviewers[].reviewer] as $selected | + (["critic","qa-tester","architecture-reviewer", + "security-reviewer","risk-reviewer"] - $selected) as $skipped | + ([$reviewers[] as $reviewer | + $reviewer.coverage[] | + { + reviewer:$reviewer.reviewer, + surface, + status, + evidence_refs, + reason + } + ]) as $coverage | + ([$reviewers[].findings[]] | sort_by(.id)) as $findings | + ($findings | to_entries | map( + .value + { + root_cause_group_id:rcg(.key + 1), + disposition:"pending" + } + )) as $union | + { + kind:"gate_synthesis_result_v1", + schema_version:1, + scope_manifest_sha256:$reviewers[0].scope_manifest_sha256, + selected_reviewers:$selected, + not_reviewed_dimensions:$skipped, + coverage_matrix:$coverage, + reviewer_finding_inventory:($findings | map({ + id, + reviewer, + severity, + hard_gate_class, + origin, + verification_expectation + })), + findings_union:$union, + root_cause_groups:($union | to_entries | map({ + id:.value.root_cause_group_id, + summary:("Fixture root cause for " + .value.id), + finding_ids:[.value.id] + })), + disagreements:[], + uncertainties:{ + finding_ids:($findings | + map(select(.origin == "uncertain") | .id)), + coverage_cells:([$reviewers[] as $reviewer | + $reviewer.coverage[] | + select(.status == "uncertain") | + { + reviewer:$reviewer.reviewer, + surface, + reason + } + ]) + }, + cautions:($findings | + map(select(.origin == "caution") | .id)), + remediation_seed:{ + kind:"remediation_closure_v1", + schema_version:1, + state:"seed", + scope_manifest_sha256:$reviewers[0].scope_manifest_sha256, + entries:($union | map({ + finding_id:.id, + reviewer, + root_cause_group_id, + disposition, + verification_expectation + })) + } + } + ' "$reviewer_documents" > "$synthesis_document" + { + printf '\n```synthesis_result_v1\n' + cat "$synthesis_document" + printf '```\n\n' + printf '## Must-Fix Order\nnone\n\n' + printf '## Advisory and Cautions\nnone\n\n' + printf '## Coverage Gaps and Uncertainties\nnone\n\n' + printf '## Recommended Verification\nnone\n' + } >> "$output_path" + rm -f -- "$reviewer_documents" "$synthesis_document" +} + pr_gate_fixture_profile_dispatch() { local executor="$1" shift @@ -169,6 +292,7 @@ pr_gate_fixture_profile_dispatch() { if [[ "$brief_file" == *-synthesis.md ]]; then printf -- '---\ngate_result_version: pr_gate_result_v1\nfinal: GO\ntier: standard\nmode: parallel\nmost_severe: advise\nreviewers:\n critic: advise\nescalation:\n recommended: false\n reviewers: []\n reason: []\n---\n\n# PR-Gate Result — stub tier\n**Date**: 2026-05-17\n**Reviewers**: stub\n**Not reviewed**: none\n\n## cross-check\nnone\n\n## Gate Conclusion\n**Overall verdict**: advise\n**Most severe individual verdict**: advise\nFinal: GO\n' > "$output_path" + pr_gate_fixture_write_synthesis_protocol "$brief_file" "$output_path" return 0 fi @@ -189,5 +313,6 @@ pr_gate_fixture_profile_dispatch() { pr_gate_fixture_write_reviewer_protocol \ "$brief_file" "$output_path" "$selected_reviewer" advise done + pr_gate_fixture_write_synthesis_protocol "$brief_file" "$output_path" printf 'Final: GO\n' >> "$output_path" } diff --git a/tests/shell/test-core-schemas.sh b/tests/shell/test-core-schemas.sh index 809966c2..8b5682dd 100755 --- a/tests/shell/test-core-schemas.sh +++ b/tests/shell/test-core-schemas.sh @@ -300,6 +300,11 @@ case_enum_sync "$CORE_DIR/schema/gate-reviewer-result.schema.json" \ "$CORE_DIR/policy/reviewer-policy.yaml" \ "reviewers" +case_enum_sync "$CORE_DIR/schema/gate-synthesis-result.schema.json" \ + '.definitions.reviewer.enum' \ + "$CORE_DIR/policy/reviewer-policy.yaml" \ + "reviewers" + # verdicts are reviewer-policy.yaml's verdicts list case_enum_sync "$CORE_DIR/schema/review.schema.json" \ '.properties.findings.items.properties.verdict.enum' \ @@ -1570,6 +1575,134 @@ case_gate_reviewer_result_abbreviated_reviewer_id_rejected() { rm -f "$tmpf" } +_gate_synthesis_result_valid_instance() { + jq -n ' + { + kind:"gate_synthesis_result_v1", + schema_version:1, + scope_manifest_sha256:("a" * 64), + selected_reviewers:["critic"], + not_reviewed_dimensions:[ + "qa-tester", + "architecture-reviewer", + "security-reviewer", + "risk-reviewer" + ], + coverage_matrix:[{ + reviewer:"critic", + surface:"changed_files", + status:"examined", + evidence_refs:[{path:"src/example.sh",line:1,symbol:null}], + reason:"The fixture examined the changed file." + }], + reviewer_finding_inventory:[{ + id:"critic-F001", + reviewer:"critic", + severity:"low", + hard_gate_class:"none", + origin:"caution", + verification_expectation:"Run the focused fixture check." + }], + findings_union:[{ + id:"critic-F001", + reviewer:"critic", + severity:"low", + hard_gate_class:"none", + origin:"caution", + source:{path:"src/example.sh",line:1,symbol:null}, + affected_behavior:"The fixture behavior remains advisory.", + why_it_matters:"Lower-severity evidence must survive synthesis.", + failure_mode:"Synthesis silently discards the caution.", + minimum_fix_boundary:"Preserve the original stable finding.", + verification_expectation:"Run the focused fixture check.", + root_cause_group_id:"RCG-001", + disposition:"pending" + }], + root_cause_groups:[{ + id:"RCG-001", + summary:"One advisory fixture root cause.", + finding_ids:["critic-F001"] + }], + disagreements:[], + uncertainties:{finding_ids:[],coverage_cells:[]}, + cautions:["critic-F001"], + remediation_seed:{ + kind:"remediation_closure_v1", + schema_version:1, + state:"seed", + scope_manifest_sha256:("a" * 64), + entries:[{ + finding_id:"critic-F001", + reviewer:"critic", + root_cause_group_id:"RCG-001", + disposition:"pending", + verification_expectation:"Run the focused fixture check." + }] + } + } + ' +} + +# Behavior: a complete synthesis parity document must satisfy its JSON schema. +# Steps: +# 1. Generate the canonical synthesis fixture. +# 2. Validate it against gate-synthesis-result.schema.json. +# 3. Assert schema validation succeeds. +case_gate_synthesis_result_valid_instance() { + local name="gate-synthesis-result: canonical parity seed validates" + should_run "$name" || return 0 + local schema_file="$CORE_DIR/schema/gate-synthesis-result.schema.json" tmpf + tmpf="$(mktemp /tmp/gate-synthesis-result-valid-XXXXXX.json)" + _gate_synthesis_result_valid_instance > "$tmpf" + if jsonschema -i "$tmpf" "$schema_file" >/dev/null 2>&1; then + pass "$name" + else + fail "$name" "schema rejected a canonical synthesis result" + fi + rm -f "$tmpf" +} + +# Behavior: a remediation seed without verification expectation must fail +# schema validation. +# Steps: +# 1. Remove verification_expectation from the canonical fixture. +# 2. Validate the mutated document against the synthesis schema. +# 3. Assert schema validation rejects the mutation. +case_gate_synthesis_result_missing_verification_rejected() { + local name="gate-synthesis-result: missing verification expectation is rejected" + should_run "$name" || return 0 + local schema_file="$CORE_DIR/schema/gate-synthesis-result.schema.json" tmpf + tmpf="$(mktemp /tmp/gate-synthesis-result-no-verification-XXXXXX.json)" + _gate_synthesis_result_valid_instance | + jq 'del(.remediation_seed.entries[0].verification_expectation)' > "$tmpf" + if jsonschema -i "$tmpf" "$schema_file" >/dev/null 2>&1; then + fail "$name" "schema accepted a seed without verification expectation" + else + pass "$name" + fi + rm -f "$tmpf" +} + +# Behavior: a remediation seed cannot claim closure before verification. +# Steps: +# 1. Change the canonical seed state to closed. +# 2. Validate the mutated document against the synthesis schema. +# 3. Assert schema validation rejects the premature closure. +case_gate_synthesis_result_closed_seed_rejected() { + local name="gate-synthesis-result: seed cannot claim closure" + should_run "$name" || return 0 + local schema_file="$CORE_DIR/schema/gate-synthesis-result.schema.json" tmpf + tmpf="$(mktemp /tmp/gate-synthesis-result-closed-seed-XXXXXX.json)" + _gate_synthesis_result_valid_instance | + jq '.remediation_seed.state = "closed"' > "$tmpf" + if jsonschema -i "$tmpf" "$schema_file" >/dev/null 2>&1; then + fail "$name" "schema accepted a remediation seed claiming closure" + else + pass "$name" + fi + rm -f "$tmpf" +} + _gate_policy_override_valid_instance() { jq -n '{ kind:"gate_policy_override_v1", @@ -1679,6 +1812,9 @@ case_gate_reviewer_result_preexisting_blocker_rejected case_gate_reviewer_result_omitted_unused_symbol_valid case_gate_reviewer_result_missing_line_and_symbol_rejected case_gate_reviewer_result_abbreviated_reviewer_id_rejected +case_gate_synthesis_result_valid_instance +case_gate_synthesis_result_missing_verification_rejected +case_gate_synthesis_result_closed_seed_rejected case_gate_policy_override_valid_instance case_gate_policy_override_non_user_approver_rejected case_gate_policy_override_extra_key_rejected diff --git a/tests/shell/test-gate-lifecycle.sh b/tests/shell/test-gate-lifecycle.sh index 4c355668..c95bf7f5 100755 --- a/tests/shell/test-gate-lifecycle.sh +++ b/tests/shell/test-gate-lifecycle.sh @@ -494,6 +494,40 @@ case_wait_resolves_failed() { fi } +# Behavior: an ordinary supervisor exit after readiness must publish a failed +# terminal sentinel that gate waiters can consume. +# Steps: +# 1. Force the fixture supervisor to exit immediately after _write_ready. +# 2. Launch the detached gate and wait for its terminal evidence. +# 3. Assert wait reports state=failed with exit 2, not indeterminate. +case_wait_reports_post_readiness_supervisor_failure() { + local name="gate-lifecycle/gate wait reports post-readiness supervisor failure" + should_run "$name" || return 0 + + local fixture="$tmp_root/c4b/fixture" work="$tmp_root/c4b/work" + mkdir -p "$work" + _mk_fixture_repo "$fixture" + _mk_fake_gate "$fixture" 0 + # Force an ordinary exit immediately after readiness. The EXIT trap must + # publish the failed terminal sentinel so wait returns failed (exit 2), not + # indeterminate (exit 3). + sed -i 's/_write_ready || _die "failed to publish supervisor readiness evidence"/_write_ready || _die "failed to publish supervisor readiness evidence"\nexit 2/' \ + "$fixture/runtime/bin/gate-supervisor.sh" + + local run_wrapper="$tmp_root/c4b/run" wait_wrapper="$tmp_root/c4b/wait" + _run_gate_wrapper "$fixture" "$run_wrapper" + _wait_wrapper "$fixture" "$wait_wrapper" + + local gate_id out code + gate_id="$($run_wrapper --cd "$work" --lifecycle detached)" + set +e; out="$($wait_wrapper "$gate_id" --cd "$work" --timeout "$_WAIT_OK" 2>&1)"; code=$?; set -e + if [[ "$code" -eq 2 ]] && [[ "$out" == *"state: failed"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + # ---- 5: gate wait on nonexistent/consumed gate_id returns 3 (indeterminate) -- case_wait_indeterminate_on_consumed_sentinel() { local name="gate-lifecycle/gate wait returns 3 for consumed/unknown gate_id" @@ -841,6 +875,7 @@ case_wait_resolves_go case_wait_reloads_verifier_over_incomplete_export case_wait_resolves_nogo case_wait_resolves_failed +case_wait_reports_post_readiness_supervisor_failure case_wait_indeterminate_on_consumed_sentinel case_wait_indeterminate_when_no_readiness_evidence case_wait_indeterminate_when_ready_supervisor_died diff --git a/tests/shell/test-pmctl-gate.sh b/tests/shell/test-pmctl-gate.sh index 41842eb6..c7a07a6d 100755 --- a/tests/shell/test-pmctl-gate.sh +++ b/tests/shell/test-pmctl-gate.sh @@ -669,6 +669,61 @@ _mk_gate_result_protocol_v3_verified() { _refresh_gate_result_protocol_v3_binding "$path" } +_mk_gate_result_protocol_v4_verified() { + local path="$1" bound_repo="${2:-$_GATE_VERIFY_REPO}" + local scope_sha synthesis + _mk_gate_result_protocol_v3_verified "$path" "$bound_repo" + sed -i \ + 's/^gate_result_version: pr_gate_result_v3$/gate_result_version: pr_gate_result_v4/' \ + "$path" + scope_sha="$(jq -r '.evidence.scope_manifest.sha256' \ + "${path}.assurance.json")" + synthesis="$( + _gate_reviewer_protocol_documents "$path" | + jq -s --arg scope_sha "$scope_sha" ' + { + kind:"gate_synthesis_result_v1", + schema_version:1, + scope_manifest_sha256:$scope_sha, + selected_reviewers:map(.reviewer), + not_reviewed_dimensions:[], + coverage_matrix:[ + .[] as $review | + $review.coverage[] | + { + reviewer:$review.reviewer, + surface, + status, + evidence_refs, + reason + } + ], + reviewer_finding_inventory:[], + findings_union:[], + root_cause_groups:[], + disagreements:[], + uncertainties:{finding_ids:[],coverage_cells:[]}, + cautions:[], + remediation_seed:{ + kind:"remediation_closure_v1", + schema_version:1, + state:"seed", + scope_manifest_sha256:$scope_sha, + entries:[] + } + } + ' + )" + { + printf '```synthesis_result_v1\n%s\n```\n' "$synthesis" + printf '## Must-Fix Order\n\nNone.\n\n' + printf '## Advisory and Cautions\n\nNone.\n\n' + printf '## Coverage Gaps and Uncertainties\n\nNone.\n\n' + printf '## Recommended Verification\n\nReuse verified fixture evidence.\n' + } >> "$path" + _refresh_gate_result_protocol_v3_binding "$path" +} + _attach_gate_scope_manifest_v3() { local path="$1" sidecar="${1}.assurance.json" local manifest manifest_digest manifest_sha @@ -849,6 +904,20 @@ _run_canonical_gate_verify() { ) } +_prepare_publication_race_sleep_hook() { + local hook_dir="$1" staged="$2" target="$3" real_sleep + real_sleep="$(command -v sleep)" + mkdir -p "$hook_dir" + cat > "$hook_dir/sleep" <&1)"; code=$?; set -e - wait "$publisher" + hook_dir="${sidecar}.hook" + _prepare_publication_race_sleep_hook "$hook_dir" "$staged" "$sidecar" + set +e + out="$(PATH="$hook_dir:$PATH" _run_canonical_gate_verify "$result" 2>&1)" + code=$? + set -e if [[ "$code" -eq 0 && "$out" == *"gate result OK"* ]]; then pass "$name" else @@ -2033,23 +2101,52 @@ case_verify_v2_publication_race_retries() { fi } +# Behavior: result-v4 canonical verification waits for the assurance sidecar +# publication rename instead of treating the first observation as a permanent +# missing-sidecar failure. +# Steps: +# 1. Build a fully verified result v4 with reviewer and synthesis protocols. +# 2. Move its sidecar aside and publish it after the verifier starts. +# 3. Assert canonical pmctl verification retries and succeeds. +case_verify_v4_publication_race_retries() { + local name="gate/verify: canonical v4 publication race retries" + should_run "$name" || return 0 + local result sidecar staged hook_dir out code + result="$(_gate_verify_result_path v4-publication-race)" + sidecar="${result}.assurance.json" + staged="${sidecar}.staged" + _mk_gate_result_protocol_v4_verified "$result" "$_GATE_VERIFY_REPO" + mv "$sidecar" "$staged" + hook_dir="${sidecar}.hook" + _prepare_publication_race_sleep_hook "$hook_dir" "$staged" "$sidecar" + set +e + out="$(PATH="$hook_dir:$PATH" _run_canonical_gate_verify "$result" 2>&1)" + code=$? + set -e + if [[ "$code" -eq 0 && "$out" == *"gate result OK"* \ + && "$out" == *"assurance: verified"* ]]; then + pass "$name" + else + fail "$name" "code=$code out=$out" + fi +} + case_verify_v2_attestation_publication_race_retries() { local name="gate/verify: canonical v2 attestation publication race retries" should_run "$name" || return 0 - local result run_root attestation staged out code publisher + local result run_root attestation staged hook_dir out code result="$(_gate_verify_result_path attestation-race)" _mk_gate_result_v2_verified "$result" "$_GATE_VERIFY_REPO" run_root="$(dirname "$(dirname "$result")")" attestation="$run_root/gate-assurance-20260727-000000.attestation.json" staged="${attestation}.staged" mv "$attestation" "$staged" - ( - sleep 0.2 - mv "$staged" "$attestation" - ) & - publisher=$! - set +e; out="$(_run_canonical_gate_verify "$result" 2>&1)"; code=$?; set -e - wait "$publisher" + hook_dir="${attestation}.hook" + _prepare_publication_race_sleep_hook "$hook_dir" "$staged" "$attestation" + set +e + out="$(PATH="$hook_dir:$PATH" _run_canonical_gate_verify "$result" 2>&1)" + code=$? + set -e if [[ "$code" -eq 0 && "$out" == *"assurance: verified"* ]]; then pass "$name" else @@ -2807,6 +2904,7 @@ case_verify_v2_sidecar_attestation_tamper case_verify_v2_subject_binding_tamper case_verify_v2_canonical_run_mismatch case_verify_v2_publication_race_retries +case_verify_v4_publication_race_retries case_verify_v2_attestation_publication_race_retries case_verify_v2_pointer_escape case_verify_v2_missing_sidecar diff --git a/tests/shell/test-pmctl-guard.sh b/tests/shell/test-pmctl-guard.sh index 8d91de8b..945fef42 100755 --- a/tests/shell/test-pmctl-guard.sh +++ b/tests/shell/test-pmctl-guard.sh @@ -264,6 +264,77 @@ if should_run "pm-prebash-deny"; then fi fi +if should_run "pm-prebash-search-literal-rm-is-allowed"; then + name="pm-prebash-search-literal-rm-is-allowed" + # A reviewer searching source text may legitimately quote a denylisted + # spelling. The guard must inspect executable command syntax, not quoted + # rg/grep data. + run_guard --event pre-bash --role pm --runtime codex \ + --command "rg -n -C 6 'failure-result|rm\\\\b.*[[:space:]](-[a-zA-Z]*r[a-zA-Z]*f)' runtime tests" + if [[ "$GUARD_EXIT" -eq 0 ]]; then + pass "$name" + else + fail "$name" "quoted search literal was incorrectly denied: code=$GUARD_EXIT out=$GUARD_OUT" + fi +fi + +if should_run "pm-prebash-search-command-substitution-denied"; then + name="pm-prebash-search-command-substitution-denied" + # Quoted search data is safe, but command substitution remains executable + # shell syntax and must retain the denylist's conservative behavior. + probe_command="rm" + probe_flags="-rf" + probe_target="/tmp/pm-dispatch-probe" + substitution='$(' + substitution+="${probe_command} ${probe_flags} ${probe_target}" + substitution+=')' + search_command="rg \"${substitution}\" runtime" + run_guard --event pre-bash --role pm --runtime codex \ + --command "$search_command" + if assert_exit "$name" "$GUARD_EXIT" "2" \ + && assert_string_contains "$name" "$GUARD_OUT" "denylisted pattern"; then + pass "$name" + fi +fi + +if should_run "pm-prebash-search-compound-quoted-fragment-denied"; then + name="pm-prebash-search-compound-quoted-fragment-denied" + # A quoted search operand is safe in isolation, but a later shell command + # must remain visible to the denylist even when its executable spelling is + # split across quotes. + search_command="grep" + safe_operand="safe" + destructive_prefix="r" + destructive_suffix="m" + destructive_flags="-rf" + destructive_target="/tmp/pm-dispatch-probe" + compound_command="${search_command} '${safe_operand}'; ${destructive_prefix}${destructive_suffix} ${destructive_flags} ${destructive_target}" + run_guard --event pre-bash --role pm --runtime codex \ + --command "$compound_command" + if assert_exit "$name" "$GUARD_EXIT" "2" \ + && assert_string_contains "$name" "$GUARD_OUT" "denylisted pattern"; then + pass "$name" + fi +fi + +if should_run "pm-prebash-executable-awk-sed-program-denied"; then + name="pm-prebash-executable-awk-sed-program-denied" + probe_command="rm" + probe_flags="-rf" + probe_target="/tmp/pm-dispatch-probe" + awk_program="BEGIN { system(\"${probe_command} ${probe_flags} ${probe_target}\") }" + sed_program="s/x/${probe_command} ${probe_flags} ${probe_target}/" + run_guard --event pre-bash --role pm --runtime codex --command "awk '${awk_program}' input.txt" + awk_exit="$GUARD_EXIT" + awk_out="$GUARD_OUT" + run_guard --event pre-bash --role pm --runtime codex --command "sed -n '${sed_program}' input.txt" + if [[ "$awk_exit" -eq 2 && "$GUARD_EXIT" -eq 2 ]]; then + pass "$name" + else + fail "$name" "awk_code=$awk_exit sed_code=$GUARD_EXIT awk_out=$awk_out sed_out=$GUARD_OUT" + fi +fi + if should_run "pm-prebash-deny-uppercase-recursive"; then # rm accepts both -r and -R for recursive; the denylist must not miss the # uppercase form (fail-open regression: a prior version's regex only diff --git a/tests/shell/test-pmctl-operation.sh b/tests/shell/test-pmctl-operation.sh index 52f6c137..9662aedf 100755 --- a/tests/shell/test-pmctl-operation.sh +++ b/tests/shell/test-pmctl-operation.sh @@ -73,6 +73,97 @@ case_reconcile_uses_trusted_terminal_claims() { fi } +case_reconcile_defers_while_producer_is_running() { + local name="operation reconcile: running producer blocks premature child convergence" + should_run "$name" || return 0 + local work="$tmp_root/producer-active-work" store="$tmp_root/producer-active-state" + local op run_id producer out rc=0 state + make_repo "$work"; run_id="run-20260724T000002Z-fedcba" + setsid sleep 30 & producer=$! + op="$(PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_create "$REPO_ROOT" "$work" gate codex)" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_expect_producer "$REPO_ROOT" gate "$op" "$work" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_register_producer "$REPO_ROOT" gate "$op" "$work" "$producer" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_attach_child "$REPO_ROOT" "$work" "$op" "$run_id" "$work" + PM_DISPATCH_STATE_ROOT="$store" _pmctl_dispatch_try_terminal_claim "$work" "$run_id" ok supervisor + out="$(PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_reconcile "$REPO_ROOT" gate "$op" --cd "$work" 2>&1)" || rc=$? + state="$(PM_DISPATCH_STATE_ROOT="$store" bash -c '. "$1/runtime/lib/state-writer.sh"; cd "$2"; _sw_project_dir' _ "$REPO_ROOT" "$work")" + kill -TERM -- "-$producer" 2>/dev/null || true; wait "$producer" 2>/dev/null || true + if [[ "$rc" -ne 0 && "$out" == *"producer-active"* \ + && "$(jq -r .state "${state%/}/operations/$op.json")" == running ]]; then + pass "$name" + else + fail "$name" "rc=$rc out=$out state=$(jq -r .state "${state%/}/operations/$op.json" 2>/dev/null || true)" + fi +} + +# Behavior: reconciliation must convert a dead registered producer into a +# diagnosable terminal state once its trusted child claim is complete. +# Steps: +# 1. Register a producer and attach a successful child terminal claim. +# 2. Kill the producer before reconciliation runs. +# 3. Assert reconciliation completes and records producer.status=stopped. +case_reconcile_recovers_dead_registered_producer() { + local name="operation reconcile: dead registered producer becomes diagnosable terminal" + should_run "$name" || return 0 + local work="$tmp_root/dead-producer-work" store="$tmp_root/dead-producer-state" + local op run_id producer out rc=0 state record + make_repo "$work"; run_id="run-20260724T000003Z-deadbe" + setsid sleep 30 & producer=$! + op="$(PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_create "$REPO_ROOT" "$work" gate codex)" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_expect_producer "$REPO_ROOT" gate "$op" "$work" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_register_producer "$REPO_ROOT" gate "$op" "$work" "$producer" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_attach_child "$REPO_ROOT" "$work" "$op" "$run_id" "$work" + PM_DISPATCH_STATE_ROOT="$store" _pmctl_dispatch_try_terminal_claim "$work" "$run_id" ok supervisor + kill -KILL -- "$producer" 2>/dev/null || true + wait "$producer" 2>/dev/null || true + out="$(PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_reconcile "$REPO_ROOT" gate "$op" --cd "$work" 2>&1)" || rc=$? + state="$(PM_DISPATCH_STATE_ROOT="$store" bash -c '. "$1/runtime/lib/state-writer.sh"; cd "$2"; _sw_project_dir' _ "$REPO_ROOT" "$work")" + record="${state%/}/operations/$op.json" + if [[ "$rc" -eq 0 && "$out" == *"state: completed"* \ + && "$(jq -r .state "$record")" == completed \ + && "$(jq -r .producer.status "$record")" == stopped ]]; then + pass "$name" + else + fail "$name" "rc=$rc out=$out record=$(jq -c . "$record" 2>/dev/null || true)" + fi +} + +# Behavior: producer identity conversion failure must fail closed as +# indeterminate rather than leaving the parent operation running forever. +# Steps: +# 1. Register a producer and attach a successful child terminal claim. +# 2. Force identity conversion to fail during reconciliation. +# 3. Assert reconciliation returns nonzero and persists state=indeterminate. +case_reconcile_rejects_malformed_producer_identity() { + local name="operation reconcile: producer identity conversion failure becomes indeterminate" + should_run "$name" || return 0 + local work="$tmp_root/malformed-producer-work" store="$tmp_root/malformed-producer-state" + local op run_id producer out rc=0 state record + make_repo "$work"; run_id="run-20260724T000004Z-badc0d" + setsid sleep 30 & producer=$! + op="$(PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_create "$REPO_ROOT" "$work" gate codex)" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_expect_producer "$REPO_ROOT" gate "$op" "$work" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_register_producer "$REPO_ROOT" gate "$op" "$work" "$producer" + PM_DISPATCH_STATE_ROOT="$store" pmctl_operation_attach_child "$REPO_ROOT" "$work" "$op" "$run_id" "$work" + PM_DISPATCH_STATE_ROOT="$store" _pmctl_dispatch_try_terminal_claim "$work" "$run_id" ok supervisor + state="$(PM_DISPATCH_STATE_ROOT="$store" bash -c '. "$1/runtime/lib/state-writer.sh"; cd "$2"; _sw_project_dir' _ "$REPO_ROOT" "$work")" + record="${state%/}/operations/$op.json" + kill -KILL -- "$producer" 2>/dev/null || true + wait "$producer" 2>/dev/null || true + out="$(PM_DISPATCH_STATE_ROOT="$store" bash -c ' + set -euo pipefail + . "$1/runtime/lib/pmctl-operation.sh" + _pmctl_operation_identity_file_from_json() { return 1; } + PM_DISPATCH_STATE_ROOT="$3" pmctl_operation_reconcile "$1" gate "$2" --cd "$4" + ' _ "$REPO_ROOT" "$op" "$store" "$work" 2>&1)" || rc=$? + if [[ "$rc" -ne 0 && "$out" == *"state: indeterminate"* \ + && "$(jq -r .state "$record")" == indeterminate ]]; then + pass "$name" + else + fail "$name" "rc=$rc out=$out record=$(jq -c . "$record" 2>/dev/null || true)" + fi +} + case_create_collision_never_overwrites_record() { local name="operation create: ID collision never overwrites an existing parent record" should_run "$name" || return 0 @@ -366,6 +457,9 @@ case_relative_cd_resolves_to_the_same_operation() { case_writer_loader_repairs_partial_inherited_functions case_reconcile_uses_trusted_terminal_claims +case_reconcile_defers_while_producer_is_running +case_reconcile_recovers_dead_registered_producer +case_reconcile_rejects_malformed_producer_identity case_create_collision_never_overwrites_record case_unknown_operation_is_diagnosed_not_silent case_reconcile_usage_on_malformed_invocation diff --git a/tests/shell/test-pr-gate.sh b/tests/shell/test-pr-gate.sh index 41d230df..3ab9a339 100755 --- a/tests/shell/test-pr-gate.sh +++ b/tests/shell/test-pr-gate.sh @@ -263,11 +263,13 @@ write_frontmatter_stub_gate_result() { local output_path="$1" local final_verdict="${2:-GO}" local final_line="Final: ${final_verdict}" - local resolved_tier resolved_mode + local resolved_tier resolved_mode staging_version frontmatter_opening resolved_tier="$(awk '/^[[:space:]]*tier\.resolved:/ {print $2; exit}' "$brief_file")" resolved_mode="$(awk '/^[[:space:]]*mode\.resolved:/ {print $2; exit}' "$brief_file")" : "${resolved_tier:=express}" : "${resolved_mode:=parallel}" + staging_version="${CODEX_GATE_STUB_RESULT_VERSION:-pr_gate_result_v1}" + frontmatter_opening="${CODEX_GATE_STUB_FRONTMATTER_OPENING:----}" # Regression seam: when CODEX_GATE_STUB_BOLD_FINAL=1, emit the Final # line wrapped in markdown bold (simulates codex applying prose emphasis). @@ -277,8 +279,9 @@ write_frontmatter_stub_gate_result() { fi cat > "$output_path" << STUB_GATE_EOF ---- -gate_result_version: pr_gate_result_v1 +${frontmatter_opening} +gate_result_version: ${staging_version} +${CODEX_GATE_STUB_ASSURANCE_FRONTMATTER:-} final: ${CODEX_GATE_STUB_FRONTMATTER_FINAL:-${final_verdict}} tier: ${resolved_tier} mode: ${resolved_mode} @@ -332,6 +335,7 @@ STUB_GATE_EOF write_reviewer_protocol_stub "$output_path" "$selected_reviewer" advise done fi + pr_gate_fixture_write_synthesis_protocol "$brief_file" "$output_path" } # Determine effective mode: synthesis briefs can have their own mode override. @@ -2223,6 +2227,9 @@ test_parallel_synthesis_brief_validates() { fail "$name" "brief-validate rejected synthesis brief (exit $vcode): $vout" return fi + assert_file_contains "$name" "$brief" "pmctl guard check --role reviewer" || return + assert_file_contains "$name" "$brief" "--event pre-write" || return + assert_file_contains "$name" "$brief" "If that call exits nonzero, abort" || return pass "$name" } @@ -3932,8 +3939,8 @@ test_gate_result_frontmatter_and_escalation() { fi local frontmatter frontmatter="$(sed -n "1,${frontmatter_end}p" "$result")" - if ! printf '%s\n' "$frontmatter" | grep -q '^gate_result_version: pr_gate_result_v3$'; then - fail "$name" "frontmatter missing gate_result_version: pr_gate_result_v3" + if ! printf '%s\n' "$frontmatter" | grep -q '^gate_result_version: pr_gate_result_v4$'; then + fail "$name" "frontmatter missing gate_result_version: pr_gate_result_v4" return fi if ! printf '%s\n' "$frontmatter" | grep -q '^gate_assurance: result.md.assurance.json$'; then @@ -4016,6 +4023,86 @@ test_gate_result_frontmatter_and_escalation() { pass "$name" } +# Behavior: executor-authored frontmatter is an untrusted staging document. +# A model that anticipates the final v4 result but omits the not-yet-published +# assurance pointer is normalized to v1 for intermediate verification; the +# shell then publishes the sidecar and atomically upgrades the result to v4. +# Steps: emit a sequential stub result with v4 frontmatter and no pointer, then +# assert successful v4 publication, the bounded pointer, and the sibling sidecar. +test_model_authored_v4_without_pointer_is_normalized_before_publication() { + local name="gate-result/model-v4-without-pointer-normalized" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" + local home="$dir/home" repo="$dir/repo" runner="$dir/runner" + local out="$dir/out" err="$dir/err" result="$dir/result.md" + mkdir -p "$dir" + create_runner "$runner" + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + create_repo "$repo" docs + + set +e + CODEX_GATE_STUB_RESULT_VERSION=pr_gate_result_v4 \ + CODEX_GATE_STUB_FRONTMATTER_OPENING=+--- \ + CODEX_GATE_STUB_SYNTHESIS_FINAL=GO run_gate \ + "$home" "$runner" "$repo" "$out" "$err" --base main --output "$result" \ + --sequential + local code=$? + set -e + if [[ "$code" -ne 0 ]]; then + fail "$name" "exit $code, expected normalized publication to succeed: $(tail -n 20 "$err")" + return + fi + assert_file_contains "$name" "$result" \ + "gate_result_version: pr_gate_result_v4" || return + assert_file_contains "$name" "$result" \ + "gate_assurance: result.md.assurance.json" || return + if [[ ! -s "${result}.assurance.json" ]]; then + fail "$name" "machine-owned assurance sidecar missing after normalized publication" + return + fi + assert_not_contains "$name" "$err" \ + "requires a bounded sibling gate_assurance pointer" || return + pass "$name" +} + +# Behavior: normalization removes at most one model-authored pointer. Multiple +# pointer keys are ambiguous input and fail closed instead of being laundered +# into a machine-owned publication. +# Steps: emit two pointer keys in the sequential staging frontmatter and assert +# the producer rejects them before publishing any assurance sidecar. +test_multiple_model_authored_assurance_pointers_fail_closed() { + local name="gate-result/multiple-model-assurance-pointers-fail" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" + local home="$dir/home" repo="$dir/repo" runner="$dir/runner" + local out="$dir/out" err="$dir/err" result="$dir/result.md" + mkdir -p "$dir" + create_runner "$runner" + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + create_repo "$repo" docs + + set +e + CODEX_GATE_STUB_RESULT_VERSION=pr_gate_result_v4 \ + CODEX_GATE_STUB_ASSURANCE_FRONTMATTER=$'gate_assurance: first.json\ngate_assurance: second.json' \ + CODEX_GATE_STUB_SYNTHESIS_FINAL=GO run_gate \ + "$home" "$runner" "$repo" "$out" "$err" --base main --output "$result" \ + --sequential + local code=$? + set -e + if [[ "$code" -eq 0 ]]; then + fail "$name" "expected ambiguous model-authored assurance pointers to fail" + return + fi + assert_file_contains "$name" "$err" \ + "staging frontmatter contains multiple model-authored gate_assurance pointers" || return + assert_file_contains "$name" "$out" "failure-result:" || return + if [[ -e "${result}.assurance.json" ]]; then + fail "$name" "ambiguous staging input must not publish an assurance sidecar" + return + fi + pass "$name" +} + # Behavior: repo-layout dispatches record the actual pmctl run id in the # machine-owned assurance envelope instead of claiming verified independence # from executor prose. @@ -4832,11 +4919,189 @@ test_scope_manifest_complete_and_shared_across_parallel_dispatch() { pass "$name" } +# Behavior: shell call-site hints are limited to files that directly reference +# the changed script, while foreign-language snippets embedded in that script +# cannot create repository-wide symbol searches. +# Steps: +# 1. Change a shell script that defines local usage and embeds a jq def flag. +# 2. Add small and large direct source consumers plus 70 unrelated same-name +# shell files; place the large consumer's match before enough padding to +# expose a pipefail/SIGPIPE race from an early-exiting grep. +# 3. Run twice on the same immutable subject and assert byte-identical +# manifests where both direct consumers become usage call-site hints; +# flag and unrelated collisions are excluded. +test_scope_manifest_shell_symbols_are_consumer_scoped() { + local name="scope-manifest/shell-symbols-are-consumer-scoped" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" + local home="$dir/home" repo="$dir/repo" + local runner="$dir/runner" out="$dir/out" err="$dir/err" + local replay="$dir/replay" out2="$dir/out2" err2="$dir/err2" + local result assurance manifest result2 assurance2 manifest2 + local first_digest second_digest code + mkdir -p "$dir" "$replay" + create_runner "$runner" + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + git init -q -b main "$repo" + ( + cd "$repo" + git config user.email test@example.com + git config user.name 'Gate Test' + mkdir -p scripts consumers unrelated + write_managed_gitignore + printf '#!/usr/bin/env bash\nold_usage() { :; }\n' > scripts/run.sh + printf '. scripts/run.sh\nusage\n' > consumers/use-run.sh + { + printf '. scripts/run.sh\nusage\n' + for n in $(seq 1 12000); do + printf '# deterministic padding %s\n' "$n" + done + } > consumers/large-use-run.sh + for n in $(seq -w 1 70); do + printf 'usage\nflag\n' > "unrelated/local-${n}.sh" + done + git add . + git commit -q -m initial + git checkout -q -b feature + cat > scripts/run.sh <<'SHELL_EOF' +#!/usr/bin/env bash +usage() { :; } +jq -n ' + def flag($pattern): $pattern; + flag("embedded-jq") +' +SHELL_EOF + git add scripts/run.sh + git commit -q -m change + ) + + set +e + run_gate "$home" "$runner" "$repo" "$out" "$err" \ + --base main --mode sequential + code=$? + set -e + [[ "$code" -eq 0 ]] || { + fail "$name" "exit $code, expected 0: $(tail -n 30 "$err" 2>/dev/null)" + return + } + result="$(awk -F'result: ' '/^result: /{path=$2} END{print path}' "$out")" + assurance="${result}.assurance.json" + manifest="$(dirname "$assurance")/$(jq -r '.evidence.scope_manifest.artifact' "$assurance")" + if ! jq -e ' + .status == "complete" and + .truncation.occurred == false and + ([.expansion.entries[] | + select(.reason == "call-site-hint" and + .source == "scripts/run.sh#usage") | .path] == + ["consumers/large-use-run.sh","consumers/use-run.sh"]) and + ([.expansion.entries[] | + select(.source == "scripts/run.sh#flag")] | length) == 0 and + ([.expansion.entries[] | + select(.reason == "call-site-hint" and + (.path | startswith("unrelated/")))] | length) == 0 + ' "$manifest" >/dev/null; then + fail "$name" "shell scope leaked across unrelated symbols: $(jq -c '{ + status,expansion,truncation + }' "$manifest" 2>/dev/null)" + return + fi + first_digest="$(sha256sum "$manifest" | awk '{print $1}')" + + set +e + run_gate "$home" "$runner" "$repo" "$out2" "$err2" \ + --base main --mode sequential --run-dir "$replay" + code=$? + set -e + [[ "$code" -eq 0 ]] || { + fail "$name" "replay exit $code, expected 0: $(tail -n 30 "$err2" 2>/dev/null)" + return + } + result2="$(awk -F'result: ' '/^result: /{path=$2} END{print path}' "$out2")" + assurance2="${result2}.assurance.json" + manifest2="$(dirname "$assurance2")/$(jq -r '.evidence.scope_manifest.artifact' "$assurance2")" + second_digest="$(sha256sum "$manifest2" | awk '{print $1}')" + if [[ "$first_digest" != "$second_digest" ]]; then + fail "$name" "same-subject manifests differ: first=$first_digest second=$second_digest" + return + fi + pass "$name" +} + +# Behavior: a language-compatible call-site query that really exceeds its +# declared per-symbol budget remains a truthful fail-closed truncation. +# Steps: +# 1. Change one TypeScript function with 65 TypeScript callers and 70 +# same-word Markdown files. +# 2. Run without acceptance and assert dispatch stops before reviewers. +# 3. Assert only the one compatible-language caller beyond the 64-path +# budget is recorded as omitted. +test_scope_manifest_semantic_search_overflow_fails_closed() { + local name="scope-manifest/semantic-search-overflow-fails-closed" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" + local home="$dir/home" repo="$dir/repo" + local runner="$dir/runner" out="$dir/out" err="$dir/err" + local manifest code + mkdir -p "$dir" + create_runner "$runner" + create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer + git init -q -b main "$repo" + ( + cd "$repo" + git config user.email test@example.com + git config user.name 'Gate Test' + mkdir -p src callers docs + write_managed_gitignore + printf 'export function scopeBudgetSymbol() { return false; }\n' > src/shared.ts + for n in $(seq -w 1 65); do + printf 'scopeBudgetSymbol();\n' > "callers/caller-${n}.ts" + done + for n in $(seq -w 1 70); do + printf 'scopeBudgetSymbol\n' > "docs/reference-${n}.md" + done + git add . + git commit -q -m initial + git checkout -q -b feature + printf 'export function scopeBudgetSymbol() { return true; }\n' > src/shared.ts + git add src/shared.ts + git commit -q -m change + ) + + set +e + run_gate "$home" "$runner" "$repo" "$out" "$err" \ + --base main --mode sequential + code=$? + set -e + [[ "$code" -eq 3 ]] || { + fail "$name" "exit $code, expected 3: $(tail -n 30 "$err" 2>/dev/null)" + return + } + assert_file_contains "$name" "$err" "INCOMPLETE: declared scope exceeded" || return + assert_not_contains "$name" "$err" "DISPATCH_STUB" || return + manifest="$(find "$repo/.gate-results" \ + -maxdepth 1 -name 'gate-scope-manifest-*.json' -print -quit)" + if ! jq -e ' + .status == "incomplete" and + .truncation.omitted.matches_per_query == 1 and + .truncation.reasons == ["search-match-budget"] and + .truncation.acceptance == { + required:true,accepted:false,source:null + } + ' "$manifest" >/dev/null; then + fail "$name" "semantic overflow was not recorded exactly: $(jq -c '{ + status,truncation + }' "$manifest" 2>/dev/null)" + return + fi + pass "$name" +} + # Behavior: the manifest producer transports a maximum-size expansion through # a file descriptor instead of one jq argv value, preserving every entry even # when the serialized array exceeds Linux MAX_ARG_STRLEN. # Steps: -# 1. Create one changed source with eight symbols and 64 callers per symbol. +# 1. Create one changed TypeScript source with eight symbols and 64 +# compatible-language callers per symbol. # 2. Run the gate so the bounded expansion contains exactly 512 entries. # 3. Assert the complete serialized expansion exceeds 128 KiB and dispatch # succeeds without an argv-size failure. @@ -4853,7 +5118,7 @@ test_scope_manifest_large_expansion_uses_file_input() { create_agents "$home" critic qa-tester architecture-reviewer security-reviewer risk-reviewer git init -q -b main "$repo" long_stem="$(printf 's%.0s' {1..220})" - source_path="src/${long_stem}.sh" + source_path="src/${long_stem}.ts" ( cd "$repo" git config user.email test@example.com @@ -4863,14 +5128,14 @@ test_scope_manifest_large_expansion_uses_file_input() { printf '# old implementation\n' > "$source_path" for n in $(seq -w 1 64); do for symbol in $(seq -w 1 8); do - printf 'scope_expansion_symbol_%s\n' "$symbol" - done > "callers/call-${n}.sh" + printf 'scope_expansion_symbol_%s();\n' "$symbol" + done > "callers/call-${n}.ts" done git add . git commit -q -m initial git checkout -q -b feature for symbol in $(seq -w 1 8); do - printf 'scope_expansion_symbol_%s() { :; }\n' "$symbol" + printf 'export function scope_expansion_symbol_%s() { return true; }\n' "$symbol" done > "$source_path" git add "$source_path" git commit -q -m change @@ -5046,6 +5311,8 @@ run_test test_rename_sensitive_old_name run_test test_binary_file_routes_to_standard run_test test_untracked_binary_routes_to_standard run_test test_scope_manifest_complete_and_shared_across_parallel_dispatch +run_test test_scope_manifest_shell_symbols_are_consumer_scoped +run_test test_scope_manifest_semantic_search_overflow_fails_closed run_test test_scope_manifest_large_expansion_uses_file_input run_test test_scope_manifest_truncation_requires_explicit_acceptance run_test test_parallel_launches_per_reviewer @@ -5056,6 +5323,8 @@ run_test test_sequential_combined_brief_validates run_test test_parallel_reviewer_brief_validates run_test test_parallel_synthesis_brief_validates run_test test_gate_result_frontmatter_and_escalation +run_test test_model_authored_v4_without_pointer_is_normalized_before_publication +run_test test_multiple_model_authored_assurance_pointers_fail_closed run_test test_repo_layout_captures_dispatch_run_id run_test test_repo_layout_preflight_failure_publishes_unattested_nogo run_test test_gate_result_final_line_back_compat @@ -5295,7 +5564,7 @@ test_post_gate_hook_runs() { fi local result_path result_path="$(awk -F'result: ' '/^result: /{path=$2} END{print path}' "$out")" - assert_file_contains "$name" "$result_path" "gate_result_version: pr_gate_result_v3" || return + assert_file_contains "$name" "$result_path" "gate_result_version: pr_gate_result_v4" || return if [[ ! -s "${result_path}.assurance.json" ]]; then fail "$name" "assurance was not finalized after the successful post-gate hook" return @@ -6228,6 +6497,8 @@ test_sequential_brief_has_citation_guard() { assert_file_contains "$name" "$brief" "Verified reference files" || return assert_file_contains "$name" "$brief" "do not invent citations" || return assert_file_contains "$name" "$brief" "agents/test-agent.md" || return + assert_file_contains "$name" "$brief" "pmctl guard check --role reviewer" || return + assert_file_contains "$name" "$brief" "--event pre-write" || return pass "$name" } @@ -6520,6 +6791,8 @@ test_seq_brief_has_reviewer_guard_constraint() { fi assert_file_contains "$name" "$brief" "pmctl guard check --role reviewer" || return assert_file_contains "$name" "$brief" "--event pre-write" || return + assert_file_contains "$name" "$brief" "construct that string from shell variables" || return + assert_file_contains "$name" "$brief" "Reference line bounds" || return pass "$name" } @@ -6558,6 +6831,8 @@ test_parallel_reviewer_brief_has_guard_constraint() { fi assert_file_contains "$name" "$reviewer_brief" "pmctl guard check --role reviewer" || return assert_file_contains "$name" "$reviewer_brief" "--event pre-write" || return + assert_file_contains "$name" "$reviewer_brief" "construct that string from shell variables" || return + assert_file_contains "$name" "$reviewer_brief" "Reference line bounds" || return pass "$name" } @@ -8195,6 +8470,48 @@ GATE_RESULT_EOF ' >> "$output_path" printf '\n```\n' >> "$output_path" done + printf '```synthesis_result_v1\n' >> "$output_path" + jq -nc --arg scope_sha "$scope_sha" ' + ["changed_files","paired_tests","sensitive_signals","public_interface", + "schema","config","install","ci","release","migration", + "bounded_expansion"] as $surfaces | + { + kind:"gate_synthesis_result_v1", + schema_version:1, + scope_manifest_sha256:$scope_sha, + selected_reviewers:["critic","qa-tester"], + not_reviewed_dimensions:[ + "architecture-reviewer","security-reviewer","risk-reviewer" + ], + coverage_matrix:([ + "critic","qa-tester" + ] | map(. as $reviewer | $surfaces | map({ + reviewer:$reviewer, + surface:., + status:"examined", + evidence_refs:[{path:"README.md",line:1,symbol:null}], + reason:"Fixture examined this declared surface." + })) | add), + reviewer_finding_inventory:[], + findings_union:[], + root_cause_groups:[], + disagreements:[], + uncertainties:{finding_ids:[],coverage_cells:[]}, + cautions:[], + remediation_seed:{ + kind:"remediation_closure_v1", + schema_version:1, + state:"seed", + scope_manifest_sha256:$scope_sha, + entries:[] + } + } + ' >> "$output_path" + printf '\n```\n\n' >> "$output_path" + printf '## Must-Fix Order\nnone\n\n' >> "$output_path" + printf '## Advisory and Cautions\nnone\n\n' >> "$output_path" + printf '## Coverage Gaps and Uncertainties\nnone\n\n' >> "$output_path" + printf '## Recommended Verification\nnone\n' >> "$output_path" fi printf 'fake Codex reviewer completed\n' > "$last" printf '%s\n' '{"type":"turn.completed","usage":{"input_tokens":10,"output_tokens":5}}' @@ -8620,7 +8937,8 @@ test_sequential_reviewer_protocol_has_independent_logical_sections() { local dir="$TMP_ROOT/$name" home="$TMP_ROOT/$name/home" local repo="$TMP_ROOT/$name/repo" runner="$TMP_ROOT/$name/runner" local out="$TMP_ROOT/$name/out" err="$TMP_ROOT/$name/err" - local result="$TMP_ROOT/$name/result.md" code block_count surface_count + local result="$TMP_ROOT/$name/result.md" code block_count synthesis_count + local surface_count mkdir -p "$dir" create_runner "$runner" create_agents "$home" critic qa-tester @@ -8635,12 +8953,14 @@ test_sequential_reviewer_protocol_has_independent_logical_sections() { return } block_count="$(grep -c '^```reviewer_result_v1$' "$result" || true)" - surface_count="$(grep -o '"surface":"' "$result" | wc -l | tr -d ' ')" - [[ "$block_count" -eq 2 && "$surface_count" -eq 22 ]] || { - fail "$name" "expected 2 complete reviewer blocks, got blocks=$block_count surfaces=$surface_count" + synthesis_count="$(grep -c '^```synthesis_result_v1$' "$result" || true)" + surface_count="$(grep -oE '"surface":[[:space:]]*"' "$result" | wc -l | tr -d ' ')" + [[ "$block_count" -eq 2 && "$synthesis_count" -eq 1 \ + && "$surface_count" -eq 44 ]] || { + fail "$name" "expected reviewer+synthesis parity blocks, got reviewer=$block_count synthesis=$synthesis_count surfaces=$surface_count" return } - assert_file_contains "$name" "$result" "gate_result_version: pr_gate_result_v3" || return + assert_file_contains "$name" "$result" "gate_result_version: pr_gate_result_v4" || return assert_file_contains "$name" "$result" "## critic -- advise" || return assert_file_contains "$name" "$result" "## qa-tester -- advise" || return pass "$name" @@ -8657,6 +8977,7 @@ test_parallel_reviewer_protocol_preserves_session_topology() { local repo="$TMP_ROOT/$name/repo" runner="$TMP_ROOT/$name/runner" local out="$TMP_ROOT/$name/out" err="$TMP_ROOT/$name/err" local result="$TMP_ROOT/$name/result.md" assurance code block_count + local synthesis_count local reviewer_brief="$TMP_ROOT/$name/reviewer-brief.md" mkdir -p "$dir" create_runner "$runner" @@ -8675,7 +8996,8 @@ test_parallel_reviewer_protocol_preserves_session_topology() { } assurance="${result}.assurance.json" block_count="$(grep -c '^```reviewer_result_v1$' "$result" || true)" - if [[ "$block_count" -ne 2 ]] || ! jq -e ' + synthesis_count="$(grep -c '^```synthesis_result_v1$' "$result" || true)" + if [[ "$block_count" -ne 2 || "$synthesis_count" -ne 1 ]] || ! jq -e ' .coordinates.mode.resolved == "parallel" and .coordinates.mode.topology == "per-reviewer-sessions" and .coordinates.coverage.selected == ["critic","qa-tester"] and @@ -8700,6 +9022,10 @@ test_parallel_reviewer_protocol_preserves_session_topology() { "soft_block/hard_block findings require severity=critical|high" || return assert_file_contains "$name" "$reviewer_brief" \ "medium/low and pre_existing/caution findings" || return + assert_file_contains "$name" "$result" \ + "gate_result_version: pr_gate_result_v4" || return + assert_file_contains "$name" "$result" \ + "## Coverage Gaps and Uncertainties" || return pass "$name" } @@ -9096,7 +9422,7 @@ test_reviewer_protocol_duplicate_heading_uses_json_verdict() { # Behavior: a blocker still completes every declared coverage surface and # becomes a formal reviewer NO-GO rather than protocol incomplete. # Steps: emit two complete blocking reviewer reports, run the parallel gate, -# then assert 22 surfaces, result v3, and Final NO-GO. +# then assert reviewer/synthesis coverage parity, result v4, and Final NO-GO. test_reviewer_protocol_blocker_completes_remaining_surfaces() { local name="reviewer-protocol/blocker-no-early-stop" should_run "$name" || return 0 @@ -9118,17 +9444,169 @@ test_reviewer_protocol_blocker_completes_remaining_surfaces() { fail "$name" "blocker run exit $code, expected reviewer NO-GO exit 1" return } - surface_count="$(grep -o '"surface":"' "$result" | wc -l | tr -d ' ')" - [[ "$surface_count" -eq 22 ]] || { - fail "$name" "blocker early-stopped coverage: expected 22 surfaces, got $surface_count" + surface_count="$(grep -oE '"surface":[[:space:]]*"' "$result" | wc -l | tr -d ' ')" + [[ "$surface_count" -eq 44 ]] || { + fail "$name" "blocker early-stopped/parity-dropped coverage: expected 44 surfaces, got $surface_count" return } - assert_file_contains "$name" "$result" "gate_result_version: pr_gate_result_v3" || return + assert_file_contains "$name" "$result" "gate_result_version: pr_gate_result_v4" || return assert_file_contains "$name" "$result" "Final: NO-GO" || return assert_not_contains "$name" "$err" "reviewer protocol INCOMPLETE" || return pass "$name" } +_write_synthesis_protocol_test_artifact() { + local artifact="$1" brief="${1}.brief" scope_sha + scope_sha="$(printf 'a%.0s' {1..64})" + printf 'artifact_sha256: %s\nartifact: %s\n' \ + "$scope_sha" "${artifact}.scope.json" > "$brief" + : > "$artifact" + pr_gate_fixture_write_reviewer_protocol \ + "$brief" "$artifact" critic block + pr_gate_fixture_write_reviewer_protocol \ + "$brief" "$artifact" qa-tester block + pr_gate_fixture_write_reviewer_protocol \ + "$brief" "$artifact" architecture-reviewer advise advisory-finding + pr_gate_fixture_write_synthesis_protocol "$brief" "$artifact" +} + +_rewrite_synthesis_protocol_json() { + local artifact="$1" filter="$2" + local original mutated rewritten start_line end_line + original="$(mktemp "${TMPDIR:-/tmp}/synthesis-original.XXXXXX")" + mutated="$(mktemp "${TMPDIR:-/tmp}/synthesis-mutated.XXXXXX")" + rewritten="$(mktemp "${TMPDIR:-/tmp}/synthesis-artifact.XXXXXX")" + awk ' + $0 == "```synthesis_result_v1" { inside=1; next } + inside && $0 == "```" { exit } + inside { print } + ' "$artifact" > "$original" + jq "$filter" "$original" > "$mutated" + start_line="$(awk '$0 == "```synthesis_result_v1" { print NR; exit }' "$artifact")" + end_line="$(awk -v start="$start_line" \ + 'NR > start && $0 == "```" { print NR; exit }' "$artifact")" + { + sed -n "1,${start_line}p" "$artifact" + cat "$mutated" + sed -n "${end_line},\$p" "$artifact" + } > "$rewritten" + mv "$rewritten" "$artifact" + rm -f -- "$original" "$mutated" +} + +# Behavior: synthesis may group two reviewers under one root cause and record +# disagreement while preserving a lower-severity caution in a separate group, +# even when all three findings cite the same file. +# Steps: build three reviewer documents, rewrite only grouping/disagreement +# judgments, then verify finding, coverage, caution, and remediation parity. +test_synthesis_protocol_preserves_grouping_disagreement_and_lower_severity() { + local name="synthesis-protocol/grouping-disagreement-lower-severity" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" artifact scope_sha + mkdir -p "$dir" + artifact="$dir/result.md" + scope_sha="$(printf 'a%.0s' {1..64})" + # shellcheck source=runtime/lib/gate-result-verify.sh + . "$REPO_ROOT/runtime/lib/gate-result-verify.sh" + _write_synthesis_protocol_test_artifact "$artifact" + _rewrite_synthesis_protocol_json "$artifact" ' + .findings_union |= map( + if .id == "architecture-reviewer-F001" + then .root_cause_group_id = "RCG-002" + else .root_cause_group_id = "RCG-001" + end) | + .root_cause_groups = [ + { + id:"RCG-001", + summary:"Critic and QA identified the same root cause.", + finding_ids:["critic-F001","qa-tester-F001"] + }, + { + id:"RCG-002", + summary:"Same file, distinct lower-severity architecture caution.", + finding_ids:["architecture-reviewer-F001"] + } + ] | + .disagreements = [{ + id:"D-001", + summary:"The reviewers disagree on the remediation emphasis.", + finding_ids:["critic-F001","qa-tester-F001"] + }] | + .remediation_seed.entries |= map( + if .finding_id == "architecture-reviewer-F001" + then .root_cause_group_id = "RCG-002" + else .root_cause_group_id = "RCG-001" + end) + ' + if ! gate_synthesis_protocol_verify \ + "$artifact" "critic qa-tester architecture-reviewer" \ + "security-reviewer risk-reviewer" "$scope_sha"; then + fail "$name" "valid grouped synthesis was rejected" + return + fi + if ! awk ' + $0 == "```synthesis_result_v1" { inside=1; next } + inside && $0 == "```" { exit } + inside { print } + ' "$artifact" | jq -e ' + .cautions == ["architecture-reviewer-F001"] and + (.reviewer_finding_inventory | + any(.id == "architecture-reviewer-F001" and .severity == "low")) and + (.root_cause_groups | + any(.finding_ids == ["critic-F001","qa-tester-F001"])) and + (.root_cause_groups | + any(.finding_ids == ["architecture-reviewer-F001"])) + ' >/dev/null; then + fail "$name" "grouping, disagreement, or lower-severity evidence was lost" + return + fi + pass "$name" +} + +# Behavior: synthesis parity fails closed for dropped/duplicate findings, +# coverage drift, missing cautions or verification expectations, and malformed +# remediation seeds. +# Steps: mutate one valid fake synthesis artifact per contract dimension and +# assert every mutation is rejected before it can become result v4. +test_synthesis_protocol_rejects_silent_drop_and_malformed_seed() { + local name="synthesis-protocol/rejects-parity-mutations" + should_run "$name" || return 0 + local dir="$TMP_ROOT/$name" artifact scope_sha mutation filter code + local failures=0 + mkdir -p "$dir" + scope_sha="$(printf 'a%.0s' {1..64})" + # shellcheck source=runtime/lib/gate-result-verify.sh + . "$REPO_ROOT/runtime/lib/gate-result-verify.sh" + while IFS='|' read -r mutation filter; do + artifact="$dir/${mutation}.md" + _write_synthesis_protocol_test_artifact "$artifact" + _rewrite_synthesis_protocol_json "$artifact" "$filter" + set +e + gate_synthesis_protocol_verify \ + "$artifact" "critic qa-tester architecture-reviewer" \ + "security-reviewer risk-reviewer" "$scope_sha" \ + >"$dir/${mutation}.out" 2>"$dir/${mutation}.err" + code=$? + set -e + if [[ "$code" -eq 0 ]]; then + fail "$name" "mutation unexpectedly passed: $mutation" + failures=$((failures + 1)) + fi + done <<'MUTATIONS' +dropped-id|.reviewer_finding_inventory |= map(select(.id != "architecture-reviewer-F001")) +duplicate-id|.reviewer_finding_inventory += [.reviewer_finding_inventory[0]] +coverage-drift|.coverage_matrix[0].reason = "Changed by synthesis." +missing-caution|.cautions = [] +missing-verification|.reviewer_finding_inventory[0].verification_expectation = "" +uncertainties-array|.uncertainties = [.uncertainties] +malformed-seed|.remediation_seed.state = "closed" +MUTATIONS + [[ "$failures" -eq 0 ]] || return + assert_file_contains "$name" "$dir/uncertainties-array.err" \ + "malformed uncertainties contract or parity mismatch" || return + pass "$name" +} + run_test test_repo_owned_reviewers_and_canonical_memory_on_clean_home run_test test_pmctl_codex_gate_uses_production_memory_on_clean_home run_test test_invalid_canonical_memory_does_not_fallback @@ -9159,5 +9637,7 @@ run_test test_parallel_reviewer_protocol_out_of_scope_reference_is_incomplete run_test test_sequential_reviewer_protocol_out_of_range_line_is_incomplete run_test test_reviewer_protocol_duplicate_heading_uses_json_verdict run_test test_reviewer_protocol_blocker_completes_remaining_surfaces +run_test test_synthesis_protocol_preserves_grouping_disagreement_and_lower_severity +run_test test_synthesis_protocol_rejects_silent_drop_and_malformed_seed th_summary diff --git a/tests/shell/test-run-tests.sh b/tests/shell/test-run-tests.sh index 29dfe25b..e72e64f7 100755 --- a/tests/shell/test-run-tests.sh +++ b/tests/shell/test-run-tests.sh @@ -304,6 +304,22 @@ case_pr_gate_protocol_contract_maps_profile_and_verifiers() { fi } +case_gate_synthesis_schema_maps_protocol_verifiers() { + local name=gate-synthesis-schema-maps-protocol-verifiers + local repo out status=0 args + args="$TMP_ROOT/$name.args" + repo="$(make_fixture "$name")" + out=$(RUN_TESTS_ARGS_LOG="$args" "$repo/tests/bin/run-tests.sh" \ + --path core/schema/gate-synthesis-result.schema.json --list 2>&1) || status=$? + if [[ "$status" -eq 0 && "$out" == *"test-core-schemas"* && + "$out" == *"test-pr-gate"* && "$out" == *"test-pmctl-gate"* && + "$out" != *"coverage gaps"* && ! -e "$args" ]]; then + pass "$name" + else + fail "$name" "status=$status out=$out" + fi +} + case_high_fanout_escalates_full() { local name=high-fanout-escalates-full repo out status=0 args args="$TMP_ROOT/$name.args" @@ -482,6 +498,7 @@ case_evidence_contract_maps_to_runner_regression case_gate_assurance_policy_maps_gate_consumers case_gate_assurance_contract_maps_runtime_verifiers case_pr_gate_protocol_contract_maps_profile_and_verifiers +case_gate_synthesis_schema_maps_protocol_verifiers case_high_fanout_escalates_full case_repeated_high_fanout_escalation_succeeds case_unknown_path_fails_without_test_evidence diff --git a/tools/generate-gate-result-verifier-fallback.sh b/tools/generate-gate-result-verifier-fallback.sh index 0093b3e3..fb76d304 100755 --- a/tools/generate-gate-result-verifier-fallback.sh +++ b/tools/generate-gate-result-verifier-fallback.sh @@ -33,6 +33,8 @@ functions=( _gate_reviewer_protocol_verdict_extract _gate_reviewer_protocol_final_extract gate_reviewer_protocol_verify + _gate_synthesis_protocol_documents + gate_synthesis_protocol_verify _gate_result_sha256_stream _gate_result_sha256_file _gate_subject_common_dir From d4cae8c6cca4be2c07721e044f6111e700231840 Mon Sep 17 00:00:00 2001 From: screenleon Date: Fri, 31 Jul 2026 17:32:42 +0900 Subject: [PATCH 2/2] fix: avoid shellcheck SC2016 in guard test --- tests/shell/test-pmctl-guard.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/shell/test-pmctl-guard.sh b/tests/shell/test-pmctl-guard.sh index 945fef42..3a381fcc 100755 --- a/tests/shell/test-pmctl-guard.sh +++ b/tests/shell/test-pmctl-guard.sh @@ -285,7 +285,7 @@ if should_run "pm-prebash-search-command-substitution-denied"; then probe_command="rm" probe_flags="-rf" probe_target="/tmp/pm-dispatch-probe" - substitution='$(' + substitution="\$(" substitution+="${probe_command} ${probe_flags} ${probe_target}" substitution+=')' search_command="rg \"${substitution}\" runtime"