diff --git a/.claude/skills/pair-capability-write-issue/SKILL.md b/.claude/skills/pair-capability-write-issue/SKILL.md index 8cc34d4d8..87a72bb7f 100644 --- a/.claude/skills/pair-capability-write-issue/SKILL.md +++ b/.claude/skills/pair-capability-write-issue/SKILL.md @@ -1,7 +1,7 @@ --- name: pair-capability-write-issue -description: "Creates or updates an issue in the adopted PM tool from a type-specific template (bug, story, epic, etc.), including topical labels (e.g. tech-debt) for deliberate promotion; `$mode: comment` posts a comment on an existing item without touching its body (the non-destructive cross-link path). Invoke directly to create/update one issue on demand. Composed by /pair-process-refine-story, /pair-process-plan-tasks, /pair-process-plan-initiatives, /pair-process-plan-epics, /pair-process-plan-stories, /pair-capability-publish-pr." -version: 0.9.0 +description: "Creates or updates an issue in the adopted PM tool from a type-specific template (bug, story, epic, etc.), including topical labels (e.g. tech-debt) for deliberate promotion; `$mode: comment` posts a comment on an existing item without touching its body (the non-destructive cross-link path). Invoke directly to create/update one issue on demand. Composed by /pair-process-refine-story, /pair-process-plan-tasks, /pair-process-plan-initiatives, /pair-process-plan-epics, /pair-process-plan-stories, /pair-process-implement, /pair-capability-publish-pr." +version: 0.11.1 author: Foomakers --- @@ -23,9 +23,10 @@ Create or update issues in the adopted PM tool. Template-driven: reads the type- | `$parent` | No | Parent issue identifier for hierarchy linking (e.g., epic → story, story → task). | | `$status` | No | Target **macrostate** — one of `Draft`, `Ready`, `In Progress`, `Review`, `Done` (never a board-specific label). Resolved to the actual board state via the `state-mapping` resolution rule ([canonical-states.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/canonical-states.md)) before the board field is updated. **Ignored in `comment` mode** (Step 6 and the board write never run — comment mode touches no board field). | | `$labels` | No | Additional **topical** labels to apply alongside the type label, e.g. `tech-debt` when a debt or quality finding is promoted to the backlog deliberately. A list of label names (created if the PM tool supports it). **Ignored in `comment` mode** (Step 7.2 never runs — labels are left byte-identical). | +| `$on-failure` | No | What a **tracker-side** failure does in write mode. `halt` (default) — it **HALTs**, as every other caller of this skill expects. `report` — it is **returned to the caller** as an outcome (`not-found`, `membership-unconfirmed`, `write-failed`) and this skill returns instead of HALTing (Step 8b). Only legitimate when the write is an **additive annotation** whose loss degrades the caller's reporting and invalidates none of its work — today exactly one composition passes it: `/pair-process-implement`'s task-progress **tick**. It exempts the three tracker-side HALTs **only** (Step 7 not-found, Step 7b beat 4 membership, Step 8 error); argument and configuration HALTs (Steps 1, 3, 6) still fire. **Ignored in `comment` mode**, which already warns instead of HALTing. | | `$assignee` | No | Who the item is **assigned to** — a login/identifier the PM tool accepts. Resolved through the cascade in Step 6b (the argument first, then the adoption default, then none) and written **as part of the create and of the update, never as a follow-up step**. Absent or unresolvable ⇒ the item is still written, unassigned, with a warning. **Ignored in `comment` mode** (an existing item's assignee is never touched by a comment). | -**In `comment` mode only `$id` and `$comment` are read**; `$type`, `$content`, `$status`, `$labels`, `$assignee` and `$parent` are ignored — passing one is a caller mistake, never a partial write. +**In `comment` mode only `$id` and `$comment` are read**; `$type`, `$content`, `$status`, `$labels`, `$assignee` and `$parent` are ignored — passing one is a caller mistake, never a partial write. `$on-failure` is ignored there too, for a different reason: comment mode already warns instead of HALTing, so there is nothing left for it to exempt. ## Algorithm @@ -152,7 +153,7 @@ The board is read **filtered by assignee**, so an item with no assignee is invis - Record the new issue identifier for return, then **re-read the created item** and report the assignee and labels the read observed (a create is a write, so the invariant above applies to it too). 3. **Act (Update)**: - Read the existing issue to confirm it exists. - - If not found → **HALT**: `Issue #$id not found.` + - If not found → **HALT**: `Issue #$id not found.` — **unless `$on-failure: report`**, which returns `not-found` to the caller instead (Step 8b) and writes nothing. - Update the issue body with the formatted content — in **write mode** (`$mode: write`) this is a **full-body overwrite**, not a merge/append: the body is replaced with what `$content` renders to. Callers that add to an existing body (EXTEND triage, plan-tasks Task Breakdown) must therefore pass the **already-merged full body**, not just the delta (see the Composition Interface below). That contract is also what makes a comment durable where the item **is** a file: on `filesystem` the back-link lives in the body's `## Activity Log` section, so a later write-mode render preserves it only because the caller read-merges the current body first — dropping the section is a caller bug, not an accepted behavior. Comment mode (Step 7c) never reaches here and never writes the body. - Preserve existing labels and hierarchy links unless explicitly changed. - Apply the assignee resolved in Step 6b **conditionally**: write it when the caller passed `$assignee` **explicitly**, or when the read above shows the item currently has **no** assignee. Otherwise **leave the existing assignee untouched** — an item deliberately assigned to someone else must not be reassigned as a side effect of a body update (the adoption default would otherwise silently pull every updated item back to the maintainer and out of that person's filtered view — the same invisibility failure, inverted). Resolved to none ⇒ leave whatever assignee the item already has; never clear one. Whether the adapter's call **adds** to or **replaces** the assignee set is the adapter's concern, documented in its guide — this skill states only which value applies and when. @@ -172,6 +173,8 @@ The order is **membership, then a read that confirms it, then the state field**. > Board membership not established — item `$id` **could not be confirmed** as a member of `[view]` after adding it (`[tool's last response]`); no board state was written either. The item itself exists and is off the board: add it to the view manually, or re-invoke. A skipped board write is **never reported as success**. + **`$on-failure: report` exempts this beat** — return `membership-unconfirmed` (Step 8b) with the same text on the `Board` row, and do not HALT. These beats run on **every** write-mode write, board state requested or not (the preamble above), so on an explicit-membership tool they sit in the path of writes that never asked to touch the board at all: an additive annotation would otherwise be sunk — and its caller stopped — by a bookkeeping read on a board it has no business with. The item's own write already landed and was confirmed by its own read in Step 7. The item is still off the board and the caller is still told exactly that: reported, never laundered into success. + 5. **Act — write the state field**, **only when Step 6 resolved one**, to the board state it resolved, per the implementation guide (e.g. the GraphQL mutation for GitHub Projects). This is the **board field**, never body text. **No board state was resolved** (no `$status` was passed) ⇒ stop here and report the membership the read in 3 confirmed, with **no state written** — the item is on the board, its column is whatever the tool's own default is, and nothing claims otherwise. 6. **Act — read the field back** and report the value the read returned. **A read-back that does not match the target is a failure**, not a success: report it as one rather than trusting the mutation's own response. 7. **Verify**: the board field, read back, equals the board state resolved in Step 6; or — when no state was requested — membership is confirmed by the read in 3 and reported as such; or one of the two failure paths above was reported. @@ -196,13 +199,28 @@ The **non-destructive** path: it appends a comment to the item and writes nothin ### Step 8: Handle Errors 1. **Check**: Did the PM tool return an error during Step 7? -2. **Skip**: If no error, proceed to output. **Comment mode is exempt** — its failures were already handled as warnings in Step 7c.2 and never reach here. +2. **Skip**: If no error, proceed to output. **Comment mode is exempt** — its failures were already handled as warnings in Step 7c.2 and never reach here. **`$on-failure: report` is exempt too** — the error is returned as `write-failed` (Step 8b), not raised. 3. **Act**: **HALT** with descriptive error: > PM tool error: `[error description]`. No fallback to alternative tools — resolve the issue with the adopted PM tool and re-invoke. 4. **Verify**: Error reported to developer. +### Step 8b: Report, Don't Raise (`$on-failure: report`) + +A **HALT propagates**: it stops the composed skill and the caller that composed it. That is right for a caller whose work the write **is** — a story that was not filed did not happen. It is wrong for a caller whose write is an **additive annotation**, where the loss degrades reporting and invalidates nothing: stopping that caller trades a whole run for a bookkeeping line. Such a caller passes `$on-failure: report`, and the three **tracker-side** failures above become returned values instead of this skill's HALT: + +| Failure | Where | Returned outcome | +| --- | --- | --- | +| `$id` does not resolve | Step 7 (update) | `not-found` — nothing was written | +| Board membership could not be confirmed by the re-read | Step 7b beat 4 | `membership-unconfirmed` — the item write landed, the board add did not | +| The PM tool returned an error | Step 8 | `write-failed` — the write did not land | + +1. **Act**: Report the outcome on the `Status` row (`Reported — `) with the tool's own message, keep the `Board` / `Labels` / `Assignee` rows on what the reads observed, and **return**. **No HALT.** +2. **Act — no retry here.** The retry budget belongs to the caller, which is the only party that knows how many attempts its own loop allows and what to record when they are spent. This skill reports once and returns. +3. **Act — the exemption is narrow, and stays narrow.** Steps 1, 3 and 6 still HALT: an unsupported `$type`, a missing template and a malformed `state-mapping` are **caller or configuration bugs** — deterministic, identical on every re-invocation, and reporting them would hand the caller an outcome it can only re-trigger. Only the **tracker-side**, intermittent failures in the table are reported. +4. **Verify**: Either the write was confirmed by its read as usual, or exactly one of the three outcomes above was returned — and in neither case did this skill HALT. + ## Output Format ```text @@ -215,8 +233,8 @@ ISSUE WRITTEN: ├── Parent: [parent issue ID | "none" | n-a (comment mode)] ├── Labels: [type label + topical labels — confirmed by read | dropped by tracker: label — finding | n-a (comment mode)] ├── Assignee: [login — confirmed by read | unchanged: login — confirmed by read | none — WARNING: invisible in an assignee-filtered view | n-a (comment mode)] -├── Board: [board state — confirmed by read | member of — confirmed by read; no state written (no $status) | n-a (no tracked view) | n-a (implicit membership; no $status, or no board state maps to the macrostate — readiness falls back to the item body) | HALT — reason] -└── Status: [Success | HALT — reason] +├── Board: [board state — confirmed by read | member of — confirmed by read; no state written (no $status) | n-a (no tracked view) | n-a (implicit membership; no $status, or no board state maps to the macrostate — readiness falls back to the item body) | membership-unconfirmed — reported, not raised ($on-failure: report) | HALT — reason] +└── Status: [Success | Reported — not-found | membership-unconfirmed | write-failed (Step 8b, $on-failure: report) | HALT — reason] ``` The `Board` row reports **membership and state separately** on purpose: an item can be a confirmed member of the tracked view with no state written (the create path, no `$status`), and a row that could only say "state" or `n-a` would render that item as absent from a board it is on. @@ -286,6 +304,13 @@ When composed by `/pair-process-plan-stories`: - **Input**: `/pair-process-plan-stories` invokes `/pair-capability-write-issue` with `$type: story`, `$content` containing the story data, and `$parent` linking to the parent epic. For an `EXTEND` triage outcome (see [to-issues-triage.md](../../../.pair/knowledge/guidelines/technical-standards/ai-development/skill-conventions/to-issues-triage.md)), it instead passes `$id` of the matched story and `$content` as the matched story's current full body with the additional scope already merged in by the caller — `/pair-capability-write-issue` overwrites the body as-is, it does not merge. - **Output**: Returns the issue identifier. `/pair-process-plan-stories` uses it for status tracking. +When composed by `/pair-process-implement` (the task-progress feedback loop, [task-progress-feedback.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md)): + +- **Input, per task (the tick)**: `$type: story`, `$id: [story-id]`, **`$on-failure: report`**, `$content` = the story's **current full body with exactly one checklist line patched** (`[ ]` → `[x]`) by the caller. The same read-merge contract `/pair-process-plan-tasks` is held to, applied to a single line: this skill overwrites the body with what it is given, so a caller that re-renders instead of patching destroys the story's AC/DoD/task breakdown. `/pair-process-implement` diff-checks its own patch before composing here, and abandons a write whose diff is not one checkbox-only line. +- **Input, once per invocation (the batch)**: `$mode: comment`, `$id: [story-id]`, `$comment` = the rendered progress batch. No `$type`, no `$content`, no `$status` — the body is not touched by the flush. +- **Output**: the issue identifier (tick) / `Commented` or `Comment warned` (batch). **Neither failure is load-bearing**: `/pair-process-implement` records the outcome in its batch and continues the story. Comment mode already warns instead of HALTing; the tick reaches the same place through `$on-failure: report`, so Step 7's `not-found`, Step 7b beat 4's unconfirmed membership and Step 8's tracker error come back as the three outcomes in **Step 8b** — `not-found`, `membership-unconfirmed`, `write-failed` — instead of HALTing. This is not a courtesy: a HALT here propagates, so without it a single 5xx while ticking task 1 of 4 would end the run — T2, T3 and T4 never implemented and no PR — because one checkbox could not be written. +- **Dedup is the caller's**, as for every comment-mode composition: the tick is naturally idempotent (an already-`[x]` item is not re-written at all), and the batch is posted exactly once per invocation by construction. + When composed by `/pair-capability-publish-pr` (the cross-link back-link, split PM-tool/code-host projects): - **Input**: `/pair-capability-publish-pr` invokes `/pair-capability-write-issue` with `$mode: comment`, `$id: ` (the PM tool's own item id, verbatim) and `$comment: "PR: "` — no `$type`, no `$content`, no `$status`. Only the comment is written: the story body (AC, DoD, task breakdown) is never re-rendered or overwritten. @@ -308,9 +333,11 @@ When invoked **independently**: - **Template not found** (Step 3) — missing knowledge base file. - **Target macrostate has no mapped board state** (Step 6) — reports the gap instead of guessing. - **Malformed `state-mapping` section** (Step 6) — points to canonical-states.md. -- **Board membership could not be confirmed** by the re-read after adding it (Step 7b) — the item is off the board, and a skipped board write is never reported as success. Fires whenever a tracked view is named, `$status` or not (membership does not depend on a requested state). **Not** the minimal-board case (Step 6), which is a documented skip of the **state field** only. -- **`$id` provided but issue not found** (Step 7) — issue does not exist. **Not a HALT in `comment` mode** — warn with the manual-link instruction (Step 7c.2). -- **PM tool error** (Step 8) — no fallback, descriptive error reported. **Not a HALT in `comment` mode** — warn (Step 7c.2). +- **Board membership could not be confirmed** by the re-read after adding it (Step 7b) — the item is off the board, and a skipped board write is never reported as success. Fires whenever a tracked view is named, `$status` or not (membership does not depend on a requested state). **Not** the minimal-board case (Step 6), which is a documented skip of the **state field** only. **Not a HALT under `$on-failure: report`** — returned as `membership-unconfirmed` (Step 8b). +- **`$id` provided but issue not found** (Step 7) — issue does not exist. **Not a HALT in `comment` mode** — warn with the manual-link instruction (Step 7c.2). **Not a HALT under `$on-failure: report`** — returned as `not-found` (Step 8b). +- **PM tool error** (Step 8) — no fallback, descriptive error reported. **Not a HALT in `comment` mode** — warn (Step 7c.2). **Not a HALT under `$on-failure: report`** — returned as `write-failed` (Step 8b). + +The three rows above are the **tracker-side** failures, and they are the only ones `$on-failure: report` exempts: they are intermittent (a 5xx, a secondary rate limit, an add that exited 0 having done nothing), so a caller that can carry on has something to carry on with. Every other row is a caller or configuration bug — deterministic, identical on re-invocation — and stays a HALT for every caller. ## Extensibility @@ -330,6 +357,7 @@ See [graceful degradation](../../../.pair/knowledge/guidelines/technical-standar - This skill **modifies PM tool state** — it creates and updates issues, and posts comments on them. It never touches code-host state (branches, PRs, reviews). - **No write is assumed** (Step 7): every write here is re-read back, because a tracker can return success for a write that did not happen. The rule is general on purpose — the concrete bug was `gh project item-add` exiting 0 with nothing created, but the class is "trusted the exit status", and only the read closes it. - **Comment mode is deliberately narrow**: one verbatim comment on one existing item, no template, no body write, no board write, warn-not-HALT on failure. It exists so a cross-link (or any additive annotation) never risks the item's body — the destructive full-body overwrite is write mode's contract alone. +- **`$on-failure: report` is warn-not-HALT for an additive write-mode write** (Step 8b) — the same reasoning as comment mode, applied to the one write-mode composition whose payload is an annotation (`/pair-process-implement`'s task-progress tick). It changes **who raises**, never what is written or what is reported: the failure still comes out of a read, still names the item, and is still never a success. - No PM tool fallback: if the adopted tool fails, the skill HALTs. **Idempotent in write mode** — see [idempotency convention](../../../.pair/knowledge/guidelines/technical-standards/ai-development/skill-conventions/idempotency.md): `$id` prevents duplicate creation on re-invocation. **Comment mode is the documented exception**: a comment carries no `$id` of its own, so nothing here can dedupe it — re-invocation appends another comment, and the duplicate-check belongs to the caller (Step 7c). - Template = source of truth for issue body format. Changes to template structure automatically affect all future issue creation. - Labels and hierarchy linking follow the PM tool implementation guide conventions. diff --git a/.claude/skills/pair-loop/SKILL.md b/.claude/skills/pair-loop/SKILL.md index 0c6340b92..fe115e9dc 100644 --- a/.claude/skills/pair-loop/SKILL.md +++ b/.claude/skills/pair-loop/SKILL.md @@ -1,7 +1,7 @@ --- name: pair-loop description: "Unattended delivery loop: per iteration, selects eligible cards via pair-next, runs a dependency + mutex analysis, composes implement-batch for a mutex-safe parallel batch (or drives one card sequentially), enacts the automation policy (auto-advance) and evaluates the stop predicate. In an environment with a fan-out runner it delegates the whole unattended run to the pair-loop workflow; elsewhere it drives exactly one eligible card and reports a continue-token." -version: 0.1.0 +version: 0.1.1 author: Foomakers --- @@ -50,6 +50,7 @@ Fan-out is ONE capability with THREE realizations, in preference order: **(1) in - **Never re-implements the per-card pipeline.** Implement→PR→review is `/implement-batch`'s (#219); this skill composes it and consumes its outcomes only (PR opened/updated, review-approved, escalated, failed). - **Never merges outside the tier the policy permits.** Merge authority belongs to `## Auto-Advance`, never to this skill inventing a looser rule, and never to branch protection it cannot count on (a project with `Review enforcement: disabled` gets no safety net there — this skill verifies the 🟢 gate set itself before ever pushing/merging). +- **Never posts its own per-task progress comments.** The breakdown-to-task feedback loop — checklist ticks plus one batched comment per run iteration — belongs to `/pair-process-implement` ([task-progress-feedback.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md)), which this skill reaches through `/implement-batch`, so a supervised run already carries it. A second writer here would double every line. The one comment this skill does post is its own: the card-level "awaits human action" note in the degraded path, which reports the loop's decision, not a task's outcome. - **Never modifies `/pair-next`.** Selection stays the frozen atom (ADR-017 §1) — this skill only ever passes `--root`/`--filter`. - **Never widens an override.** A pin-sequential or exclude override may only narrow the parallel set the dependency+mutex analysis computed; it can never add back an excluded card. diff --git a/.claude/skills/pair-process-implement/SKILL.md b/.claude/skills/pair-process-implement/SKILL.md index 38f65cf88..2855e1910 100644 --- a/.claude/skills/pair-process-implement/SKILL.md +++ b/.claude/skills/pair-process-implement/SKILL.md @@ -1,7 +1,7 @@ --- name: pair-process-implement description: "Implements a refined user story task-by-task, via a 5-step cycle per task (context, branch, implementation, quality, commit). At the closing phase it writes a checkpoint and publishes the PR through a handoff-only subagent (clean context), resuming from the checkpoint when re-invoked on an interrupted story. Composes /pair-capability-verify-quality, /pair-capability-record-decision, /pair-capability-checkpoint, /pair-capability-publish-pr." -version: 0.6.1 +version: 0.7.4 author: Foomakers --- @@ -23,6 +23,7 @@ Implement a user story by processing its tasks sequentially. Each task follows a | `/pair-capability-record-decision` | Capability | Yes — invoked when a decision needs recording | | `/pair-capability-checkpoint` | Capability | Yes — `$mode=resume` at the opening phase (resume probe), `$mode=write` at the closing phase (handoff artifact). If not installed, degrade to git+PM-tool resume (see Graceful Degradation). | | `/pair-capability-publish-pr` | Capability | Yes — the closing phase composes it (gate → PR → board) inside a handoff-only subagent. If not installed → **HALT** (implement never re-implements PR creation). | +| `/pair-capability-write-issue` | Capability | Optional — the task-progress feedback writer ([task-progress-feedback.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md)): write mode for the tick (Step 2.8, composed with **`$on-failure: report`** so a tracker error returns an outcome instead of a propagating HALT), `$mode: comment` for the batched progress comment (Step 3.1b). If not installed, warn and continue **without** ticks or comment — missing feedback never blocks the story. | | `/pair-capability-assess-stack` | Capability | Optional — invoked when a new dependency is detected. If not installed, warn and continue. | | `/pair-capability-verify-adoption` | Capability | Optional — invoked before commit to check adoption compliance. If not installed, warn and continue. | @@ -128,7 +129,7 @@ Ask: _"Ready to proceed with implementation?"_ > **Commit strategy for this story:** > 1. **Commit per task** (recommended) — develop one task, ask dev, commit, update checkbox, next task. Single PR at end. - > 2. **Commit per story** — develop all tasks continuously, then ask dev, commit all, update all checkboxes, single PR. + > 2. **Commit per story** — develop all tasks continuously, ticking each checkbox as it completes, then ask dev, commit all, single PR. 4. **Verify**: Strategy is set. Apply consistently for the entire story. @@ -139,8 +140,11 @@ Process tasks **sequentially**, one at a time. For each task: ### Step 2.1: Select Next Task 1. **Check**: Scan all tasks in dependency order. Find the first task that is not yet completed. - - A task is "completed" if its checklist item is marked ✅ in the story AND (if commit-per-task) the commit exists on the branch. -2. **Skip**: If all tasks are completed, move to Phase 3. + - A task is "completed" if its checklist item is marked ✅ in the story **AND the work it claims is still where this run can see it**: under `commit-per-task`, its commit exists on the branch; under `commit-per-story`, either the story's single commit exists (Step 3.1 already ran) **or** the working tree still carries that task's change. A ✅ with neither is **not** completion evidence — treat the task as **pending and re-attempt it** (re-ticking is a no-op: the patch never unticks). + - **Why the `commit-per-story` half is load-bearing.** On that strategy Step 2.8 item 7 ticks _before_ any commit exists, so the tick can outrun the work: 4-task story, `commit-per-story` (Step 1.3 auto-selects it for every single-task story). T1–T3 complete ⇒ three `[x]` on the body and a current checkpoint; the run then HALTs on T4's red gate (Step 2.7) and the worktree is discarded. The next attempt starts from the branch head, which under this strategy carries **zero** commits. Trusting the ✅ alone, this step declares T1–T3 completed and never re-implements them, Step 3.1 commits a T4-only (or empty) tree, and Step 3.3 opens a PR missing three tasks' work over a body showing them done — "reports work that was not done and nothing later contradicts it", the failure [task-progress-feedback.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md) forbids for guess-ticks. + - A task that **cannot be attempted this iteration** — an unmet dependency, an external blocker, work the developer defers or puts out of scope — is neither a HALT nor a failure: **queue it as `skipped`** with its one-line reason (the vocabulary is [task-progress-feedback.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md)'s), leave its checklist item unticked, and continue the scan to the next task. `failed` is for a task that was attempted and did not land: a deliberate deferral reported as `failed` misnames it, and one left silent shrinks the iteration's headline count without saying why. + - **At most once per invocation.** This scan restarts from the top after every task (item 1 is re-entered from Step 2.8 item 9), so a task that cannot be attempted is reached again on every later pass: a task **already queued as `skipped` this invocation** is passed over **silently** — no second queue entry, no second line. Otherwise a 5-task story blocked on T3 queues it after T2 _and_ after T4, and the one comment carries six lines for five tasks — `Task progress — 4 done, 2 skipped of 5 tasks this iteration` — with T3 listed twice. +2. **Skip**: If no task remains that this iteration can attempt — all completed, or the rest queued as `skipped` — move to Phase 3. 3. **Act**: Set the active task. Update session state: ```text @@ -219,7 +223,7 @@ Follow the TDD discipline rules strictly, and the [Design Rules](../../../.pair/ ### Step 2.8: Task Completion 1. **Check**: Is the commit strategy `commit-per-task`? -2. **Skip**: If `commit-per-story`, continue to next task — return to Step 2.1. No inter-task confirmation. +2. **Skip**: If `commit-per-story`, there is no inter-task confirmation and no commit here — but **still apply item 7 (tick and queue) and item 8** for the task just completed, then go to item 9. The tick-and-queue is not a property of the commit strategy: a 4-task `commit-per-story` story whose gate goes red on T3 (Step 2.7 → HALT) would otherwise queue only the failure, and the body would still show `- [ ] T1`, `- [ ] T2` over finished work — the state where "on task 3 of 4" and "failed on task 2" look identical, which is what this loop exists to end. Step 1.3 auto-selects this strategy for every single-task story, so it is the common path, not an exotic one. (The tick records the task's **work** as done, not a commit: on this strategy the single commit lands at Step 3.1. What the tick does **not** become is standalone completion evidence — Step 2.1 accepts a `commit-per-story` ✅ only while the commit or the working-tree change backs it, so a tick that survives a discarded worktree is re-attempted, not trusted.) 3. **Act** (BLOCKING): Present task summary and **ask developer for confirmation BEFORE committing**: ```text @@ -246,9 +250,10 @@ Follow the TDD discipline rules strictly, and the [Design Rules](../../../.pair/ ``` 6. **Verify**: Commit created. -7. **Act**: Update the PM tool story issue body: - - Mark the completed task checkbox (`- [x] **T-N**`) in the **Task Breakdown** section. - - Mark any **Definition of Done** checkboxes that are now factually satisfied by this task's work (e.g., "SKILL.md created", "template validated"). Leave unchecked items that require reviewer confirmation (e.g., "Code reviewed and merged"). +7. **Act — tick and queue** (**every strategy** — item 2 routes `commit-per-story` through here too; the task-progress feedback loop, per [task-progress-feedback.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md); the mechanism is **not** restated here): + - Locate the task's checklist item in the **Task Breakdown** section by its **task ID**, and tick it (`- [ ]` → `- [x]`) with the tick-only, diff-checked patch the guideline defines. Mark any **Definition of Done** checkbox now factually satisfied by this task's work (e.g., "SKILL.md created", "template validated") by the same rule — **one write per checkbox**, each with its own read and its own one-line diff check, never two boxes in one body; leave the ones needing reviewer confirmation (e.g., "Code reviewed and merged"). + - **Queue** the task's outcome line for this invocation's batched comment — `ticked`, or the outcome the guideline's vocabulary gives when the tick did not land. The queue is flushed once, at Step 3.1b. + - **Never blocking**: a locator mismatch, a rejected patch or a failed write is queued and the run continues. A missing tick is a reporting defect, never a reason to stop implementing. The transport is held to it: the tick composes `/pair-capability-write-issue` write mode with **`$on-failure: report`**, so an unresolvable id, a tracker error and an unconfirmed board membership come back as `not-found` / `write-failed` / `membership-unconfirmed` (its Step 8b) instead of a **HALT that would propagate here** and end the story on the task it only annotates. `membership-unconfirmed` says nothing about the tick — the body write landed — so it goes in the batch's `
`, not on the task's outcome line. 8. **Act — persist progress**: If `/pair-capability-checkpoint` is installed, compose `/pair-capability-checkpoint $mode=write` to update `.pair/working/checkpoints/.md` with the tasks now done. This keeps the checkpoint current so an interruption after this task resumes from the next pending one (Step 0.0). If `/pair-capability-checkpoint` is not installed, skip — git+PM state still supports the git-based resume. 9. **Check**: Is this the last task? - **Yes**: Move to Phase 3 (Closing: checkpoint + PR). @@ -261,7 +266,7 @@ The task cycle is done. The closing phase writes the checkpoint (handoff artifac ### Step 3.1: Final Commit (if commit-per-story) 1. **Check**: Is the commit strategy `commit-per-story`? -2. **Skip**: If `commit-per-task`, all commits already exist. Move to Step 3.2. +2. **Skip**: If `commit-per-task`, all commits already exist. Move to **Step 3.1b** — the batch still has to be flushed; the recommended strategy is not the one that reports nothing. 3. **Act** (BLOCKING): Present summary and **ask developer for confirmation BEFORE committing**: ```text @@ -286,9 +291,17 @@ The task cycle is done. The closing phase writes the checkpoint (handoff artifac ``` 6. **Verify**: Commit created with all changes. -7. **Act**: Update the PM tool story issue body: - - Mark ALL task checkboxes (`- [x] **T-N**`) in the **Task Breakdown** section. - - Mark all **Definition of Done** checkboxes that are factually satisfied by the implementation. Leave unchecked items that require reviewer confirmation (e.g., "Code reviewed and merged"). +7. **Act — tick and queue (catch-up only)**: every task completed **in this invocation** was already ticked and queued at Step 2.8 item 7, on either strategy. Apply that item here only to a task completed **in this invocation** that did not reach it (an interrupted pass) — same locator, same tick-only patch, same queued outcome line. A task this invocation never attempted — already `[x]` from an earlier one — is **neither re-written nor queued**: a re-invocation that found everything done must reach Step 3.1b with an empty queue and post nothing, instead of accreting one identical progress comment per attempt. Leave Definition-of-Done boxes that require reviewer confirmation unchecked. + +### Step 3.1b: Flush the Task-Progress Comment + +The batch queued across this invocation is posted **exactly once**, here, before the hand-off — so the story carries its progress narrative whether or not the PR pipeline that follows succeeds. + +1. **Check**: Is the queue empty — nothing queued this invocation, or **already flushed** by an earlier pass through this step? If so, post nothing and move to Step 3.2: an empty batch never becomes a "nothing to report" comment, and a flushed one never becomes a second comment. +2. **Act**: Render the batch in the D22 shape [task-progress-feedback.md](../../../.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md) defines (headline, one line per task, everything longer collapsed in `
`) and post it with `/pair-capability-write-issue` `$mode: comment`, `$id` = the story id. One comment per run iteration — never a second one, whatever the batch holds. +3. **Act — degradation**: `/pair-capability-write-issue` not installed, or the comment warns instead of posting ⇒ report the queued lines in this skill's own output and continue. The PR is not held for a comment. +4. **Act — drain**: Whatever the outcome (posted, or degraded to this skill's output), the queue is now **empty and marked flushed for this invocation**. A later HALT reports its blocker without re-posting the batch it already sent. +5. **Verify**: Either the comment is confirmed by a read of the item's comments, or the warning plus the queued lines were surfaced. In both cases the story body, labels and board state are unchanged by this step. ### Step 3.2: Write the Checkpoint (handoff artifact) @@ -338,6 +351,7 @@ IMPLEMENTATION COMPLETE: ├── Strategy: [commit-per-task | commit-per-story] ├── Tasks: [N/N completed] ├── Commits: [N commits on branch] +├── Progress: [N ticked, N failed/skipped — 1 comment posted | queue empty, no comment | comment warned — lines below] ├── Checkpoint: [.pair/working/checkpoints/.md — written] ├── Context: [clean — subagent handoff-only | degraded — inline publish, no subagent reset] ├── PR: [#PR-number — URL — Created | Updated (from /publish-pr)] @@ -361,7 +375,7 @@ Implementation stops immediately when: - **Quality gate red inside `/pair-capability-publish-pr`** (Step 3.3) — propagates as implement's HALT; no PR side effects (the PR-template-not-found and gate HALTs live in `/pair-capability-publish-pr`) - **PR state is not `ready-to-merge`** (Step 4.1) — `merge_allowed` fails: red gate, review not approved/still pending, or 🔴 without an explicit non-author human approval on the current head. Never bypass a required check to merge -On HALT: report the blocker clearly, propose resolution, wait for developer. +On HALT: **flush the task-progress batch first — unless Step 3.1b already flushed it this invocation** (Step 3.1b's rendering, posting and drain rules, unchanged — the run that stopped is the one with the most to report, and a flush reached only on the success path would leave exactly the failed iteration silent; but the Step 3.3 HALTs fire _after_ Step 3.1b has posted, and re-flushing there is the second comment per iteration the guideline forbids), then report the blocker clearly. A HALT after the flush therefore leaves the item's comment reading all-✅ with **nothing on it saying no PR was produced** — by design, not by omission: the batch is a per-task narrative and the run's outcome belongs to the PR's own state and to this skill's output block (see the guideline's batching rules). Do not append a run-outcome comment to compensate, propose resolution, wait for developer. The failing task is queued as `failed` with its reason, so its checklist item stays unticked. ## Idempotent Re-invocation @@ -370,7 +384,7 @@ See [idempotency convention](../../../.pair/knowledge/guidelines/technical-stand 1. **Checkpoint**: the opening-phase resume probe (Step 0.0) reads the checkpoint and jumps to the first pending task — never repeating completed tasks. When `/pair-capability-checkpoint` is absent, the git+PM resume below applies. 2. **Branch**: detects existing branch, switches to it. 3. **Commit strategy**: if commits already exist on branch, infer strategy from history. -4. **Tasks**: scans task checklist and git log to identify completed tasks. Skips them. +4. **Tasks**: scans task checklist and git log to identify completed tasks, and does **not re-run them** — this is **not the `skipped` outcome** (Step 2.1): a task an earlier invocation completed is neither re-written nor queued, so a re-invocation that finds everything done reaches Step 3.1b with an empty queue and posts nothing. 5. **PR**: the closing phase re-composes `/pair-capability-publish-pr`, which detects an existing PR and updates it in place — never a duplicate. A subagent that failed mid-PR is recovered this way (the checkpoint stays valid, the rerun is idempotent). 6. **Quality gates**: re-runs all gates (fast if already passing). 7. **Merge**: if a PR exists and its state synthesizes to `ready-to-merge`, proceeds directly to Phase 4 (merge); otherwise Phase 4's Step 4.1 HALTs with the unmet condition. diff --git a/.claude/workflows/pair-implement-batch.js b/.claude/workflows/pair-implement-batch.js index f95817702..f18751020 100644 --- a/.claude/workflows/pair-implement-batch.js +++ b/.claude/workflows/pair-implement-batch.js @@ -795,7 +795,7 @@ const MAX_FIX_ROUNDS = PIPELINE.maxFixRounds // truncated structured output) — and the contentless shape is the one this repo // actually measured on #432 (the machine slept mid-response), i.e. the retry // missed the exact incident it was written for while covering its rarer sibling. -// The review step therefore passes `hasVerdict`, the SAME predicate its +// The review step therefore passes `hasReviewEvidence`, the SAME predicate its // convergence guard uses, so "did not review" means one thing at both sites: the // transient gets its second chance, and a step that comes back contentless twice // still fails closed. @@ -812,6 +812,11 @@ async function agentRetry(prompt, opts, isUsable = r => !!r) { // ONE predicate, asked by the retry and by the convergence guard, so the two // cannot drift into disagreeing about what a dead reviewer is. const hasVerdict = r => !!r && !!String(r.verdict ?? '').trim() +const REVIEWED_HEAD_PATTERN = /^[0-9a-f]{40}$/ +// A review also has to identify the immutable PR revision it actually inspected. +// Without that baseline a later reviewer cannot distinguish the fix delta from the +// already-audited PR surface, which turns each re-review into another full scan. +const hasReviewEvidence = r => hasVerdict(r) && REVIEWED_HEAD_PATTERN.test(String(r.reviewedHead ?? '')) // ── Schemas (orchestration return-value contracts) ───────────────────────── // These are the compact values agents RETURN for control-flow — NOT the artifact @@ -854,6 +859,9 @@ const LOOSE_REVIEW_SCHEMA = { // Control flow keys on `nonActionable` + actionable count, never on specific // verdict strings. verdict: { type: 'string' }, + // Immutable full SHA of the PR head reviewed. This is workflow evidence, not + // part of the human-facing review template vocabulary. + reviewedHead: { type: 'string', pattern: '^[0-9a-f]{40}$' }, needsHumanDecision: { type: 'boolean' }, findings: { type: 'array', @@ -879,7 +887,7 @@ const LOOSE_REVIEW_SCHEMA = { }, }, }, - required: ['verdict'], + required: ['verdict', 'reviewedHead'], } const FIX_SCHEMA = { type: 'object', @@ -977,7 +985,17 @@ const contracts = STORIES.length ? await parallel(CONTRACT_SPECS.map((s) => () = const crContract = contracts.find((c) => c.name === 'code-review') // Schema the reviewer returns: template-derived when the contract is usable, // the loose skeleton otherwise. Control flow stays value-agnostic either way. -const REVIEW_SCHEMA = crContract?.schema ?? LOOSE_REVIEW_SCHEMA +const REVIEW_SCHEMA_BASE = crContract?.schema ?? LOOSE_REVIEW_SCHEMA +// Template contracts own human verdict/finding vocabulary. The orchestration-only +// baseline is layered on top so a template refresh cannot accidentally remove it. +const REVIEW_SCHEMA = { + ...REVIEW_SCHEMA_BASE, + properties: { + ...REVIEW_SCHEMA_BASE.properties, + reviewedHead: { type: 'string', pattern: '^[0-9a-f]{40}$' }, + }, + required: [...new Set([...(REVIEW_SCHEMA_BASE.required ?? []), 'verdict', 'reviewedHead'])], +} // Reviewer prompt vocabulary: `verdictOptions` and `severities` are CANONICAL, // required contract keys (ensure-contract.mjs's validateContract rejects any // contract missing either) — so whenever a contract IS present, both are @@ -1006,6 +1024,16 @@ const TEXT_SHAPE = '(specific inputs/state -> the wrong output or the loss that follows) and the EVIDENCE it is real ' + '(what you ran, what it printed). Cut narration, never evidence.' +const AUTHORITATIVE_BOUNDARY_PROOF = + 'AUTHORITATIVE BOUNDARY PROOF (mandatory): when a table row, equivalence, normalization or remediation depends on an external command, service, file format or runtime, name the exact real producer/consumer that defines it and run a minimal isolated end-to-end probe for every such claim. Keep rows distinct until that boundary proves them equivalent. A unit test of the function being changed cannot establish external semantics or prove that user-facing repair advice works: apply the advice in a clean temporary environment and verify the promised postcondition.' + +const CONTRACT_INVENTORY = + 'CONTRACT INVENTORY (mandatory): before reporting findings, map each changed observable contract to its authoritative producer, inputs, consumers and representations. A FIRST review inventories every changed contract; a re-review inventories only its fix delta and directly changed boundary. For a finite protocol, parser, configuration, state transition or command-output domain, build a finite decision table of every supported state plus its invalid/boundary pair, and probe the real behavior. Report every defect that table exposes now; do not leave ordinary rows for a later review. ' + + AUTHORITATIVE_BOUNDARY_PROOF + +const FINITE_STATE_COMPLETENESS = + 'FINITE-STATE COMPLETENESS (mandatory when a change parses, selects, snapshots, or branches on a finite protocol/state domain): identify the authoritative grammar or producer, make the complete decision table of supported states and invalid/boundary cases, then write and run a real test for every row before editing the canonical source. Do not implement one newly discovered row at a time and wait for re-review to name the next ordinary variant. ' + + AUTHORITATIVE_BOUNDARY_PROOF const SEVERITIES = (REVIEW_VOCAB?.severities ?? DEFAULT_SEVERITIES).join(', ') const VERDICTS = (REVIEW_VOCAB?.verdictOptions ?? DEFAULT_VERDICTS).join(', ') @@ -1021,7 +1049,28 @@ const SEVERITY_SCALE = resolveSeverityScale(REVIEW_VOCAB?.severities, crContract // unranked one stays unranked until the template changes, and the next caller who does pass // a floor gets a hard stop. Better the operator sees it on the run that generated it. if (SEVERITY_SCALE.rankError) log(`contract:code-review: severities are NOT ranked (${SEVERITY_SCALE.rankError}) — \`severityFloor\` is unavailable until the contract is regenerated`) -const SEVERITY_FLOOR = parseFloor(PARSED.severityFloor) +// The floor DEFAULTS to `Minor`, so Major and Minor block and drive fix rounds while +// everything below them is carried to the merge gate. Measured on PR #477 across three +// cycles: the PR reached a zero-actionable APPROVED, the next round implemented review +// Questions the reviewer had marked "No change requested", and the re-review found new +// Minors inside the code that round had just added — three the first time, two the second. +// Questions are, by the review template's own definition, questions FOR THE HUMAN; putting +// them in the fix set contradicts what they are and makes convergence a moving target. +// An explicit `severityFloor` still wins, including a lower one that restores the old +// block-everything behaviour. +// +// The default is applied SOFTLY, unlike a caller-passed floor: a template whose vocabulary +// does not declare `Minor`, or whose contract carries no ranking, falls back to no floor +// rather than throwing. A default must never break a run that never asked for it; a floor +// the CALLER spelled wrong still throws, because that is a configuration error they made. +const DEFAULT_SEVERITY_FLOOR = 'Minor' +function defaultFloor() { + if (!SEVERITY_SCALE.ranks) return null + const key = normSeverity(DEFAULT_SEVERITY_FLOOR) + if (!Object.hasOwn(SEVERITY_SCALE.ranks, key)) return null + return { name: DEFAULT_SEVERITY_FLOOR, rank: SEVERITY_SCALE.ranks[key] } +} +const SEVERITY_FLOOR = String(PARSED.severityFloor ?? '').trim() ? parseFloor(PARSED.severityFloor) : defaultFloor() // ── Isolation convention ─────────────────────────────────────────────────── // The AUTHORING chain (implement -> PR -> fix) runs inside a dedicated, PERSISTENT @@ -1048,20 +1097,28 @@ function baseOf(story) { return String(story.base ?? '').trim() || PIPELINE.baseBranch } -function wtClause(story) { +function wtClauseBase(story) { const base = baseOf(story) return `ISOLATION (mandatory): do ALL git/file work inside a dedicated worktree at \`${PIPELINE.worktreeRoot}/${story.id}\` — create-or-reuse it: \`git worktree add ${PIPELINE.worktreeRoot}/${story.id} -B ${story.branch} ${base}\` on first setup, or \`git worktree add ${PIPELINE.worktreeRoot}/${story.id} ${story.branch}\` if the branch already has commits; if the path already exists, just \`cd\` into it. NEVER modify the repo's main working tree and NEVER switch its branch.${base === PIPELINE.baseBranch ? '' : ` This story is STACKED on \`${base}\`: that branch is its base, so its commits are already in your history and must NOT be reverted, duplicated or re-implemented — only ADD your own work on top. When you open the PR, target \`${base}\` as the PR base branch, not \`main\`, so the diff shows only this story's change.`}` } +function wtClause(story) { + return `${wtClauseBase(story)} ${FINITE_STATE_COMPLETENESS}` +} + // Reviewer isolation: read-only inspection in a DETACHED throwaway worktree pinned // to the PR's pushed head. Detached HEAD never occupies the branch, so it can't // collide with the authoring worktree (which holds it) or with other stories' // reviewers in a parallel batch — and it never touches the main checkout's branch. -function revWtClause(story) { +function revWtClauseBase(story) { const p = `${PIPELINE.worktreeRoot}/${story.id}-review` return `ISOLATION (mandatory, read-only): NEVER switch the main checkout's branch. Inspect the code in a DETACHED throwaway worktree pinned to the PR's current pushed head: \`git worktree remove --force ${p} 2>/dev/null; git fetch origin -q; git worktree add --detach ${p} origin/${story.branch}\`, then \`cd ${p}\`. Read the code there (the untracked checkpoint is absent here — good, stay blind to it). When finished, remove it: \`git worktree remove --force ${p}\`.` } +function revWtClause(story) { + return `${revWtClauseBase(story)} ${CONTRACT_INVENTORY}` +} + // #373 finding 3: the escalate-flush shared block — supersede-the-prior-flush + the manual // out-of-band CONVENTION + the untracked-worktree-persistence note — is identical across BOTH // escalation prompts (MAX_FIX_ROUNDS + needsHumanDecision). Authored ONCE here so a future @@ -1216,6 +1273,7 @@ async function driveStory(story) { // before honouring it, so the escalation is deferred by a round rather than dropped. let humanDecisionPending = false let prevFindings = [] + let prevReviewedHead = null // ACCUMULATES across rounds — never reassigned. A finding accepted in round 0 (by-design, or // below the floor) is not re-raised by the round-1 reviewer, because round 1 only sees the // fixed code and has no memory of what the human was already told would be carried. So a @@ -1230,7 +1288,10 @@ async function driveStory(story) { const acceptedKeys = new Set() const accept = (findings) => { for (const f of findings) { - const key = `${f.location ?? ''}${f.description ?? ''}` + // Keep a collision-free delimiter without embedding an invisible raw NUL in the shipped + // JavaScript source. A readable space collapses `(location, description)` pairs such as + // (`"a b"`, `"c"`) and (`"a"`, `"b c"`), silently dropping one accepted finding. + const key = `${f.location ?? ''}\u0000${f.description ?? ''}` if (acceptedKeys.has(key)) continue acceptedKeys.add(key) accepted.push(f) @@ -1249,10 +1310,14 @@ async function driveStory(story) { // in-flight log AND no first-review comment already on the PR. Either signal makes // round-0 a SILENT re-review, so a PR never accrues a second first-review. const first = round === 0 && !isContinuation && !firstReviewPosted + // An initial/resumed-without-history review establishes the whole-PR baseline. + // Once a fix is in flight, even the file inventory must start at that baseline; + // otherwise the pacing loop invites a second full audit before its delta rule. + const reviewBase = prevFindings.length ? prevReviewedHead : baseOf(story) const review = await agentRetry( - `Independently review PR #${pr.prNumber} for story ${tag}, following ${SK.review}. ${revWtClause(story)} PACING (mandatory — this is what killed the previous four attempts at this review, measured): a supervisor kills any agent that goes 180 seconds without emitting a TEXT MESSAGE. Tool calls do NOT count as progress: the last stalled reviewer was calling \`sed\`/\`cat\` every ~5 seconds and was still killed, because it had not written a sentence in 200 seconds. So: after EVERY file you inspect, write ONE SHORT LINE of prose saying what you found or that it is clean — before moving to the next file. Never read two files in a row without speaking in between, and never go into a long silent analysis pass. Start by listing the changed files (\`git diff ${baseOf(story)}...origin/${story.branch} --name-only\`), say aloud the order you will take them, then go file by file, narrating as you go. Brevity is fine — one line is enough — but silence is fatal. Review ONLY from the story's acceptance criteria, the PR diff+description, and the code. Do NOT read ${BLIND_PATHS}, nor any checkpoint, handoff or working log under them — they are the author's private context and this review is independent and blind to it. Report EVERY finding regardless of severity (including minor/nit), using the ${REVIEW_TEMPLATE_LABEL} vocabulary: each finding = \`location\` (File:Line), \`severity\` ∈ {${SEVERITIES}}, \`description\` (the CONCRETE FAILURE CASE — inputs/state -> wrong output — not a retelling of the diff), \`recommendation\` (the change, in one or two lines); verdict ∈ {${VERDICTS}}. ${TEXT_SHAPE} DO NOT FILE NEW ISSUES. This is a hard rule, and it overrides any habit of deferring work to a follow-up card: a debt you find in this diff is resolved IN PLACE, in this same PR, within this story's scope. Never invoke ${SK.writeIssue}, never write \`Deferred to #\`, and never recommend "track this separately" — a finding parked in a fresh card is a finding nobody fixes, and it converts a reviewed PR into an unreviewed backlog. Set \`nonActionable: true\` ONLY if fixing it would be genuinely WRONG — byte-consistent with a source of truth, matching an existing convention, an ALREADY-EXISTING tracked story (cite its number; do not create one), or something that can only resolve after merge. Being outside this story's originally stated scope is NOT a reason: fix it here. Whenever you set \`nonActionable: true\`, ALSO set \`disposition\` with a concrete reason replacing the bare label (\`By convention …\` / \`Historical record\` / \`Already tracked in #\` / \`Resolves after merge\`); never leave "non-actionable" as the only explanation. If a finding is SO large that fixing it here would genuinely swamp the story, say so explicitly in \`description\` and leave it ACTIONABLE — the human decides at the merge gate whether to accept the bigger PR or carve it out; that decision is not yours to pre-empt by filing a card. ${first ? `This is the FIRST review: POST your full review report as a PR comment on #${pr.prNumber} (${REVIEW_TEMPLATE_LABEL} structure), and include the marker line \`${firstReviewMarker}\` VERBATIM as the first line of the comment body — it is an HTML comment (invisible in the rendered markdown, so no visible noise) that lets a later resume detect this first review by an EXACT substring match rather than a semantic reading (finding 1). Then return findings + verdict.` : prevFindings.length - ? `This is a RE-REVIEW: do NOT post any PR comment (the orchestrator synthesizes the cycle at the end). Return findings + verdict only. Verify these prior findings were genuinely resolved: ${JSON.stringify(prevFindings)}.` - : `This is a RE-REVIEW on a resumed in-flight cycle (round-0 of this run carries no prior findings): do a FRESH, independent full review pass. do NOT post any PR comment (the orchestrator synthesizes the cycle at the end). Return findings + verdict only.`} Return findings and a verdict.`, + `Independently review PR #${pr.prNumber} for story ${tag}, following ${SK.review}. ${revWtClause(story)} PACING (mandatory — this is what killed the previous four attempts at this review, measured): a supervisor kills any agent that goes 180 seconds without emitting a TEXT MESSAGE. Tool calls do NOT count as progress: the last stalled reviewer was calling \`sed\`/\`cat\` every ~5 seconds and was still killed, because it had not written a sentence in 200 seconds. So: after EVERY file you inspect, write ONE SHORT LINE of prose saying what you found or that it is clean — before moving to the next file. Never read two files in a row without speaking in between, and never go into a long silent analysis pass. Start by listing the changed files (\`git diff ${reviewBase}...origin/${story.branch} --name-only\`), say aloud the order you will take them, then go file by file, narrating as you go. Brevity is fine — one line is enough — but silence is fatal. Review ONLY from the story's acceptance criteria, the PR diff+description, and the code. Do NOT read ${BLIND_PATHS}, nor any checkpoint, handoff or working log under them — they are the author's private context and this review is independent and blind to it. Report EVERY finding regardless of severity (including minor/nit), using the ${REVIEW_TEMPLATE_LABEL} vocabulary: each finding = \`location\` (File:Line), \`severity\` ∈ {${SEVERITIES}}, \`description\` (the CONCRETE FAILURE CASE — inputs/state -> wrong output — not a retelling of the diff), \`recommendation\` (the change, in one or two lines); verdict ∈ {${VERDICTS}}. ${TEXT_SHAPE} DO NOT FILE NEW ISSUES. This is a hard rule, and it overrides any habit of deferring work to a follow-up card: a debt you find in this diff is resolved IN PLACE, in this same PR, within this story's scope. Never invoke ${SK.writeIssue}, never write \`Deferred to #\`, and never recommend "track this separately" — a finding parked in a fresh card is a finding nobody fixes, and it converts a reviewed PR into an unreviewed backlog. Set \`nonActionable: true\` ONLY if fixing it would be genuinely WRONG — byte-consistent with a source of truth, matching an existing convention, an ALREADY-EXISTING tracked story (cite its number; do not create one), or something that can only resolve after merge. Being outside this story's originally stated scope is NOT a reason: fix it here. Whenever you set \`nonActionable: true\`, ALSO set \`disposition\` with a concrete reason replacing the bare label (\`By convention …\` / \`Historical record\` / \`Already tracked in #\` / \`Resolves after merge\`); never leave "non-actionable" as the only explanation. If a finding is SO large that fixing it here would genuinely swamp the story, say so explicitly in \`description\` and leave it ACTIONABLE — the human decides at the merge gate whether to accept the bigger PR or carve it out; that decision is not yours to pre-empt by filing a card. ${first ? `This is the FIRST review: POST your full review report as a PR comment on #${pr.prNumber} (${REVIEW_TEMPLATE_LABEL} structure), and include the marker line \`${firstReviewMarker}\` VERBATIM as the first line of the comment body — it is an HTML comment (invisible in the rendered markdown, so no visible noise) that lets a later resume detect this first review by an EXACT substring match rather than a semantic reading (finding 1). Then return findings + verdict.` : prevFindings.length + ? `This is a RE-REVIEW: do NOT post any PR comment (the orchestrator synthesizes the cycle at the end). Verify these prior findings were genuinely resolved: ${JSON.stringify(prevFindings)}. The last complete review covered immutable head ${prevReviewedHead}. First inspect ONLY the fix delta with \`git diff ${prevReviewedHead}...origin/${story.branch} --name-status\`, then its directly changed producer/consumer contract boundaries. Do NOT re-audit the unchanged PR surface. A new finding is actionable only if it is in this delta or a contract boundary changed by this delta; otherwise report it as a Question for the human, not a new fix round.` + : `This is a RE-REVIEW on a resumed in-flight cycle (round-0 of this run carries no prior findings): do a FRESH, independent full review pass. do NOT post any PR comment (the orchestrator synthesizes the cycle at the end).`} Return findings, verdict, and \`reviewedHead\`: the lower-case 40-character SHA printed by \`git rev-parse origin/${story.branch}\` after your inspection.`, // effort was 'xhigh'. The measured cause of the repeated kills was NOT effort and NOT a // stuck command: transcript timing showed the reviewer issuing a tool call every ~5s // (97 events, mean gap 4.9s, max 49s — zero gaps over 180s) yet still killed, because @@ -1263,10 +1328,9 @@ async function driveStory(story) { // narration reliable, restoring 'xhigh' is legitimate: it costs review depth, which is // the whole point of this gate. Do not read this line as "xhigh causes stalls". withModel({ agentType: 'pair-reviewer', phase: 'Review', label: `rev:${tag} r${round}`, effort: 'high', schema: REVIEW_SCHEMA }), - // A review is USABLE only if it carries a verdict — the same predicate the guard below - // converges on. Without it the retry covered the dead reviewer (`null`) and skipped the - // contentless one (`{}`), which is the shape actually measured on #432. - hasVerdict, + // A review is USABLE only with a verdict and its immutable reviewed head. Without the + // latter, the next pass cannot be an evidence-bounded re-review. + hasReviewEvidence, ) // A DEAD reviewer is not a clean review. `agent()` returns null when the subagent // dies, and `review?.findings ?? []` then yields zero findings — which the @@ -1286,15 +1350,16 @@ async function driveStory(story) { // So the test is inverted: a VERDICT must be present. Absence of findings is not evidence // that a review happened; presence of a verdict is. Every real review emits one — it is a // required field of the contract schema — so this costs a genuine clean review nothing. - // `hasVerdict` is the SAME function `agentRetry` was given above: the contentless return is - // retried once like any other dead step, and only then does it land here. - if (!hasVerdict(review)) + // `hasReviewEvidence` is the SAME function `agentRetry` was given above: a contentless or + // unanchored return is retried once like any other dead step, then lands here. + if (!hasReviewEvidence(review)) // `acceptedFindings` travels on EVERY terminal arm, this one included. A card whose // reviewer dies mid-cycle otherwise reports the by-design and below-floor findings of // every earlier round as if none had been raised — and those are precisely the findings // the fixer never receives, so they are recoverable from nowhere else. AC4 says an // accepted finding always reaches the human; a failure is not an exception to that. return { story, prNumber: pr.prNumber, status: 'failed-review', round, acceptedFindings: accepted, reviewLog: cycleHasRemediation ? reviewLog : undefined } + const reviewedHead = String(review.reviewedHead).toLowerCase() const findings = review.findings ?? [] const allActionable = findings.filter((f) => !f.nonActionable) // Below the floor: still reported, still shown to the human, just not blocking. Marked @@ -1357,11 +1422,12 @@ async function driveStory(story) { round++ prevFindings = actionable + prevReviewedHead = reviewedHead cycleHasRemediation = true // FIX — implementer resumes checkpoint (if present) + resolves actionable findings. // Logs the round to the working review log INSTEAD of posting a per-round PR comment. const fix = await agentRetry( - `Resume story ${tag}. ${wtClause(story)} Read the checkpoint if present (${SK.checkpoint} $mode=resume); otherwise work from the PR diff + code. Resolve EVERY one of these actionable review findings on PR #${pr.prNumber} — including minor/nit, do not defer any: ${JSON.stringify(prevFindings)}. Fix them IN PLACE, in this PR: do NOT file a follow-up issue for any of them, do NOT invoke ${SK.writeIssue}, and do NOT leave a "tracked separately" note in lieu of the fix. If a finding turns out to be genuinely larger than this story, still fix what belongs here and say plainly in the working log what remains — the human decides at the merge gate, not a new card. Follow ${SK.implement} for the change itself (test-first where a finding describes a defect), verify with ${SK.verifyQuality} (tier-resolved — do not improvise a gate command), and record any decision a finding forces with ${SK.recordDecision}. Commit and push. Then re-invoke **${SK.publishPr}**: it is create-or-update and idempotent, and re-running it is what keeps the PR body, the classification tags and the \`pr-state:*\` label in sync with the NEW head commit instead of describing the pre-fix state. As in the open-PR step it will emit \`Review: review-dispatch-required\` rather than nesting — expected: this orchestrator drives the re-review. ${TEXT_SHAPE} Re-running it REWRITES the PR body, and this is the only step that does so once a cycle is under way: rewrite it to describe the CURRENT head, do not append a round-by-round history — a body that grows by one section per fix round is re-read in full by every later reviewer of this same cycle. Do NOT post a remediation PR comment; INSTEAD append this round to the working log \`${reviewLog}\` (create it if absent) as a COMPACT TABLE under a \`## Round N\` heading — one row per finding, columns \`severity | location | what changed | commit\`. One row, one line: no paragraph per finding, and do not restate the finding's description (its location identifies it). Add prose ONLY where a fix diverged from the recommendation, and then only the reason. Only for a genuine design disagreement set needsHumanDecision instead of forcing a fix. Do NOT merge.`, + `Resume story ${tag}. ${wtClause(story)} Read the checkpoint if present (${SK.checkpoint} $mode=resume); otherwise work from the PR diff + code. Resolve EVERY one of these actionable review findings on PR #${pr.prNumber} — including minor/nit, do not defer any: ${JSON.stringify(prevFindings)}. Fix them IN PLACE, in this PR: do NOT file a follow-up issue for any of them, do NOT invoke ${SK.writeIssue}, and do NOT leave a "tracked separately" note in lieu of the fix. If a finding turns out to be genuinely larger than this story, still fix what belongs here and say plainly in the working log what remains — the human decides at the merge gate, not a new card. CONVERGENCE SWEEP (mandatory): the finding location is the starting point, not the contract boundary. Before changing code, make a finite map of the same observable contract: the reported case and its paired success/failure path; any state transition or resume path the contract owns; and the canonical source plus every distributed representation of that behavior (generated asset, dataset, installed copy, or documented command). Change every map cell required for that one contract, then stop — do not use the sweep for unrelated cleanup, new behavior, or speculative hardening. For a generated/distributed artifact, resolve the canonical source from the asset registry, edit only that source, then run the declared generator/installer and inspect its output; never hand-edit a derived copy. PROVISIONED ARTIFACT CONTRACT (mandatory when a change installs, builds, publishes, names, or invokes an executable/package): map \`producer -> published identity -> consumer\` — for example installer/release step -> package manifest/bin/file/export -> workflow or user command. Prove the exact path in a clean temporary environment using the real built or installed artifact. Never stub, alias, or fake the exact producer, published identity, or consumer boundary; external effects may be isolated only after that boundary is crossed. For each logic defect, write a test that executes the real function/script against a real or realistic fixture and asserts output/side effects, never a source-string regex. Re-run the finding's evidence command and the mapped boundary cases before commit. Follow ${SK.implement} for the change itself: its TDD discipline and adoption-compliance phase are mandatory. Verify with ${SK.verifyQuality} (tier-resolved — do not improvise a gate command), and record any decision a finding forces with ${SK.recordDecision}. Commit and push. Then re-invoke **${SK.publishPr}**: it is create-or-update and idempotent, and re-running it is what keeps the PR body, the classification tags and the \`pr-state:*\` label in sync with the NEW head commit instead of describing the pre-fix state. As in the open-PR step it will emit \`Review: review-dispatch-required\` rather than nesting — expected: this orchestrator drives the re-review. ${TEXT_SHAPE} Re-running it REWRITES the PR body, and this is the only step that does so once a cycle is under way: rewrite it to describe the CURRENT head, do not append a round-by-round history — a body that grows by one section per fix round is re-read in full by every later reviewer of this same cycle. Do NOT post a remediation PR comment; INSTEAD append this round to the working log \`${reviewLog}\` (create it if absent) as a COMPACT TABLE under a \`## Round N\` heading — one row per finding, columns \`severity | location | what changed | commit\`. One row, one line: no paragraph per finding, and do not restate the finding's description (its location identifies it). Add prose ONLY where a fix diverged from the recommendation, and then only the reason. Only for a genuine design disagreement set needsHumanDecision instead of forcing a fix. Do NOT merge.`, withModel({ agentType: 'pair-implementer', phase: 'Review', label: `fix:${tag} r${round}`, effort: 'high', schema: FIX_SCHEMA }), ) // failed-fix: the fixer died mid-round; a partial working log may exist. Surface diff --git a/.claude/workflows/pair-implement-batch.test.mjs b/.claude/workflows/pair-implement-batch.test.mjs index b3360499b..b75b2d8e5 100644 --- a/.claude/workflows/pair-implement-batch.test.mjs +++ b/.claude/workflows/pair-implement-batch.test.mjs @@ -25,12 +25,25 @@ const SRC = readFileSync(new URL('./pair-implement-batch.js', import.meta.url), '', ) const AsyncFunction = Object.getPrototypeOf(async () => {}).constructor +const REVIEWED_HEAD = 'a'.repeat(40) async function runWorkflow({ args, dispatch }) { const calls = [] const agent = async (prompt, opts) => { calls.push({ prompt, opts }) - return dispatch(prompt, opts) + const result = await dispatch(prompt, opts) + // A real reviewer now returns the immutable revision it reviewed. Keep legacy + // fixtures concise while allowing focused tests to provide an invalid/missing + // value explicitly. + if ( + opts.agentType === 'pair-reviewer' && + result && + typeof result === 'object' && + String(result.verdict ?? '').trim() && + result.reviewedHead === undefined + ) + return { ...result, reviewedHead: REVIEWED_HEAD } + return result } // Mirrors the real primitive's contract: "a thunk that throws (or whose agent errors) // resolves to null in the result array — the call itself never rejects". The earlier @@ -105,7 +118,14 @@ test('valid contract: reviewer schema derives from contract.json (AC1) and cache dispatch: stdDispatch({ contractResult: { status: 'cache-hit', contract } }), }) const rev = calls.find(c => c.opts.agentType === 'pair-reviewer') - assert.deepEqual(rev.opts.schema, contract.schema) + assert.deepEqual(rev.opts.schema, { + ...contract.schema, + properties: { + ...contract.schema.properties, + reviewedHead: { type: 'string', pattern: '^[0-9a-f]{40}$' }, + }, + required: ['verdict', 'reviewedHead'], + }) assert.ok(rev.prompt.includes('Blocker'), 'severity vocabulary threaded from the contract') assert.ok(rev.prompt.includes('Rework'), 'verdict vocabulary threaded from the contract') assert.deepEqual(result.contracts, [{ name: 'code-review', status: 'cache-hit' }]) @@ -212,8 +232,16 @@ test('contract with usable schema but missing canonical vocabulary keys: prompt dispatch: stdDispatch({ contractResult: { status: 'cache-hit', contract } }), }) const rev = calls.find(c => c.opts.agentType === 'pair-reviewer') - // Schema is still enum-locked from the (structurally usable) contract... - assert.deepEqual(rev.opts.schema, contract.schema) + // Schema is still enum-locked from the (structurally usable) contract, with + // the orchestration-owned reviewed revision layered on top. + assert.deepEqual(rev.opts.schema, { + ...contract.schema, + properties: { + ...contract.schema.properties, + reviewedHead: { type: 'string', pattern: '^[0-9a-f]{40}$' }, + }, + required: ['verdict', 'reviewedHead'], + }) // ...but the prompt vocabulary text falls back to the documented defaults, // since verdictOptions/severities (the canonical keys it's threaded from) // are absent. In practice ensure-contract.mjs's validateContract now rejects @@ -1002,6 +1030,129 @@ test('the fix step is likewise barred from deferring a finding into a new issue' ) }) +test('the fix step sweeps the bounded contract surface before re-review', async () => { + const finding = { location: 'x.ts:1', severity: 'Major', description: 'd', recommendation: 'r' } + let round = 0 + const { calls } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: (prompt, opts) => { + if (opts.agentType === 'pair-contract-generator') return { status: 'cache-hit', contract: validContract() } + if (opts.agentType === 'pair-reviewer') return round++ === 0 ? { verdict: 'Rework', findings: [finding] } : { verdict: 'Approved', findings: [] } + if (opts.phase === 'Implement') return { gatesPassed: true, branch: 'b' } + if (opts.phase === 'PR') return { prNumber: 7 } + return { fixed: true } + }, + }) + + const fix = calls.find(c => c.opts.label?.startsWith('fix:')).prompt + assert.match(fix, /CONVERGENCE SWEEP/, 'the fixer must make the bounded contract explicit') + assert.match(fix, /location is the starting point/i, 'a finding location is not the contract boundary') + assert.match(fix, /success\/failure/i, 'paired execution paths are checked together') + assert.match(fix, /every distributed representation/i, 'source and shipped representations are checked together') + assert.match(fix, /PROVISIONED ARTIFACT CONTRACT/, 'a provisioned command has an explicit end-to-end check') + assert.match(fix, /producer.*published identity.*consumer/i, 'the provisioner, artifact metadata and invocation are mapped together') + assert.match(fix, /clean temporary environment/i, 'the actual installed or built artifact is exercised') + assert.match(fix, /never stub.*boundary/i, 'a stub cannot stand in for the published command boundary') + assert.match(fix, /unrelated cleanup/i, 'the sweep stays bounded and is not scope creep') + assert.doesNotMatch(fix, /touch ONLY what each finding's location names/, 'line-only scope discipline would recreate the gap') +}) + +test('review and fix exhaust finite protocol states before another round', async () => { + const finding = { location: 'state.ts:1', severity: 'Major', description: 'd', recommendation: 'r' } + let round = 0 + const { calls } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: (prompt, opts) => { + if (opts.agentType === 'pair-contract-generator') return { status: 'cache-hit', contract: validContract() } + if (opts.agentType === 'pair-reviewer') return round++ === 0 ? { verdict: 'Rework', findings: [finding] } : { verdict: 'Approved', findings: [] } + if (opts.phase === 'Implement') return { gatesPassed: true, branch: 'b' } + if (opts.phase === 'PR') return { prNumber: 7 } + return { fixed: true } + }, + }) + const review = calls.find(c => c.opts.agentType === 'pair-reviewer').prompt + const fix = calls.find(c => c.opts.label?.startsWith('fix:')).prompt + assert.ok(review.includes('CONTRACT INVENTORY (mandatory)'), 'the reviewer inventories a contract before reporting its first hole') + assert.ok(review.includes('finite decision table of every supported state'), 'a finite protocol/state space is exhausted in the same review') + assert.ok(review.includes('AUTHORITATIVE BOUNDARY PROOF (mandatory)'), 'the reviewer must prove externally-defined state semantics at the real boundary') + assert.ok(fix.includes('FINITE-STATE COMPLETENESS (mandatory when'), 'the fixer must preserve that complete state model') + assert.ok(fix.includes('Do not implement one newly discovered row at a time'), 'the next re-review is not used to discover ordinary variants serially') + assert.ok(fix.includes('A unit test of the function being changed cannot establish external semantics'), 'the fixer cannot infer external-tool behavior from its own unit tests') +}) + +test('re-review is anchored to the reviewed revision and checks only the fix delta plus prior findings', async () => { + const finding = { location: 'workflow.yml:4', severity: 'Major', description: 'd', recommendation: 'r' } + let round = 0 + const { calls } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: (prompt, opts) => { + if (opts.agentType === 'pair-contract-generator') return { status: 'cache-hit', contract: validContract() } + if (opts.agentType === 'pair-reviewer') + return round++ === 0 + ? { verdict: 'Rework', findings: [finding] } + : { verdict: 'Approved', findings: [] } + if (opts.phase === 'Implement') return { gatesPassed: true, branch: 'b' } + if (opts.phase === 'PR') return { prNumber: 7 } + return { fixed: true } + }, + }) + + const reviews = calls.filter(c => c.opts.agentType === 'pair-reviewer') + assert.match(reviews[0].prompt, /reviewedHead/i, 'every review returns the immutable head it covered') + assert.match(reviews[1].prompt, new RegExp(`git diff ${REVIEWED_HEAD}\\.\\.\\.origin/feat/#292-x --name-only`), 're-review inventories the fix delta, not the entire PR') + assert.match(reviews[1].prompt, new RegExp(`git diff ${REVIEWED_HEAD}\\.\\.\\.origin/feat/#292-x`), 're-review starts from the previous review baseline') + assert.match(reviews[1].prompt, /only if it is in this delta or a contract boundary changed by this delta/i, 'unchanged PR surface is not repeatedly re-audited') +}) + +test('a review without an immutable baseline cannot converge', async () => { + const { result, calls } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: (prompt, opts) => { + if (opts.agentType === 'pair-contract-generator') return { status: 'cache-hit', contract: validContract() } + if (opts.agentType === 'pair-reviewer') return { verdict: 'Approved', findings: [], reviewedHead: 'not-a-sha' } + if (opts.phase === 'Implement') return { gatesPassed: true, branch: 'b' } + if (opts.phase === 'PR') return { prNumber: 7 } + return { fixed: true } + }, + }) + + assert.equal(result.batch[0].status, 'failed-review') + assert.equal(calls.filter(c => c.opts.agentType === 'pair-reviewer').length, 2, 'missing review evidence is retried once') +}) + +test('a review baseline must be lower-case like the review contract declares', async () => { + const { result } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: (prompt, opts) => { + if (opts.agentType === 'pair-contract-generator') return { status: 'cache-hit', contract: validContract() } + if (opts.agentType === 'pair-reviewer') return { verdict: 'Approved', findings: [], reviewedHead: 'A'.repeat(40) } + if (opts.phase === 'Implement') return { gatesPassed: true, branch: 'b' } + if (opts.phase === 'PR') return { prNumber: 7 } + return { fixed: true } + }, + }) + + assert.equal(result.batch[0].status, 'failed-review') +}) + +test('accepted-findings key is collision-free for location and description pairs', async () => { + const { result } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: stdDispatch({ + contractResult: { status: 'cache-hit', contract: validContract() }, + review: { + verdict: 'Approved', + findings: [ + { location: 'a b', severity: 'Minor', description: 'c', nonActionable: true }, + { location: 'a', severity: 'Minor', description: 'b c', nonActionable: true }, + ], + }, + }), + }) + + assert.equal(result.batch[0].acceptedFindings.length, 2) +}) + // ── A run that drove nothing must not report success ─────────────────────── // Observed: two workflows were launched concurrently on a saturated machine, every // implementer stalled past the supervisor's window, `parallel` returned six nulls, @@ -1247,6 +1398,68 @@ test('an unknown severity blocks regardless of the floor (fail safe), and a bad ) }) +// ── The DEFAULT floor ────────────────────────────────────────────────────── +// Measured across three cycles on PR #477: the PR reached APPROVED with zero actionable +// findings, the next round implemented review Questions the reviewer had marked "No change +// requested", and the re-review found new Minors INSIDE the code that round added (three the +// first time, two the second). Questions are, by the review template's own definition, +// questions for the human — putting them in the fix set contradicts what they are. The floor +// therefore defaults to `Minor`: Major and Minor block and drive fix rounds, everything below +// is carried to the merge gate. A caller can still pass a floor explicitly to override it. +function contractWithQuestions() { + const c = validContract() + c.vocabulary.severities = ['Blocker', 'Major', 'Minor', 'Questions'] + c.severityRanks = { Blocker: 4, Major: 3, Minor: 2, Questions: 1 } + return c +} +const QUESTION = { location: 'c.ts:3', severity: 'Questions', description: 'is this intended?', recommendation: 'no change requested' } + +test('by default a Questions-only review converges: carried to the gate, never fixed', async () => { + const { result, calls } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: stdDispatch({ + contractResult: { status: 'cache-hit', contract: contractWithQuestions() }, + review: { verdict: 'Rework', findings: [QUESTION, { ...QUESTION, location: 'c.ts:9' }] }, + }), + }) + const b = result.batch[0] + assert.equal(b.status, 'ready-for-merge', 'Questions alone do not keep the loop open') + assert.equal(b.acceptedFindings.length, 2, 'both Questions reach the human') + assert.match(b.acceptedFindings[0].disposition, /Below severity floor \(Minor\)/) + assert.ok(!calls.some(c => c.opts.label?.startsWith('fix:')), 'no fix round is spent on a Question') +}) + +test('by default a Minor still blocks and still drives a fix round', async () => { + let round = 0 + const { result, calls } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: (prompt, opts) => { + if (opts.agentType === 'pair-contract-generator') return { status: 'cache-hit', contract: contractWithQuestions() } + if (opts.agentType === 'pair-reviewer') + return round++ === 0 ? { verdict: 'Rework', findings: [MINOR, QUESTION] } : { verdict: 'Approved', findings: [QUESTION] } + if (opts.phase === 'Implement') return { gatesPassed: true, branch: 'b' } + if (opts.phase === 'PR') return { prNumber: 7 } + return { fixed: true } + }, + }) + assert.equal(result.batch[0].status, 'ready-for-merge') + assert.ok(calls.some(c => c.opts.label?.startsWith('fix:')), 'the Minor drove a fix round') +}) + +test('a vocabulary without Minor falls back to no floor instead of throwing', async () => { + const c = validContract() + c.vocabulary.severities = ['Severe', 'Trivial'] + c.severityRanks = { Severe: 2, Trivial: 1 } + const { result } = await runWorkflow({ + args: { stories: [STORY] }, + dispatch: stdDispatch({ + contractResult: { status: 'cache-hit', contract: c }, + review: { verdict: 'Rework', findings: [{ location: 'x:1', severity: 'Trivial', description: 'd', recommendation: 'r' }] }, + }), + }) + assert.equal(result.batch[0].status, 'escalate', 'without a usable default every actionable finding blocks, as before') +}) + // ── Options must survive a JSON-string `args` ─────────────────────────────── // Real bug: the runtime can hand this script `args` as a JSON STRING. parseBatchArgs // normalized it, but severityFloor was read off the RAW value, where diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml new file mode 100644 index 000000000..1bfc1fff8 --- /dev/null +++ b/.github/workflows/format.yml @@ -0,0 +1,114 @@ +# Formatting gate (story #413). +# +# Since #394 the pre-push hook CHECKS formatting instead of applying it — right +# shape for a hook, but it left the LOCAL HOOK AS THE ONLY ENFORCEMENT POINT: a +# push with `--no-verify`, or a contributor whose hooks are not installed, landed +# unformatted code with every CI check green. This workflow runs the identical +# `pnpm format:check` on every PR and on push to `main`. +# +# A DEDICATED WORKFLOW, not a job in ci.yml: that workflow carries +# `paths-ignore: ['.changeset/**']` at WORKFLOW level, which a job inherits — so a +# PR touching only `.changeset/**` would run no formatting check at all. A check +# with a hole in its TRIGGER reads as enforcement and is not. Its own workflow also +# means its own status context (`format`), separately requirable once branch +# protection is written. +# +# THE SHAPE OF THIS FILE IS ASSERTED, so a red `pnpm test` (or `pnpm +# gate:composition`) on an edit here is expected, not a flake: every trigger, filter, +# key, action, input, command and condition is an allow-list, read by +# packages/dev-tools/src/quality-gates/format-workflow-composition.ts. That guard +# PARSES this file with `yaml@2.8.2`, so the SPELLING is free (flow style, anchors, +# aliases, JSON steps, CRLF all resolve to the same document GitHub runs) and only +# the meaning is constrained. That module's header is the rule inventory and the +# reason for each rule (the exact checking command, the checkout's inputs, the +# workflow/job/step key allow-lists, the runner label below, the toolchain-only install +# shell, the say-only remedy, …); the per-block notes below say only what is not obvious +# from the YAML itself. + +name: Format + +on: + pull_request: + branches: + - main + push: + branches: + - main + # Manual runs on an arbitrary ref. Re-running the job from the Actions UI only + # replays the ref it already ran on; this is the escape hatch ci.yml already has. + workflow_dispatch: + +# SUPERSESSION, not de-duplication. Keyed on `github.ref` the two triggers never +# meet: a `pull_request` run is `refs/pull//merge`, a push to main is +# `refs/heads/main`, so they sit in different groups (and after a merge the PR is +# closed, so `pull_request` does not fire at all). What this buys is dropping a run +# whose ref has already moved on — push three commits to a PR branch, only the last +# verdict is worth a runner. +# +# Hence the CONDITIONAL cancel: two merges to main a minute apart share +# `format-refs/heads/main`, and an unconditional `true` would cancel the first +# commit's run, leaving that commit with no formatting verdict of its own. Main is +# linear, so the surviving run's tree still contains that commit and the drift is +# caught one commit later — this is the stronger of two acceptable spellings, not the +# only correct one (the guard allow-lists a bare `true` as the weaker). On main, runs +# queue. +concurrency: + group: format-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + format: + # GitHub-hosted, and asserted as such: `runs-on` picks the machine, which decides + # what `pnpm` and `prettier` are and who watches the run — the `container:` question + # spelled as a value. On a public repo a `pull_request` run executes the PR's own + # copy of this file, so `self-hosted` here would be a machine the PR chose. + runs-on: ubuntu-latest + # This job runs `pnpm install`, i.e. PR-authored lifecycle scripts. It takes the + # least token it can and a timeout that turns a hung install into a red instead + # of a six-hour default. + permissions: + contents: read + timeout-minutes: 10 + steps: + - name: Checkout code + uses: actions/checkout@v4 + - name: Install pnpm + uses: pnpm/action-setup@v4 + with: + version: '10.15.0' + - name: Ensure pnpm is available (fallback) + run: | + if ! command -v pnpm >/dev/null 2>&1; then + echo "pnpm not found in PATH; attempting to enable via corepack" + corepack enable || true + corepack prepare pnpm@10.15.0 --activate || true + fi + # Pinned to the same major ci.yml and release.yml pin — see the note on + # ci.yml's `Setup Node.js` step. Raising this means raising those together. + - name: Setup Node.js + uses: actions/setup-node@v4 + with: + node-version: '20' + cache: 'pnpm' + - name: Install dependencies + run: pnpm install + # The identical script the pre-push hook and a developer run. It names the + # offending files and exits non-zero. + - name: Check formatting + id: format_check + run: pnpm format:check + # …but it does NOT name the remedy: `--list-different` prints the paths and + # suppresses prettier's own "Run Prettier with --write to fix" line. The + # contributor this workflow exists for (hooks not installed, or pushed with + # `--no-verify`) would otherwise read a bare filename and no instruction. The + # annotation surfaces on the PR's Checks tab, not only in the log. + # + # SCOPED to the check's own outcome, not a bare `if: failure()` — that is + # JOB-scoped and fires for ANY earlier failure. A `pnpm install` broken by a + # lockfile drift or a registry outage would otherwise be annotated "not + # formatted, run 'pnpm format'": the contributor runs it, nothing changes, and + # the real cause sits under a confident wrong diagnosis. + - name: Explain how to fix a formatting failure + if: failure() && steps.format_check.outcome == 'failure' + run: | + echo "::error title=Formatting check failed::The files listed in the previous step are not formatted. Run 'pnpm format' locally, commit the result, and push again. CI never rewrites your branch." diff --git a/.gitleaks.toml b/.gitleaks.toml index 5a0a429a6..840e4db92 100644 --- a/.gitleaks.toml +++ b/.gitleaks.toml @@ -3,7 +3,9 @@ title = "pair secret-scanning allowlist" [extend] useDefault = true # keep gitleaks' built-in detection rules — this file only adds exceptions -[allowlist] +# Repo-wide exceptions: strings that are false positives WHEREVER they appear, because +# each is a documented, synthetic, illustrative value with no real counterpart. +[[allowlists]] description = "Project-specific false-positive exceptions, confirmed one at a time" regexes = [ '''AKIAABCDEFGHIJKLMNOP''', # documented worked example in secret-scanning.md/ADR-015 — a synthetic, illustrative AWS-key-shaped string, not a real credential @@ -11,3 +13,18 @@ regexes = [ '''sess_xyz789uvw012''', # placeholder sessionToken example in the same file — not a real credential '''aBcD1234efGH5678ijKL''', # synthetic token-shaped value in web-cloud-environment.test.ts (credential-shape fixtures) — not a real credential ] + +# PATH-SCOPED exception, `condition = "AND"` so BOTH the path and the value must match. +# A 40-hex string is a git object id only where it is a smoke-test fixture; the same +# literal anywhere else must still be scanned, so this allowance cannot travel. +# (Measured against gitleaks 8.30.1: with this block the fixture path is allowed and the +# same literal in `apps/` is still reported; scoping it as a `[[rules]] id = +# "generic-api-key"` override instead DISABLES the built-in rule outright — 2 findings +# became 0 on the same fixtures.) +[[allowlists]] +description = "smoke-test HEAD SHA fixture — a git object id, not a credential" +condition = "AND" +paths = ['''scripts/smoke-tests/.*'''] +regexes = [ + '''cc1fba122f0c912ba01288fe90ab2632e7e41057''', # synthetic 40-hex HEAD SHA fixture in scripts/smoke-tests (pr-state-flow.sh, github-pr-reviews.json); matched generic-api-key only through the `TOKEN_HEAD=` variable name at commit 5e99d538 — a git object id, not a credential +] diff --git a/.pair/adoption/decision-log/2026-08-28-task-progress-feedback-is-one-mechanism-owned-by-implement.md b/.pair/adoption/decision-log/2026-08-28-task-progress-feedback-is-one-mechanism-owned-by-implement.md new file mode 100644 index 000000000..cad2da9aa --- /dev/null +++ b/.pair/adoption/decision-log/2026-08-28-task-progress-feedback-is-one-mechanism-owned-by-implement.md @@ -0,0 +1,56 @@ +# Decision: The breakdown-to-task feedback loop is one mechanism owned by /pair-process-implement, batched per invocation + +## Date + +2026-08-28 + +## Status + +Active + +## Category + +Process Decision + +## Context + +Story #220 asked for progress visibility at task granularity during autonomous runs: as the AI advances through a story's inline task breakdown, the checklist item should be ticked and a progress comment posted (R4.6, spec G10). Two shapes were open, and both are load-bearing enough to record. + +**Where the mechanism lives.** The story names three surfaces — `/pair-process-implement` (#256, the insertion point), the supervisor loop (#219/#250), and `/pair-capability-write-issue` (the PM write patterns). The acceptance criteria require the manual and the supervised path to behave *identically* ("one mechanism"). A loop-side implementation would satisfy the supervised half and leave a human-invoked `/pair-process-implement` silent; two implementations would satisfy both and then drift. + +**How often it speaks.** A comment per task honours "task granularity" and produces exactly the comment feed D22's reading budget exists to prevent — on a 4-task story, four notifications that each say one line. A comment only at the end is quiet but says nothing while the run is in flight. + +**How the tick is written.** `/pair-capability-write-issue`'s write mode is a **full-body overwrite**; there is no partial-body write in the corpus. Ticking through it means the caller passes the current body with one line patched — the same read-merge contract `/pair-process-plan-tasks` is already held to. The alternative was a new tick-only mode on `/pair-capability-write-issue`. + +## Decision + +**One mechanism, owned by `/pair-process-implement`, specified once in the KB, flushed once per invocation.** + +1. **The mechanism is a guideline**, `knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md`: the task-ID locator, the tick-only diff-checked patch, the batch format, the outcome vocabulary and the fallbacks. The skills point at it and never restate it (D18 — one owner per rule). +2. **`/pair-process-implement` is the only caller.** A supervised run reaches the same code path through `/pair-loop` → `/implement-batch` → `/pair-process-implement`, so the two paths are identical *by construction*, not by two implementations kept in agreement. `/pair-loop` carries an explicit boundary forbidding a second per-task writer. +3. **The batch unit is one `/pair-process-implement` invocation over one story** — a manual session is one iteration; a supervised run contributes one per card per pass. The queue is flushed exactly once, at Step 3.1b, including on the way out of a HALT. An empty queue posts nothing. +4. **The transport is `/pair-capability-write-issue`**, declared an **Optional** composed skill: write mode for the tick (caller-patched full body, diff-checked before the call, **`$on-failure: report`**), `$mode: comment` for the batch. Optional is the point — a missing writer degrades the feedback and never blocks the story — and `$on-failure: report` is what makes a *present but failing* writer degrade the same way instead of HALTing the run through it. + +## Alternatives Considered + +- **Implement it in the supervisor loop (#219/#250)**: Rejected. It satisfies the supervised path only, and a human-invoked `/pair-process-implement` — the majority of runs today — stays silent. It also puts a merit-free orchestrator (D18) in the business of reading task outcomes. +- **A dedicated `/report-progress` capability skill**: Rejected. The whole mechanism is ~2 call sites and a document; a skill would add an invocation surface, a version and a degradation contract for logic that has exactly one caller. If a second caller ever appears, the guideline is already the shared owner and promoting it is mechanical. +- **One comment per task**: Rejected on D22. Four tasks become four notifications for the same story; the batched form carries the same four lines in one read. +- **One comment per story, at merge**: Rejected. It reports after the fact and says nothing while an unattended run is in flight, which is the visibility the story asked for. +- **A new tick-only mode on `/pair-capability-write-issue`**: Rejected for now. It duplicates the read-merge contract `/pair-process-plan-tasks` already lives under, and it would make `/pair-capability-write-issue` aware of the checklist grammar — a second owner of the locator rule. The caller-side diff check gives the same safety with no new mode. What the tick *does* need from the transport is not a mode but a **failure policy** — `$on-failure: report` (see Consequences), which changes who raises and nothing about what is written. +- **Persisting the progress queue across invocations** (writing it beside the checkpoint at Step 2.8 and letting a resumed invocation flush its predecessor's lines): Rejected for now. It closes a real gap (see Consequences) but re-opens the accretion the invocation scoping was introduced to close — the resumed run would post a catch-up comment for work a human may already have acted on — and it obliges every resumed invocation to reconcile a queue against work it may have redone. The window it would cover is bounded by one invocation and loses no state, only narrative. + +## Consequences + +- `/pair-process-implement` gains one optional composition and one closing step; its 5-step task cycle is unchanged. +- The tick's safety rests on a caller-side diff check (exactly one checkbox-only line changed, or the write is abandoned), because the transport cannot enforce it. This is stated in the guideline and in `/pair-capability-write-issue`'s composition interface, and asserted in conformance tests. +- Feedback failures are never blocking: a locator mismatch, a body edit race or a tracker error degrades to *comment-only* and the run continues. A story can therefore be complete with an unticked item — the comment names the ID so a human can tick it. +- **The transport must not HALT for the tick, and now does not.** `/pair-capability-write-issue` write mode HALTs on an unresolvable id (Step 7), a board membership its re-read cannot confirm (Step 7b beat 4 — which runs on *every* write-mode write on an explicit-membership tool, so every tick and every DoD box passes through it) and any tracker error (Step 8); a HALT in a composed skill propagates to its caller. Unaddressed, one 5xx while ticking task 1 of 4 would end the run — no T2/T3/T4, no PR — over a checkbox. The tick therefore passes **`$on-failure: report`**, added to that skill (Step 8b): those three become the returned outcomes `not-found`, `membership-unconfirmed`, `write-failed`. The exemption is deliberately narrow — an unsupported `$type`, a missing template and a malformed `state-mapping` still HALT for every caller, because they are deterministic caller/configuration bugs a report would only invite the caller to re-trigger. +- **A batch lost with its session is not recovered.** The queue is invocation-scoped (that is what makes a re-run silent), and it lives in session memory while the checkpoint is persisted after every task. An invocation that ticks T1/T2 and dies before the flush leaves those two lines — including any `write-failed`/`not-found` diagnostic a human would need in order to tick by hand — unposted on any invocation. **The ticks stand; the lines do not.** Accepted, not overlooked: the alternative is under Alternatives Considered. +- Cadence and verbosity are defaults, not law: a project that wants different ones declares them in its own adoption. + +## Adoption Impact + +- [way-of-working.md](../tech/way-of-working.md): no change required — the loop introduces no new gate, no new board state and no new PM tool obligation; the existing Assignment and PM-tool sections already cover the writes it makes. +- No dataset mirror for this ADL: sibling entries in `adoption/decision-log/` are adoption-only records, per the convention recorded in [2026-07-18-conformance-test-per-file-not-per-story.md](./2026-07-18-conformance-test-per-file-not-per-story.md). +- The *mechanism* is KB content and therefore ships: `packages/knowledge-hub/dataset/.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md` plus its generated `.pair/knowledge/**` mirror. diff --git a/.pair/adoption/decision-log/2026-08-31-review-baseline-and-provisioned-artifact-contract.md b/.pair/adoption/decision-log/2026-08-31-review-baseline-and-provisioned-artifact-contract.md new file mode 100644 index 000000000..79afd4533 --- /dev/null +++ b/.pair/adoption/decision-log/2026-08-31-review-baseline-and-provisioned-artifact-contract.md @@ -0,0 +1,58 @@ +# Decision: Review re-checks use an immutable baseline and prove provisioned artifacts + +## Date + +2026-08-31 + +## Status + +Active + +## Category + +Process Decision + +## Context + +PR #474 / story #217 ran on `89793d27`, which already required a convergence sweep. In round 4, +the fixer added a CLI installation step and retained a runner invocation, but tested a stub named +`pair` rather than the installed package's declared `pair-cli` bin. Round 5 therefore found the +new functional defect. The generic sweep named distributed representations but did not require an +end-to-end proof across installation, published identity, and invocation. Re-review also rescanned +the whole accumulated PR, so each fix expanded the next review surface. + +## Decision + +1. Every reviewer return includes the lower-case 40-character SHA it inspected (`reviewedHead`). + Missing or invalid evidence is retried once then fails closed; it never converges a PR. +2. The initial review remains complete. Each later re-review verifies prior findings plus + `git diff ...origin/` and directly changed producer/consumer boundaries. + A new blocking finding must come from that delta or boundary; an unchanged surface is not + re-audited as a new fix round. +3. A fix touching an installed, built, published, named, or invoked artifact maps + `producer -> published identity -> consumer` and proves it in a clean temporary environment + using the real artifact. The exact boundary cannot be stubbed, aliased, or faked. + +## Alternatives Considered + +- **Keep full-PR scans on each round**: rejected — a growing diff makes a re-review another + independent first review and creates unbounded new scope. +- **Accept findings after a fixed round cap**: rejected — it hides genuine defects rather than + bounding their cause. +- **Only strengthen source-string tests**: rejected — #217 passed such a test while the published + CLI contract was broken. + +## Consequences + +- Re-review is bounded without downgrading Major or Minor findings. +- A caller resuming an older cycle has one fresh full review to establish a new baseline. +- Workflow authors must keep dataset source and root mirror byte-identical; tests cover both. + +## Adoption Impact + +- `.pair/adoption/tech/way-of-working.md`: records the review baseline and provisioned-artifact + proof convention. +- `packages/knowledge-hub/dataset/.workflows/pair-implement-batch.js` and its installed mirror: + enforce the convention. +- Both dry-run copies of `pair-implement-batch.test.mjs`: pin the schema, bounded re-review, and + real-artifact prompt requirements. diff --git a/.pair/adoption/decision-log/2026-09-01-external-boundary-proof-prevents-false-equivalence.md b/.pair/adoption/decision-log/2026-09-01-external-boundary-proof-prevents-false-equivalence.md new file mode 100644 index 000000000..1fa09a35a --- /dev/null +++ b/.pair/adoption/decision-log/2026-09-01-external-boundary-proof-prevents-false-equivalence.md @@ -0,0 +1,54 @@ +# Decision: External boundary proof prevents false equivalence + +## Date + +2026-09-01 + +## Status + +Active + +## Category + +Process Decision + +## Context + +The finite-state inventory rule exposed #416's CRLF problem, but the fixer extended the repair to +bare CR by treating all carriage-return forms as equivalent. Its unit tests proved only the drift +checker’s classification. A minimal real Git probe showed the promised remedy was false: Git +normalizes CRLF to LF, but preserves a lone CR blob through checkout. The next review therefore +found a major regression: its advice forbade the only repair that actually worked. + +## Decision + +Whenever a decision-table row, equivalence, normalization or user-facing repair depends on an +external command, service, file format or runtime, identify its authoritative producer/consumer +and prove the claim with a minimal isolated end-to-end probe. Keep variants distinct until that +boundary demonstrates equivalence. Apply any proposed repair in the probe and verify its stated +postcondition. Unit tests of the changed function remain required but cannot substitute for this +boundary evidence. + +## Alternatives Considered + +- **Trust the implementation unit tests**: rejected — they cannot establish Git’s checkout + semantics or prove user guidance outside the function’s process. +- **Treat all syntactically similar inputs as one state**: rejected — the external producer can + distinguish them, as CRLF and lone CR demonstrate. +- **Defer external proof to re-review**: rejected — that makes the reviewer discover a repair + regression after the fixer has already committed it. + +## Consequences + +- A finite-state table may contain an external-boundary probe per row or equivalence class. +- Fix reports include command, observed result and postcondition for external repair claims. +- Re-review may still stop on increased findings; it remains the signal that the required proof + was absent or incorrect. + +## Adoption Impact + +- `.pair/adoption/tech/way-of-working.md`: extends Review Convergence with the boundary-proof + requirement. +- `packages/knowledge-hub/dataset/.workflows/pair-implement-batch.js` and installed mirror: + require reviewers and fixers to prove externally-defined semantics and repair advice. +- Both workflow test copies: pin the prompt requirement so it cannot silently disappear. diff --git a/.pair/adoption/decision-log/2026-09-01-quoted-echo-arguments-are-data-not-commands.md b/.pair/adoption/decision-log/2026-09-01-quoted-echo-arguments-are-data-not-commands.md new file mode 100644 index 000000000..df1b345a5 --- /dev/null +++ b/.pair/adoption/decision-log/2026-09-01-quoted-echo-arguments-are-data-not-commands.md @@ -0,0 +1,92 @@ +# Decision: a workflow guard reads a quoted `echo` argument as data, never as a command + +## Date + +2026-09-01 + +## Status + +Active + +## Category + +Convention Adoption + +## Context + +`.github/workflows/format.yml` (#413) has to satisfy two rules that name the same +literal: + +- **AC1** — a failing check must print the remedy, and the remedy is `pnpm format`. +- **AC6 / ADL 2026-07-31** — no step reachable from the gate may WRITE, and + `pnpm format` expands (through the root scripts) to `prettier:fix`, a writer. + +The write-mode scan is a text scan over `run:` blocks, expanded transitively against +the root `package.json` scripts, shared with the pre-push guard next door +(`findWriteModeFormatters` + `expandScriptReferences`). Measured on the shipped +guard before this decision: + +| `run:` text | guard verdict | +| --- | --- | +| `pnpm format:check \|\| { echo "... Run pnpm format ..."; exit 1; }` | `ok=false` — "reaches 1 step(s) that WRITE files: prettier:fix" | +| same line with the literal `pnpm format` removed from the message | `ok=true` | + +So the guard rejected the obvious spelling of the remedy, and the workflow shipped +with no remedy at all: the CI run for commit `888391cc` printed the offending path +(`packages/dev-tools/src/quality-gates/__fixture-t3/unformatted.ts`) and nothing a +contributor could act on. + +## Decision + +For the format-workflow guard, a **quoted argument to `echo`/`printf` is data** and +is removed from the text before the "what does this step run" scans (the +`format:check`-is-invoked rule and the write-mode rule). Everything else — the whole +of an unquoted command, and any other step — is still scanned as a command. + +The exemption is bounded by execution, not by punctuation: a quoted string +containing `$` or a backtick can still execute (`echo "$(prettier --write .)"` really +does write), so those quotes are **not** stripped. The `${{ … }}` injection scan +reads the raw, unstripped text on purpose — `echo "${{ github.event.pull_request.title }}"` +is the classic sink precisely because it is quoted and echoed. + +A separate rule then REQUIRES the remedy: at least one step with an `if:` naming +`failure()` must mention `pnpm format` (word-bounded, so `pnpm format:check` does not +satisfy it). + +## Alternatives Considered + +- **Word around it in the workflow** (say "run the repo formatter" and never spell + `pnpm format`): the message stops being copy-pasteable, which is the entire value of + naming the remedy, and the next author who writes the real command gets a confusing + gate failure. +- **Exempt the whole `Explain how to fix it` step by name**: a guard keyed on a step's + NAME is defeated by renaming the step, and it would exempt that step's real commands + too. +- **Parse the shell** (a real POSIX parser to separate words from operators): correct, + and out of proportion for a twenty-line workflow — the same trade-off already recorded + for `stripComments` in this module. +- **Apply the same relaxation to `pre-push-gate-composition`**: not done. That guard + scans `package.json` script bodies, where there is no equivalent need — a script does + not print a remedy — and widening a security-adjacent scan with no driving case is how + the two copies drift. + +## Consequences + +- The format workflow can print `pnpm format` on the failure path (AC1 satisfied) while + the write-mode ban stays enforced: `run: pnpm format` still fails the guard, and so + does `echo "$(prettier --write .)"`. +- A guard asymmetry now exists between the two modules in + `packages/dev-tools/src/quality-gates/`: the format-workflow guard strips inert quoted + messages, the pre-push guard does not. Deliberate, and stated here so a later reader + does not "fix" it by copying the relaxation across. +- Residual gap, same class as the one ADL 2026-07-31 records for the offender list: an + inert-looking quoted string that a future shell feature makes executable would be + skipped. Bounded by the `$`/backtick exclusion, which covers every substitution form + POSIX sh has. + +## Adoption Impact + +None. This is guard-internal scanning semantics for one module; it changes no adopted +tool, process or gate composition. `adoption/tech/way-of-working.md` already lists +`format` in the required-check inventory (#413) and needs no edit — the check's identity, +trigger and command are unchanged. diff --git a/.pair/adoption/decision-log/2026-09-01-repo-wide-guard-enforced-by-turbo-root-input.md b/.pair/adoption/decision-log/2026-09-01-repo-wide-guard-enforced-by-turbo-root-input.md new file mode 100644 index 000000000..24f073f5a --- /dev/null +++ b/.pair/adoption/decision-log/2026-09-01-repo-wide-guard-enforced-by-turbo-root-input.md @@ -0,0 +1,133 @@ +# Decision: a repo-wide guard is enforced by a `$TURBO_ROOT$` cache input, not necessarily by a thin CLI + root gate step + +## Date + +2026-09-01 + +## Status + +Active + +## Category + +Tooling Preference + +## Context + +Story #413 adds `format-workflow-composition` to `@pair/dev-tools`: a guard that reads +`.github/workflows/format.yml` and fails if that workflow ever declares `paths-ignore`, switches to +`pull_request_target`, drops the `push: main` trigger or the concurrency group, gains a secret, or +runs a write-mode formatter. Its input therefore lives **outside** the package that owns it. + +`way-of-working.md` § Quality Gates states the rule this fits: + +> **A guard whose only caller is a turbo task is not enforced.** `turbo ts:check test lint` are +> cacheable with package-scoped inputs, so a change OUTSIDE the guard's package replays a cached +> PASS and the guard never executes. A guard over repo-wide state therefore gets a thin CLI and a +> **root gate step** (`hygiene:check`, `smoke-modes:check`, `docs:staleness`, `skills:conformance`). + +Read literally, #413 owes the repo a `format-workflow:check` CLI, a root script, an entry in +`quality-gate` and a step in `ci.yml` — four wiring points for one twenty-line workflow. + +But the rule's **premise** ("cacheable with package-scoped inputs") stopped being universally true +in this repo. `turbo.json` already carries two entries — `@pair/knowledge-hub#test` and +`@pair/dev-tools#test` — that name repo-root paths as task `inputs` via `$TURBO_ROOT$` (turbo >= +2.1), precisely so a repo-wide artifact busts the cache of the package-local test that guards it. +`@pair/dev-tools#test` already lists `$TURBO_ROOT$/scripts/format-lib/**` for exactly this reason, +and `@pair/knowledge-hub#test` already lists `$TURBO_ROOT$/.github/workflows/**`. + +The two adoption statements disagree about the mechanism, not the goal. The goal is: **the guard +must actually execute when the thing it guards changes.** + +## Decision + +**Enforcement mechanism is chosen per guard by what invalidates it, not by a blanket "repo-wide ⇒ +CLI + root gate step" rule.** A guard over an artifact outside its package satisfies the rule when +its owning task declares that artifact as a `$TURBO_ROOT$` input — the cache can no longer serve a +stale PASS, which is the entire failure mode the CLI + root-step pattern was invented to avoid. + +For #413 this means: no new root script, no new `quality-gate` segment and no new `ci.yml` +step. The guard is a vitest file, and `@pair/dev-tools#test` / `#test:coverage` gain **two** +inputs: `$TURBO_ROOT$/.github/workflows/format.yml` and `$TURBO_ROOT$/package.json`. (The +first version of this decision also said "no `format-workflow:check` CLI"; see the Amendment +below — a thin CLI now runs under the EXISTING `gate:composition` segment, without changing +the reasoning above.) + +The second is not bookkeeping. Both guards in that folder resolve **script delegation** against the +root scripts — `checkThisRepoGate` parses `package.json` outright, and `checkFormatWorkflow` defaults +`rootScripts` to `readRootScripts()`, which is the only reason `pnpm format` in a workflow step +resolves to `prettier:fix` at all. MEASURED without it: `turbo run test --filter=@pair/dev-tools +--dry=json` hashed `7271baf2a672a276`, and rewriting the root `format:check` script to `pnpm +prettier:fix` — i.e. CI now WRITES — left the hash at `7271baf2a672a276`, a cached PASS with neither +guard running. With the entry the same mutation moves the hash and both guards fire. + +Two facts make this sufficient rather than merely lighter: + +1. **CI is cold on every run** — no remote cache, no `.turbo` restore in `ci.yml` — so `pnpm test` + already executes the guard on every PR. The turbo input closes the **local** false green (which + is also what the pre-push hook's `turbo test` sees), not a merge-gate hole. This is the same + analysis already recorded in `turbo.json`'s `@pair/knowledge-hub#test` note. +2. **The guarded file cannot change without the input firing** — `format.yml` is named exactly, so + there is no glob under- or over-reach to drift. + +The CLI + root-gate-step pattern stays mandatory for the case that produced it: a guard whose input +set cannot be expressed as task inputs, or whose failure must be diagnosable as its own CI status +context rather than inside `pnpm test`. + +## Amendment — 2026-09-02 (PR #477 review round 12) + +Story #413's AC6 reads: check-only holds in CI "guarded by `pnpm gate:composition`". As decided +above, the new guard was enforced through `pnpm test` only and `gate:composition` ran the +pre-push guard alone, so the AC's named mechanism was not literally true. Resolved by wiring the +thin CLI this ADL had declined: `@pair/dev-tools format-workflow:check` (a `main()` behind a +`require.main` guard in the same module — the ADR-014 shape `pre-push-gate-composition` uses), +run by the existing root `gate:composition` script beside `pre-push-gate:check`. Cost: one line +in `packages/dev-tools/package.json`, one `&&` in the root `gate:composition` — no new root +script, no new `quality-gate` segment, no new `ci.yml` step, so the "four wiring points" +objection above does not apply. What the decision keeps: the `$TURBO_ROOT$` input is still +what makes `pnpm test` (and the pre-push hook's `turbo test`) honest locally — the CLI is a +second enforcement point, not a replacement, and the per-guard rule ("choose by what +invalidates it") stands; a guard may use both. Smoke-tested: `pnpm gate:composition` exits 0 +on the shipped workflow and prints both guards' lines; with `with: ref: main` injected into +`format.yml`, `format-workflow:check` exits 1 naming the input. + +## Alternatives Considered + +- **Thin CLI + root script + `quality-gate` segment + dedicated `ci.yml` step** (the literal + reading of the way-of-working rule): four wiring points, a fifth root script and a second CI step + for a check already executed by `pnpm test` on a cold runner. It buys one real thing — a distinct + status context — which #413 does not need: the workflow it guards already has its own `format` + context, and a composition failure is a code failure, correctly reported by the test job. +- **Nothing at all — rely on `pnpm test` alone**: rejected. It is green in CI but stale locally, so + the developer who reintroduces `paths-ignore` gets a cached PASS on the pre-push hook and only + learns of it after pushing. That is the exact "guard that does not run" defect the way-of-working + rule names. +- **Add `$TURBO_ROOT$/.github/workflows/**` (the whole directory)** instead of the one file: + rejected as over-invalidation — every unrelated workflow edit would rerun the dev-tools suite, + and `@pair/knowledge-hub#test` already covers the directory for the conformance suite that + genuinely reads all of it. + +## Consequences + +- #413 ships two files plus two `turbo.json` input entries, plus (Amendment) one package script + and one `&&` in the existing `gate:composition`, instead of four wiring points; the root + `quality-gate` string and `ci.yml` are untouched by this story. +- A future guard in this repo must answer "what invalidates it?" before "does it need a CLI?" — + the way-of-working bullet now names both mechanisms, so the choice is explicit rather than + inferred from the older of two precedents. +- **Residual risk, accepted**: if CI ever gains a remote cache or a `.turbo` restore, the "CI is + cold" leg of this reasoning disappears, and every guard relying on task inputs starts depending + on those inputs being *correct* rather than merely helpful. The `$TURBO_ROOT$` entries are what + would keep it honest — which is why the exact-file spelling matters. +- The two `@pair/dev-tools` entries (`#test` and `#test:coverage`) must keep identical input lists, + the same hand-maintained duplication `turbo.json` already documents for the knowledge-hub pair. + +## Adoption Impact + +- `adoption/tech/way-of-working.md` § Quality Gates — the "A guard whose only caller is a turbo + task is not enforced" bullet gains the `$TURBO_ROOT$`-input mechanism as the second way to + satisfy it, and states when the CLI + root-gate-step form is still required. +- `turbo.json` — `@pair/dev-tools#test` and `@pair/dev-tools#test:coverage` declare + `$TURBO_ROOT$/.github/workflows/format.yml` and `$TURBO_ROOT$/package.json` (the change itself, + recorded here as the rationale). Trimming the `package.json` entry as "undocumented" reopens the + measured stale-cache false green above, for BOTH guards in that folder. diff --git a/.pair/adoption/decision-log/2026-09-01-review-contract-inventory-prevents-serial-findings.md b/.pair/adoption/decision-log/2026-09-01-review-contract-inventory-prevents-serial-findings.md new file mode 100644 index 000000000..ee73fa744 --- /dev/null +++ b/.pair/adoption/decision-log/2026-09-01-review-contract-inventory-prevents-serial-findings.md @@ -0,0 +1,58 @@ +# Decision: Review contract inventory prevents serial findings + +## Date + +2026-09-01 + +## Status + +Active + +## Category + +Process Decision + +## Context + +The bounded delta re-review correctly exposed defects introduced by fixes, but it found them one +ordinary state at a time. On #419, fixes moved from an already-dirty path, to an unsupported +`git status --porcelain` shape, to a silent skip of another normal shape. The reviewer had one +concrete failure case per round and the fixer repaired that example, not the finite input domain. +Stopping on a count increase is therefore useful evidence for human investigation, not a reason +to hide the increase. + +## Decision + +Before reporting or fixing a changed observable contract, the reviewer/fixer maps its authoritative +producer, inputs, consumers and distributed representations. For a finite protocol, parser, +configuration, state transition or command-output domain, it builds a decision table containing +every supported state and invalid/boundary pair, and probes/tests each row against real behavior. +The first review inventories the full PR surface; a re-review inventories only its fix delta and +directly changed boundary. A fixer may not implement one newly found normal row and wait for a +later re-review to reveal the next one. + +## Alternatives Considered + +- **Keep only the generic convergence sweep**: rejected — it names paired paths but did not make + a finite protocol domain explicit, so a scalar repro still drove a scalar fix. +- **Rescan the full PR on every re-review**: rejected — it reopens unchanged scope; the immutable + baseline/delta rule remains in force. +- **Ignore a rise in findings**: rejected — the monitor's stop is the deliberate signal that a + fix may have introduced a defect and requires investigation. + +## Consequences + +- The initial review may front-load more findings, but ordinary variants are no longer deferred to + later rounds. +- Fixes touching finite state domains carry a complete test matrix before code changes. +- A count increase remains a valid human stop condition; it now points to a specific missing + inventory rather than being treated as the convergence mechanism itself. + +## Adoption Impact + +- `.pair/adoption/tech/way-of-working.md`: records the inventory requirement under Review + Convergence. +- `packages/knowledge-hub/dataset/.workflows/pair-implement-batch.js` and its installed mirror: + require the inventory in reviewer/fixer prompts. +- Both dry-run copies of `pair-implement-batch.test.mjs`: pin the prompt path and finite-state + rule. diff --git a/.pair/adoption/decision-log/2026-09-01-workflow-guard-rejects-what-it-cannot-read.md b/.pair/adoption/decision-log/2026-09-01-workflow-guard-rejects-what-it-cannot-read.md new file mode 100644 index 000000000..98d70a9cf --- /dev/null +++ b/.pair/adoption/decision-log/2026-09-01-workflow-guard-rejects-what-it-cannot-read.md @@ -0,0 +1,310 @@ +# Decision: the format-workflow guard PARSES `format.yml` with `yaml@2.8.2`, and every rule is an allow-list over the parsed document + +## Date + +2026-09-01 (amended 2026-09-03) + +## Status + +Active (amended 2026-09-03 — the decision this file recorded on 2026-09-01, "the guard REJECTS +the YAML spellings it cannot read", is **superseded in place**: the parser migration it listed as +a costed follow-on is TAKEN, and **has landed in PR #477** — the module imports `yaml@2.8.2`, +`@pair/dev-tools` declares the catalog entry, the line reader and the four spelling-rejection rule +families are deleted, and the suite is migrated. The superseded text is kept below, marked as +superseded, not deleted. Filename unchanged deliberately — history, the way-of-working link and +the `.pair/llms.txt` entry all point at this path.) + +## Category + +Convention Adoption + +## Context + +`packages/dev-tools/src/quality-gates/format-workflow-composition.ts` asserts the shape of +`.github/workflows/format.yml`. Until this decision it did so with a hand-rolled, line-based +reader (`blockUnder` collects the lines indented deeper than a key; `listValueOf`, `scalarAt`, +`keysAt`, `stepsOf`, `withoutBlockScalars` read inside what it collects). That reader understands +a subset of YAML — and the subset moved, round by round, every time a reviewer measured it. + +The evidence below is kept in full: it is the justification for the decision, and it is what a +future reader needs in order not to re-litigate it. What changed on 2026-09-03 is only the +conclusion drawn from it. + +### Round 5 — the reader failed OPEN on flow mappings (mapping-KEY position) + +Every trigger rule failed **open**. A flow mapping sits entirely on its key's own line, so it +yields an EMPTY block; `listValueOf` then finds no key inside it and returns `null`, which for a +trigger filter means "no filter, therefore every value". Four one-line edits, all valid YAML +GitHub honours, left `checkFormatWorkflow` returning `ok=true` on the shipped workflow (measured, +PR #477 review round 5): + +- `pull_request: { branches: [main], paths-ignore: ['**/*.md'] }` — a markdown-only PR runs no + formatting check, asserted green by the guard whose entire reason for existing is that key; +- `pull_request: { branches: [release] }` — no PR targeting `main` is ever checked and the + `format` context never reports; +- `pull_request: { branches: [main], types: [closed] }` — the check runs only once the PR is + closed; +- `push: { branches: [release] }` — post-merge drift on `main` invisible (AC7). + +Anchors, aliases and merge keys (`pull_request: *filters`, `<<: *filters`, `- *step`) are the same +class: they relocate content the line reader cannot follow, with the same "absent ⇒ no filter" +reading, and an alias under `steps:` hides a whole step from the write-mode scan. + +### Round 6 — the same class in YAML's OTHER node position (sequence ITEM) + +The round-5 pass rejected unreadable spellings on mapping KEYS only. A step is a sequence ITEM: +`- { name: Fix, run: npx prettier --write . }` is a step GitHub executes, `stepsOf` accepts the +line as a step, and then both readers that look INSIDE a step want their key at line start +(`scalarAt(step, 'uses', …)`, `extractRunBlocks`) and find none — so the step was invisible to +`usesProblems` and to the write-mode scan at once. Measured on the shipped workflow, `ok=true` on +each of `- { name: Fix, run: npx prettier --write . }`, `- {run: prettier --write .}`, +`- { uses: creyD/prettier_action@v4 }`, `- { uses: stefanzweifel/git-auto-commit-action@v5 }` and +the JSON spelling; also on `- [a, b]`, `- &fixer run: …` and a bare `-` with the node on the next +line. Placed before the checking step, each rewrites the runner's checkout and `pnpm format:check` +passes on unformatted code with the `format` context green — the AC6 loss both of those rules +exist to prevent. + +### Rounds 12–14 — the reader failed CLOSED, five times, on CORRECT workflows + +Once the rejection bound was in place the failures inverted: the reader started reporting +**correct** workflows red, or red with the wrong cause. Each is a spelling GitHub resolves +identically to the shipped one, and each was measured, not argued: + +- **Round 12** — an **indentless** block sequence (`branches:\n- main`, `steps:\n- name: …`) is + block style, parses identically to the indented form (`yaml@2.8.2`, measured) and is honoured by + GitHub (probe run on PR #477); the reader reported the shipped workflow as "does not cover + `main` (no branch)" and "no failure-path step names the remedy". A filter-level alias + (`branches: *shared`) fell through to the list reader and was reported as "no branch" — the + wrong cause. +- **Round 13** (probe [run 33676806439](https://github.com/foomakers/pair/actions/runs/33676806439) + on PR #477) — a quoted `run: "pnpm format:check"` (red, "runs `\"pnpm format:check\"`"); the file + with CRLF line endings (red with the wrong cause, "spells `on:` as a list of events"); + `permissions:` at workflow level with none on the job (red, "declares no `permissions:`" — the + token log shows the job inherits the workflow scope, and a job's own block replaces it). +- **Round 14** — `isSetupCommand` did not `unquote` while `normalizeCommand`, added in the same + round-13 commit, did: `run: "pnpm install"` on the shipped file went red with the "toolchain + install" cause. A correct workflow turned red by the guard's own previous round. + +Each was patched in place, one line at a time, because each finding's own recommendation offered +the patch as the first option. That is nine measured instances across five rounds, in **both** +directions — the reader is silently permissive on spellings it cannot see, and actively wrong on +spellings it half-sees. The property that keeps recurring is not any one rule: it is that the +guard re-implements a YAML reader, and the re-implementation is never finished. + +ADL [2026-07-29](./2026-07-29-yaml-parser-for-generated-yaml-tests.md) already adopted `yaml@2.8.2` +and already rejected hand-rolled YAML checkers, with an argument that lands squarely here: "a +parser written to the same (mis)understanding as the generator validates nothing". The same +sentence, read for a guard rather than a generator: a reader written to the same understanding as +the rule set asserts only what its author already imagined. + +## Decision + +**The guard PARSES `.github/workflows/format.yml` with `yaml@2.8.2`, and expresses every rule as an +allow-list over the parsed document. The hand-rolled line reader is retired.** + +**This has landed** (PR #477). The list below is the migration contract as it was written, and +each clause is now a statement about the shipped module rather than an obligation: + +- **One parse, at the top.** The workflow text is parsed once with `yaml@2.8.2`. A parse error is + itself a reported problem, so an unparseable file is red — the fail-closed direction the + superseded decision established survives the migration unchanged, now enforced by the parser + rather than by a rejection list. +- **Every rule reads nodes, not lines.** Indent arithmetic goes: `blockUnder`, `listValueOf`, + `scalarAt`, `keysAt`, `stepsOf`, `withoutBlockScalars`, `isIndentlessItem`, `blockItems` and + their callers are deleted. +- **The four spelling-rejection rule families go with the reader that needed them** — + `flowStyleProblems`, `relocationProblems`, `aliasProblems`, `eventListProblem`. Flow mappings, + JSON-spelled steps, indentless sequences, anchors, aliases and merge keys are resolved by the + parser to the same document GitHub runs, so they are READ and their resolved values are subject + to every semantic rule. They stop being problems because they stop being invisible. +- **Every SEMANTIC rule survives, restated over the parsed document, and stays an ALLOW-LIST.** + Trigger set and filters (`paths`/`paths-ignore` banned, base branch, `types`, `branches-ignore`, + tags-only `push`, `pull_request_target`); `concurrency.group` keyed on the ref and + `cancel-in-progress` scoped to `pull_request`; `permissions` at both levels with no write scope; + job identity — the host job, its display name, no `strategy:`, no `needs:`, no `if:`, no + `defaults:`; the step allow-lists — `uses:`, the checkout's `with:` inputs, the checking-command + EQUALITY on `pnpm format:check`, the remedy's scope-and-`decides()` contract, and + `SETUP_COMMAND_LINES`; and the write-mode formatter/auto-commit ban. None of these is relaxed by + the migration: a rule that was an equality stays an equality, on the value the parser resolves. +- **`run:` bodies stay shell, not YAML.** The parser hands each `run:` scalar over as a string and + `extractRunBlocks`' consumers scan it as shell — which is exactly the boundary + `withoutBlockScalars` was hand-maintaining, now drawn by the parser for free. (`#` comments + inside that body are still stripped quote-aware: that is the SHELL's comment rule, applied to + shell text, and YAML's own comments never reach it.) + +**One clause of this contract was WRONG, and the boundary probe is what corrected it.** The +contract said merge keys "are resolved by the parser to the same document GitHub runs". They are +not, in either direction: `yaml@2.8.2` leaves `<<` unmerged by default (it surfaces as a literal +`<<` key), and GitHub **refuses to run the file at all** — measured on PR #477, probe run +[33724280781](https://github.com/foomakers/pair/actions/runs/33724280781), a `jobs:` block using +`<<: *base`: zero jobs, "invalid workflow file". So `<<` stays rejected, now by the job/workflow +key allow-list and with the producer's own verdict behind it. The same probe series settled the +rest of the class empirically rather than by reading the spec — flow trigger mapping +([33724282425](https://github.com/foomakers/pair/actions/runs/33724282425)), anchors +([33724282478](https://github.com/foomakers/pair/actions/runs/33724282478)), an alias DECIDING a +trigger ([33724282535](https://github.com/foomakers/pair/actions/runs/33724282535)), a JSON step +([33724282504](https://github.com/foomakers/pair/actions/runs/33724282504)) and a job-level `env:` +reaching a step ([33724282486](https://github.com/foomakers/pair/actions/runs/33724282486)) all +RAN; an unknown top-level key +([33724281525](https://github.com/foomakers/pair/actions/runs/33724281525)) was rejected with the +merge key. The guard's allow-lists agree with the producer on every row. + +- **The test suite migrates with the module, it is not rewritten.** Every mutation case is the + contract and stays: a mutation of the shipped workflow that was RED stays RED with the same + cause. The cases that asserted the REJECTION of a legal spelling INVERT — a flow trigger mapping, + a JSON step, an anchored `cancel-in-progress`, an aliased `branches:`, a CRLF file, a quoted + `run:` are correct workflows and must go GREEN, with the underlying semantic rule still enforced + on the resolved value (e.g. `pull_request: { branches: [release] }` stays RED — for being off the + base branch, not for being flow). + +**Why now, and on whose authority.** The flip trigger the superseded version wrote down — +"anything that still fails OPEN, or the first rule needing real structure (nested `with:`, matrix +jobs)" — **FIRED at round 12**, when the guard gained a `with:` rule for the checkout's inputs. +That version recorded the firing but left the consequence as "the human's call at the merge gate". +The maintainer has now made that call, at the merge gate, on **2026-09-02 and again on +2026-09-03**: take the migration, in this PR. It is recorded here as taken and binding so that no +later reading of this file re-derives "defer" — the previous framing is exactly what produced two +rounds of re-deferral against a decision already made. + +### Superseded — the decision recorded here on 2026-09-01 (reject rather than parse) + +**No longer in force.** Kept verbatim in substance, because it is what shipped between rounds 5 and +14 and a reader of that code needs it: + +> ~~The guard **rejects** any spelling it does not read, rather than parsing it or defaulting it to +> "absent".~~ Concretely, on **both** of YAML's node positions: ~~every structural KEY it reads as a +> block (`on`, each trigger under `on`, `concurrency`, `jobs`, each job, `steps`) must carry a BLOCK +> value — nothing after the colon on its own line; every sequence ITEM it walks into must be a BLOCK +> MAPPING (`- key: value`) or a plain scalar (`- main`, under a trigger filter). `- {`, `- [`, +> `- *alias`, `- &anchor` and a bare `-` are rejected — the contents of `steps:` are read by walking +> into each item, so an item the reader cannot follow is a whole step out of view. Anchors, aliases +> and merge keys are rejected anywhere in the file.~~ +> +> ~~And "block style" means what YAML means by it: an **indentless** block sequence is block style +> and is read (round 12). The bound: a spelling the reader cannot follow is rejected by name; a +> spelling it can follow is read.~~ +> +> The part that SURVIVES the supersession, because it was never about the reader: **the guard's +> incompleteness must fail CLOSED**, and the module's rules are ALLOW-LISTS, not deny-lists — +> `if:`, `uses:`, `with:`, the checking command, the setup commands. Each is a narrow canonical +> spelling; everything else is a deliberate edit to the guard. The parser changes what "a spelling +> the guard can follow" means; it does not license a deny-list. +> +> ~~The bound is deliberate: spellings the reader DOES read correctly stay accepted — `branches: +> [main]`, `permissions: { contents: read }`, and the body of a `run:` block scalar. A guard that +> fails a correct workflow is the kind that gets weakened or deleted.~~ The last sentence stands on +> its own terms, and rounds 12–14 are five instances of the guard doing exactly that. + +## Alternatives Considered + +- **Keep the hand-rolled reader under the "reject what you cannot read" bound** (the superseded + 2026-09-01 decision): **rejected on 2026-09-03**, having been the adopted decision for nine + rounds. It closed the fail-open class at the cost of a rejection surface that then produced five + fail-closed findings of its own (rounds 12–14 above), and it required the reader to be extended + every time YAML was spelled differently. The trade it offered — ~~"the parser migration is a + known, costed follow-on rather than an open hole"~~, ~~"what remains open is a judgement the + human makes at the merge gate, not a new card"~~ — is **superseded**: the judgement was made, the + answer is the migration, and it lands in this PR. + + The round-by-round record of that framing is kept because it is the audit trail of how the call + was reached, and it is struck rather than deleted: + + - ~~**Round 6**: the round-5 trigger ("the NEXT finding of this class") fired and was answered by + closing the second node position instead of migrating, because that is what the finding + recommended.~~ + - **Rounds 7–9 did not fire it.** Their findings were RULE gaps — semantics GitHub defines (a + tags-only `push` filter fires for tag refs only; `concurrency.group` must be keyed on the ref; + the check context is the job's DISPLAY name, so `name:` and a matrix rename it) — read + correctly by the line reader and simply unasserted. **This observation survives the + supersession and is the honest bound on what the migration buys**: a parser would not have + caught any of them. Round 13's `SETUP_COMMAND_LINES` (the last deny-list in the module, + replaced by a toolchain allow-list) is the same kind. Parsing closes the spelling class; it + closes nothing in the rule class. + - ~~**Round 12, for the human at the merge gate**: the trigger text names "nested `with:`" as a + flip condition and round 12 added a `with:` rule; whether that counts as the trigger firing is + recorded as the human's call.~~ **It counted.** The human's call, recorded above: it fired. + - ~~**Round 13, for the same human**: three more reader false-fails, patched in place because + each was one line. The call stays where round 9 put it: accept the line reader, or require the + `yaml@2.8.2` migration before merge.~~ **The call is made: require the migration.** +- **Defer the migration to a follow-up card**: rejected. Per ADL + [2026-08-12](./2026-08-12-implementation-never-files-a-card-it-extends-the-story.md), + implementation extends the story rather than filing a card — and deferring would ship the guard + built on the reader whose incompleteness is the entire reason the rules exist, with the trigger + already fired. +- **Teach `blockUnder` to parse flow mappings**: rejected — a second reader to keep in step with the + first, and the next spelling neither knows fails open again. It is exactly the "hand-rolled + checker" ADL 2026-07-29 rejected, with the failure mode round 5 measured. +- **Keep parsing block style only, and pattern-match the flow spelling into the existing rules** + (e.g. widen the `paths`/`paths-ignore` regex to drop its `^\s*` anchor): rejected — it fixes the + one key a reviewer happened to name and leaves `branches`, `types` and every future trigger key + reading as "no filter". +- **A different parser (`js-yaml`, a GitHub-Actions-specific schema validator)**: rejected — + `yaml@2.8.2` is already this repo's adopted parser (ADL 2026-07-29) and already a catalog entry, + so a second one would be an unargued divergence. An Actions schema validator asserts that the + file is a valid workflow; it asserts nothing about the properties this guard exists to hold. + +## Consequences + +- **Tech-stack impact: `yaml` gains a second consumer.** `@pair/dev-tools` declares + `"yaml": "catalog:"` (the catalog already pins `yaml: 2.8.2` in `pnpm-workspace.yaml` for + `apps/pair-cli`), so no new package is resolved and the lockfile delta is a workspace entry, not + a download. It stays dev-only: `@pair/dev-tools` is `private: true` and the module runs under + `ts-node` from `format-workflow:check` and `pnpm test` — nothing ships to an adopting project. + This widens ADL 2026-07-29's rule ("generated YAML is asserted by parsing") to its mirror image: + **consumed YAML is asserted by parsing too**. +- **The module is expected to shrink.** It is 1984 lines at `37cf84b4`; the reader helpers and the + four spelling-rejection rule families come out with the migration. Shrinkage is an expectation, + not an acceptance criterion — the criterion is that every semantic rule keeps its mutation test + and every mutation that was RED stays RED. +- **A correct workflow stops being red for its spelling.** Flow style, anchors, merge keys, + indentless sequences, CRLF and quoted scalars become the contributor's choice again, in this file + as anywhere else. The workflow header comment and the module header stop advertising a + block-style requirement, because there is none. +- **The class of finding that closes is "spelling X is invisible / spelling X false-fails".** The + class that remains open is a MISSING RULE — rounds 7–9, round 12's rule gaps and round 13's + setup-command allow-list are all of that kind, and no parser prevents them. That is the honest + scope of this decision, and it is why the allow-list discipline in the superseded text is carried + forward rather than dropped. +- **The migration's own risk is carried by the test suite**, which is the reason it is migrated + rather than rewritten: 321 tests over the shipped workflow and its mutations, each naming the + property it holds. Measured after the migration: 362 tests, every RED mutation still RED with the + same cause, and the rows that asserted the rejection of a legal spelling inverted to GREEN with + the semantic rule asserted on the resolved value. +- **An allow-list over ELEMENTS is not an allow-list over the VALUE** (added 2026-09-03, review + round 17). `runs-on` accepted any sequence whose every label was allow-listed. GitHub **ANDs** + the labels of a list, and no hosted image carries two image labels, so `[ubuntu-latest, + ubuntu-22.04]` — every element allow-listed — names NO machine: probe run + [33782665948](https://github.com/foomakers/pair/actions/runs/33782665948) on PR #477 left + `d8-two-ubuntu` and `d8-three-ubuntu` `queued`, never started, while `[ubuntu-latest]` and + `[ubuntu-latest, ubuntu-latest]` (GitHub dedupes) completed. The rule now COUNTS: exactly one + label. It is deliberately narrower than the producer on one measured row — the duplicate-label + spelling runs on GitHub and is rejected here — and that narrowing is the one place this module + fails a workflow GitHub would honour. It is accepted because the direction is a false RED with a + message naming the one-label spelling, never a false green, and because the alternative (dedupe + before counting) buys a spelling nobody writes at the cost of a second acceptance path. +- **Two surfaces became allow-lists in the same change, both of them the "relocation" shape this + module keeps meeting.** (a) The WORKFLOW's own keys and every JOB's, because a job-level `env:` + reaches the checking step whatever the step-level allow-list says: measured end to end, + `NODE_OPTIONS=--require=` makes the repo's own pinned prettier 3.6.2 print the + offending filename and exit **0**, so `pnpm format:check` names the file and the `format` context + reports SUCCESS on unformatted code — the `with: ref: main` loss class spelled as a job key. + `container:`/`services:` are the `uses:` third-party-code argument one level up. (b) The REMEDY's + shell, the module's last deny-list: a formatter no offender list names (`npx dprint fmt`) and a + `git commit -am style && git push` beside the required message were both green. The remedy SAYS + what to run; its shell is now quoted `echo`/`printf` and nothing else. With those two, the module + header's claim that every surface is an allow-list is true as written. + +## Adoption Impact + +- `.pair/adoption/tech/way-of-working.md` — the `format` required-check bullet listed "the + block-style requirement" among the asserted properties. **Edited**: that clause is gone (there is + no spelling requirement), replaced by the key allow-lists, and the ADL reference now states that + the guard parses and that an unparseable file is itself a problem. +- `.pair/adoption/tech/tech-stack.md` — the `yaml v2.8.2` entry read "(devDependency) for parsing + generated YAML in tests". **Edited**: it now covers YAML this repo generates OR consumes, and + names both declaring packages. `packages/dev-tools/package.json` declares `"yaml": "catalog:"`; + the catalog already pinned 2.8.2, so no new package resolves, and `@pair/dev-tools` is + `private: true`, so it stays dev-only. +- ADL [2026-07-29](./2026-07-29-yaml-parser-for-generated-yaml-tests.md) is **extended, not + superseded** — same parser, same argument, second consumer. +- `.pair/llms.txt` — regenerated with the production generator (the title of this record changed). diff --git a/.pair/adoption/decision-log/2026-09-03-the-review-severity-floor-defaults-to-minor.md b/.pair/adoption/decision-log/2026-09-03-the-review-severity-floor-defaults-to-minor.md new file mode 100644 index 000000000..407c8a197 --- /dev/null +++ b/.pair/adoption/decision-log/2026-09-03-the-review-severity-floor-defaults-to-minor.md @@ -0,0 +1,58 @@ +# Decision: the review severity floor defaults to Minor — Questions are carried, never fixed + +- **Status**: Active +- **Date**: 2026-09-03 +- **Category**: Process +- **Deciders**: maintainer + +## Context + +`pair-implement-batch` drives a card through a review<->fix loop that closes when the +independent review returns zero ACTIONABLE findings. `severityFloor` names the lowest +severity that blocks; findings below it are carried to the merge gate in `acceptedFindings` +with `disposition: 'Below severity floor'`. The parameter was optional and **defaulted to +no floor**, i.e. every finding the reviewer returned — Questions included — entered the set +the fixer must resolve. + +Measured on PR #477 (story #413), three times in one cycle: + +| head | review verdict | what the next round did | re-review | +| --- | --- | --- | --- | +| `b518ba6b` | APPROVED, 0 actionable, 2 Questions | implemented both Questions | 3 new Minor, all inside the added code | +| `88f852ab` | APPROVED, 0 actionable, 3 Questions | implemented them | 2 new Minor, again inside the added code | +| `0f804485` | CHANGES-REQUESTED | reverted wholesale to `88f852ab` | — | + +The pattern is not a fixer defect. The review template defines Questions as questions **for +the human** — items the reviewer explicitly marks "No change requested". Feeding them to an +agent whose instruction is "resolve EVERY finding, including minor/nit, do not defer any" +contradicts what they are, and each answer enlarges the diff, creating fresh surface for the +next review. Convergence becomes a moving target: the loop can only end by exhausting +`maxFixRounds`. + +## Decision + +The floor **defaults to `Minor`**. Work on a card is complete when Critical, Major and Minor +are closed. Questions are carried to the merge gate for the human, and are acted on only if +the maintainer asks. + +An explicitly passed `severityFloor` still wins, including a lower one that restores the +previous block-everything behaviour. + +The default is applied **softly**, unlike a caller-passed floor: a review template whose +vocabulary does not declare `Minor`, or whose contract carries no usable ranking, falls back +to no floor instead of throwing. A default must never break a run that never asked for it — +while a floor the caller spelled wrong still throws, because that is their configuration +error. + +## Consequences + +- The fix loop terminates on a bounded set. Sub-floor findings are neither discarded nor + silently accepted: they accumulate across every round and reach the human with their + disposition. +- Adopters whose template lacks `Minor` see no behaviour change. +- A caller who wants the old semantics passes the lowest severity their template declares. + +## Adoption Impact + +None beyond this record: `severityFloor` was already documented as caller-configurable, and +its meaning is unchanged — only the default moves. diff --git a/.pair/adoption/tech/tech-stack.md b/.pair/adoption/tech/tech-stack.md index 8b31531ad..101591c48 100644 --- a/.pair/adoption/tech/tech-stack.md +++ b/.pair/adoption/tech/tech-stack.md @@ -31,7 +31,7 @@ Use `turbo` from the repository root to run cross-workspace tasks (e.g. `turbo b - vite-tsconfig-paths v5.1.4 is adopted to enable TypeScript `paths` resolution for Vite and Vitest. - @testing-library/react and @testing-library/jest-dom are adopted for React component testing (peer dependencies of UI workspaces). - vitest-axe v1.0.0-pre.5 is adopted for automated accessibility testing in vitest unit tests. -- yaml v2.8.2 is adopted (devDependency) for parsing generated YAML in tests — generated YAML sinks are asserted by parsing, never by string-matching (see `.pair/adoption/decision-log/2026-07-29-yaml-parser-for-generated-yaml-tests.md`). +- yaml v2.8.2 is adopted (devDependency) for parsing YAML this repo generates OR consumes — generated YAML sinks are asserted by parsing, never by string-matching (`.pair/adoption/decision-log/2026-07-29-yaml-parser-for-generated-yaml-tests.md`), and the repo's own workflow guards parse the workflow they read rather than re-implementing a reader (`@pair/dev-tools`' `format-workflow-composition`, `.pair/adoption/decision-log/2026-09-01-workflow-guard-rejects-what-it-cannot-read.md`). Declared by `apps/pair-cli` and `@pair/dev-tools`; both are dev-only, nothing ships to an adopting project. - @axe-core/playwright v4.11.1 is adopted for automated accessibility testing in Playwright CT. - jsdom v25.0.1 is adopted as the DOM environment for vitest unit tests in React/UI workspaces (`apps/website/`, `packages/brand/`). - Playwright is adopted for E2E and component testing: diff --git a/.pair/adoption/tech/way-of-working.md b/.pair/adoption/tech/way-of-working.md index 3bbc3b8ed..2acd03348 100644 --- a/.pair/adoption/tech/way-of-working.md +++ b/.pair/adoption/tech/way-of-working.md @@ -54,17 +54,38 @@ Resolution order, the split-tool routing and why the fallback is never the authe - When a bug fix or feature changes behavior covered by an existing CP, the corresponding test case MUST be updated. - **CP5's docs page list is machine-asserted against the filesystem** — `packages/knowledge-hub/src/conformance/docs-page-coverage.test.ts` compares it to `apps/website/content/docs/**/*.mdx`, so adding a docs page without listing it in CP5 fails CI ([ADL](../decision-log/2026-08-20-cp5-page-list-is-asserted-against-the-filesystem.md)). +## Review Convergence + +- **Baseline then delta:** the first review is complete and returns the immutable 40-character + head it inspected. A re-review verifies prior findings plus only the diff from that head and + directly changed producer/consumer boundaries; an unchanged PR surface does not create another + fix round. Missing or invalid review-head evidence fails closed, never converges a PR. +- **Provisioned artifact proof:** a fix that installs, builds, publishes, names, or invokes an + artifact maps `producer -> published identity -> consumer` and proves the real path in a clean + temporary environment. The exact boundary is never stubbed, aliased, or faked. See ADL + [2026-08-31-review-baseline-and-provisioned-artifact-contract.md](../decision-log/2026-08-31-review-baseline-and-provisioned-artifact-contract.md). +- **Contract inventory before a loop:** before reporting or fixing a changed contract, inventory + its authoritative producer, inputs, consumers and representations. A finite protocol, parser, + configuration or state transition gets a complete decision table of supported and + invalid/boundary states, with a real probe/test per row. When a row, equivalence, normalization + or repair depends on an external tool/service/format, prove it at that authoritative boundary; + an internal unit test cannot prove external semantics or that repair advice works. Re-review + applies the same rule only to its delta and changed boundary. See ADLs + [2026-09-01-review-contract-inventory-prevents-serial-findings.md](../decision-log/2026-09-01-review-contract-inventory-prevents-serial-findings.md) + and [2026-09-01-external-boundary-proof-prevents-false-equivalence.md](../decision-log/2026-09-01-external-boundary-proof-prevents-false-equivalence.md). + ## Quality Gates - `pnpm quality-gate` is the adopted project-level quality gate command. -- Quality gate includes: type checking (`ts:check`), testing (`test`), linting (`lint`), formatting and markdown lint in **check mode** (`format:check`), plus a guard that the gate stays check-mode (`gate:composition`) and the smoke-scenario mode guard (`smoke-modes:check`). +- Quality gate includes: type checking (`ts:check`), testing (`test`), linting (`lint`), formatting and markdown lint in **check mode** (`format:check`), plus two composition guards under `gate:composition` (the gate stays check-mode — `pre-push-gate:check`; `format.yml` keeps its shape — `format-workflow:check`) and the smoke-scenario mode guard (`smoke-modes:check`). - **`format:check`/`format` coverage is whole-repo, derived from git, not turbo's per-workspace scope** (#414): `scripts/format-lib/run-format.sh` lists every path `git ls-files --cached --others --exclude-standard` reports (extension-filtered), so "gitignored ⇒ never checked" is git's own rule — nested `.gitignore` files and the user's global `core.excludesFile` apply by construction, with no re-implementation in the wrappers. Coverage excludes almost nothing: root-level and non-workspace files (`.claude/**`, `.pair/adoption/**`, `qa/**`, `scripts/**`) are checked exactly like workspace files. **One documented exception**: third-party skills installed under `.claude/skills/` (any directory not matching the `pair-*` prefix, e.g. a marketplace skill) are never checked — their formatting is not this project's to maintain. An empty derived file set is treated as a broken wrapper (exit 2), never a silent pass — see `scripts/format-lib/git-tracked-paths.sh`. The per-package, glob-based invocation (`pnpm --filter prettier:check`/`mdlint:check`) is unaffected and still uses the wrappers' own `_ignore-args.sh`/`_ignore-file.sh` ignore assembly. -- **A guard whose only caller is a turbo task is not enforced.** `turbo ts:check test lint` are cacheable with package-scoped inputs, so a change OUTSIDE the guard's package replays a cached PASS and the guard never executes. A guard over repo-wide state therefore gets a thin CLI and a **root gate step** (`hygiene:check`, `smoke-modes:check`, `docs:staleness`, `skills:conformance`), which run unconditionally — a unit test alone is the enforcement point only for logic whose inputs live inside its own package (#400). +- **A guard whose only caller is a turbo task is not enforced.** `turbo ts:check test lint` are cacheable with package-scoped inputs, so a change OUTSIDE the guard's package replays a cached PASS and the guard never executes. A guard over repo-wide state therefore gets a thin CLI and a **root gate step** (`hygiene:check`, `smoke-modes:check`, `docs:staleness`, `skills:conformance`), which run unconditionally — a unit test alone is the enforcement point only for logic whose inputs live inside its own package (#400). **Second, lighter mechanism for the same guarantee**: declare the guarded artifact as a `$TURBO_ROOT$` **task input** in `turbo.json` (turbo >= 2.1), so the cache can no longer serve a stale PASS — used by `@pair/knowledge-hub#test` (the KB/skills/docs artifacts), and by `@pair/dev-tools#test` for `scripts/format-lib/**`, `.github/workflows/format.yml` and **the root `package.json`** — that third entry is not optional bookkeeping: both guards in that folder resolve script delegation against the root scripts (`checkThisRepoGate` parses it, `checkFormatWorkflow` defaults `rootScripts` to it), so without it rewriting root `format:check` to `pnpm prettier:fix` left the task hash unchanged at `7271baf2a672a276` — a cached PASS with neither guard running. Sufficient because CI is **cold on every run** (no remote cache, no `.turbo` restore in `ci.yml`), so `pnpm test` already executes the guard on every PR and the input entry closes the LOCAL false green — which is also what the pre-push hook sees. Pick per guard by what invalidates it: the CLI + root-step form stays **required** when the input set is not expressible as task inputs, or when a failure must report as its own CI status context rather than inside `pnpm test` — and a guard may use BOTH, as `format-workflow-composition` does (the input entry keeps `pnpm test` honest; `format-workflow:check` under the existing `gate:composition` segment is the mechanism #413's AC6 names). See ADL [2026-09-01-repo-wide-guard-enforced-by-turbo-root-input.md](../decision-log/2026-09-01-repo-wide-guard-enforced-by-turbo-root-input.md). - **No step reachable from the gate writes files**: the gate reports, `pnpm format` / `pnpm lint:fix` fix deliberately. `gate:composition` enforces this through an **explicit offender list** — the two formatters, eslint autofix, and the repo's write scripts (`sync-version`, `test:perf`) — so **adding a new write-mode script to this repo means adding it to that list**; a differently named writer passes the guard green. See ADL [2026-07-31-pre-push-gate-is-check-only.md](../decision-log/2026-07-31-pre-push-gate-is-check-only.md). - **Pre-merge tiering**: `disabled` (default) — every PR runs the full pre-merge check suite. Set to `enabled` to opt into risk-tier-scoped pre-merge checks (lighter checks on lower-risk PRs) per [tier-aware-pipeline.md](../../knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md); `/pair-capability-setup-gates` reads this flag before generating the pipeline. - **Review enforcement**: `disabled` (default) — the pair review **runs and publishes its verdict**, but nothing it says blocks a merge: `pair-review` and `pair-explicit-approval` are not required status checks, and the 🔴 explicit-approval rule is advisory. Set to `enabled` to make them required and the rule binding, per [pr-states.md](../../knowledge/guidelines/collaboration/project-management-tool/pr-states.md); `/pair-capability-setup-gates` reads this flag before touching branch protection, and `/pair-process-bootstrap` asks for it when no decision exists. Disabled is the default deliberately: a review that blocks by default turns a first install into a repository nobody can merge into — on a single-maintainer repo the 🔴 non-author approval is unobtainable outright. The tier requirements themselves (reviewer count, SLA, checklist depth, whether 🔴 needs explicit approval) are redefinable in this file; that the review **runs** is not. - **Coverage guardrail**: `enabled` — pair dogfoods its own capability: the [`Coverage guardrail` step](../../../.github/workflows/ci.yml) in CI sources [`coverage-gate.sh`](../../knowledge/assets/coverage-gate.sh), extracts the line-coverage % from each package's istanbul `coverage-summary.json`, and blocks a PR whose coverage drops below the human-committed baseline in [`tech/coverage-baseline.md`](./coverage-baseline.md) (maintaining/improving passes — not an absolute wall). The framework **default remains `disabled`** (the dataset template ships off); this line is pair's project-level opt-in only. See [coverage guardrail](../../knowledge/guidelines/infrastructure/cicd-strategy/tier-aware-pipeline.md#coverage-guardrail-opt-in-regression-gate-consumed-by-this-pipeline) + [config format](../../knowledge/assets/coverage-config-example.md); `/pair-capability-setup-gates` reads this flag before generating the pipeline. **Coverage baseline commit-back**: `disabled` — the separate, nested opt-in ratchet (#372, framework default also `disabled`): when `enabled`, a **push to the base branch** (never a PR run, never a fork) proposes a raised `baseline.` as a **bot pull request** from `chore/coverage-baseline-ratchet`, never a push to `main`, and requires a repo-scoped `COVERAGE_RATCHET_TOKEN` (`contents: write` + `pull requests: write`, no protection bypass) — without it the step warns and the gate's verdict is unchanged. It stays `disabled` here until story #234's branch protection is applied and that secret is provisioned (ADR-018 lands with that story, so it is not linked from here yet); see ADL [2026-07-30-coverage-ratchet-pr-not-push.md](../decision-log/2026-07-30-coverage-ratchet-pr-not-push.md). The step that runs it is the **shipped** KB asset `node .pair/knowledge/assets/coverage-ratchet.cjs` (ADR-023) — the same one an adopter's generated pipeline invokes, so this flag being `enabled` means the same thing here as anywhere else. - **Pair review required checks**: `pair-review` + `pair-explicit-approval` are the required status checks that make the judgment review unskippable (R5.7) and enforce the 🔴 explicit-human-approval rule (D10) — see [pr-states.md](../../knowledge/guidelines/collaboration/project-management-tool/pr-states.md) and [ADR-018](adr/adr-018-pr-state-flow-required-checks.md). Status on this repo: **not yet applied** — writing branch protection needs admin scope, so it is a deliberate human step; until applied, enforcement here is advisory (the documented degraded mode). **Ordering constraint** (applies in this order, or every merge stops): 1. provision the `pr-state:*` labels + add the `pair-explicit-approval` workflow (neither needs admin scope — this repo has not added the workflow yet, so the context does not report today); 2. confirm on a real PR that `pair-review` and `pair-explicit-approval` both report on the head commit, **and** that the approval context re-reports on that same head SHA after a review submission; 3. only then `PUT` the branch protection, keeping `enforce_admins` off until one PR has merged through it. The whole sequence (including the merge-block outcomes per tier) was executed on a throwaway repository — see `github-implementation.md` § "Verified on a throwaway repository" — so what remains here is applying it, not discovering whether it works. **This repo is single-maintainer**, so a 🔴 PR cannot satisfy `pair-explicit-approval` (GitHub rejects a self-approval): a second human reviewer account is a prerequisite for making that context required here — otherwise leave it out of the required list and keep the 🔴 rule advisory. The solo-maintainer alternative (a verified human approval token instead of a second account) is tracked as [#398](https://github.com/foomakers/pair/issues/398). **When the protection is written here, use the `checks` form with `app_id` pinned** for `pair-explicit-approval` (an unpinned status context is satisfiable by any push-access token, including the agent's); `pair-review` stays unpinned and is an anti-accident control, not an authorization control — see `github-implementation.md` § "What each context proves". +- **`format` required check**: `format` is the third context in the "declared but not yet required" set, alongside the two above — CI runs `pnpm format:check` on every pull request and on push to `main` (plus `workflow_dispatch`, the same manual escape hatch `ci.yml` carries), via its own [`format.yml`](../../../.github/workflows/format.yml) workflow (#413), published as the status context `format`. A **dedicated workflow, not a job in `ci.yml`**: that workflow's workflow-level `paths-ignore: ['.changeset/**']` is inherited by every job, so a `.changeset/**`-only PR would run no formatting check — trigger coverage is part of check coverage. The workflow's shape — triggers and filters, concurrency, permissions, the exact command, the scoped remedy, the checkout's inputs, the workflow/job/step key allow-lists and every other allow-list — is asserted by [`format-workflow-composition`](../../../packages/dev-tools/src/quality-gates/format-workflow-composition.ts) in `@pair/dev-tools`: that module's header is the rule inventory (one source, not restated here), and ADL [2026-09-01-workflow-guard-rejects-what-it-cannot-read.md](../decision-log/2026-09-01-workflow-guard-rejects-what-it-cannot-read.md) records that the guard PARSES the file with `yaml@2.8.2` (no spelling requirement: flow style, anchors, aliases, JSON steps and CRLF are read) and that a file the parser refuses is itself a problem. Enforced twice: by `pnpm test` (the `$TURBO_ROOT$` input, bullet above) and by `pnpm gate:composition` (`format-workflow:check`). Status on this repo: **not yet applied** as a required check — same pending admin-scope step, and the same ordering constraint, as the row above; until then it reports and is advisory. Unlike `pair-explicit-approval` it needs **no `app_id` pinning and no second human account**: it asserts a mechanical property of the tree, so it is an anti-accident control like `pair-review`, not an authorization control. - **Gate & tooling code:** a gate's logic lives in a tested module in its owning package (white-box unit tests); scripts/CLIs are thin entrypoints and a root gate delegates (`pnpm --filter `). Scripts are never unit-tested — CLI-level checks go to smoke tests. See ADL [2026-07-13-gate-tooling-code-in-tested-modules.md](../decision-log/2026-07-13-gate-tooling-code-in-tested-modules.md). Gate/tooling packages are organized by bounded context, not one package per tool family — a new tool family sharing an existing package's bounded context is a new folder there, not a new package. See [ADR-014](adr/adr-014-tool-package-boundary-by-bounded-context.md). - **Conformance tests** (`packages/knowledge-hub/src/conformance/`): one test file per target KB artifact (a `SKILL.md`, guideline, or template), not per introducing story — a new story extends the matching file's `describe` block instead of adding a new story-named file. See ADL [2026-07-18-conformance-test-per-file-not-per-story.md](../decision-log/2026-07-18-conformance-test-per-file-not-per-story.md). - **Monorepo tooling gotchas** (e.g. `pnpm --filter` bypassing turbo's `dependsOn` graph on a fresh checkout): documented once, centrally, in `DEVELOPMENT.md`'s `Turbo Caching` section — affected packages' READMEs carry only a short pointer, not a full copy. See ADL [2026-07-18-workspace-gotcha-doc-placement.md](../decision-log/2026-07-18-workspace-gotcha-doc-placement.md). diff --git a/.pair/knowledge/guidelines/collaboration/project-management-tool/README.md b/.pair/knowledge/guidelines/collaboration/project-management-tool/README.md index e45f8f1fd..26d5c984d 100644 --- a/.pair/knowledge/guidelines/collaboration/project-management-tool/README.md +++ b/.pair/knowledge/guidelines/collaboration/project-management-tool/README.md @@ -34,6 +34,8 @@ This framework does not cover: **[definition-of-ready-and-done.md](definition-of-ready-and-done.md)** - Canonical Definition of Ready (R3.8, 6 criteria) and Definition of Done (R3.9, 4 criteria + per-tier hooks) — companion to canonical-states.md, the source `/pair-capability-verify-done` and the user-story template read +**[task-progress-feedback.md](task-progress-feedback.md)** - The breakdown-to-task feedback loop: the task-ID locator, the tick-only body patch, and the single batched progress comment per run iteration (D22) — how an inline task checklist reports progress while `/pair-process-implement` works through it, on the manual and the supervised path alike + **[pr-states.md](pr-states.md)** - The PR state flow (gate ≠ review): the 3 PR states (`to-be-reviewed` → `ready-to-merge` / `not-approved`), the synthesis of gates × review verdict × tier × explicit approval, and the required `pair-review` / `pair-explicit-approval` checks that make merge blocking mechanical — the pull-request companion to canonical-states.md ### Implementation Guides diff --git a/.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md b/.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md new file mode 100644 index 000000000..b11fcab1a --- /dev/null +++ b/.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md @@ -0,0 +1,133 @@ +# Task-Progress Feedback — checklist ticks + one batched comment + +A story broken down by `/pair-process-plan-tasks` carries its tasks **inline**, as a checklist in its own body. This guideline defines what happens to that checklist while `/pair-process-implement` works through it: the completed item is **ticked**, the outcome is **queued**, and the run posts **one batched comment** carrying a line per task. That is the whole of the feedback loop — progress is visible at task granularity, on the item itself, without anyone watching a terminal. + +It exists because an unattended run is otherwise silent between "In Progress" and a pull request. The story that is on task 3 of 4 and the story that failed on task 2 look identical on the board, and the only witness to the difference is a session that no longer exists. + +## Scope — one mechanism, one owner + +The call sites belong to `/pair-process-implement` — Step 2.8 (per task) and its closing phase (the flush). **`/pair-process-implement` is also the only caller**: a supervised run reaches this loop through `/pair-loop` → `/implement-batch` → `/pair-process-implement`, so the manual and the supervised path run the same code path by construction, not by two implementations kept in agreement. An automation layer that posted its own per-task progress comment would double every line this one already carries. + +What this guideline owns: the locator, the tick, the batch, its format, and the fallbacks when a write does not land. What it does not own: + +- **The transport.** The tick and the comment are both written through `/pair-capability-write-issue` — write mode for the body, `$mode: comment` for the batch — and every per-tool mechanic (which API call, which flag) stays in that skill and the implementation guide it resolves. +- **The board state.** Ticking an item is not a state transition: `In Progress` is written once, at Step 0.1b, and this loop writes no board **state** field, ever. What it does not get to opt out of is the transport's own contract — `/pair-capability-write-issue` runs its **membership beats** (a project-item read, an idempotent add, a confirming re-read) on **every** write-mode write, board state requested or not. So on an explicit-membership tool (GitHub Projects) each tick — and each Definition-of-Done box, which is its own write — costs one board read and, at most once per item, one add. The tick therefore passes **`$on-failure: report`**, and a membership that cannot be confirmed is **reported to the caller** and carried in the batch's `
`, never raised as a HALT. +- **The task model.** Tasks are inline checklist items; **no separate task issues are created**, and none are created here either — a tick is an edit to one line of one body. + +## The task-ID locator + +The item to tick is found by its **task ID**, never by its position and never by its title. The match is **anchored on the task ID token** — `T` followed by the task's number, with an optional hyphen and optional bold markers — so that every rendering the corpus actually produces resolves to the same item: + +| Rendering | Where it comes from | +| --- | --- | +| `- [ ] **T-3**: Comment batcher` | `/pair-process-plan-tasks`' canonical checklist line | +| `- [ ] T3 — Comment batcher` | a story body's breakdown as hand-written during refinement — the shape real issue bodies carry (story #220's own body is one) | +| `- [ ] **T3**: Comment batcher` | the two above, mixed | + +Titles are not part of the anchor: a task renamed during implementation still ticks, and a title that happens to contain another task's words never steals the match. + +The locator must resolve to **exactly one** unticked-or-ticked checklist line in the `## Task Breakdown` section. Anything else is a mismatch, and a mismatch is **reported, never resolved**: + +- **Zero matches** (the body was edited meanwhile, the section was renamed, the ID never existed): record `not-found` for that task in the batch — the comment says which ID was not found in which section — and write nothing to the body. +- **More than one match** (an ambiguous ID, e.g. `T1` also appearing inside `T1`'s own follow-up line): record `ambiguous` with the count, and write nothing to the body. + +Both outcomes leave the checklist exactly as it was. **Never guess-tick**: a tick on the wrong line is worse than no tick at all, because the story then reports work that was not done and nothing later contradicts it. + +## The Definition-of-Done locator + +A Definition-of-Done checkbox has no task ID. When a task factually satisfies one, name the pair the body currently carries: the exact Definition-of-Done section heading and the exact criterion text. This locator is separate from, and **not the task-ID locator**: `T3` may identify the work, but it cannot identify `Automated tests written and passing` among the story's DoD boxes. + +Read the current body and find the checklist section whose normalized heading exactly equals the named Definition-of-Done heading. Within that section only, compare each checklist line's normalized label — trim outer whitespace, collapse internal whitespace, ignore Markdown emphasis — with the named **exact criterion**. Do not use a position, a substring, a task title, or an inferred “closest” DoD box. + +The Definition-of-Done locator must resolve to **exactly one** unticked-or-ticked checklist line. **Zero matches** (`DoD locator: not-found`) or **more than one match** (`DoD locator: ambiguous`) are reported in that task's batch details and write nothing to the body. **Never guess-tick** a DoD box: an ambiguous completion claim is worse than leaving the box for a reviewer. + +## The tick-only body patch + +The patch is a single-line edit. On the matched line, and on nothing else, `[ ]` becomes `[x]`; **every other byte of the body is identical** to the body that was just read — same sections, same ordering, same trailing whitespace, same Definition-of-Done boxes. + +Four properties make that a rule rather than an intention: + +1. **The transport is a full-body overwrite.** `/pair-capability-write-issue`'s write mode replaces the body with what the caller passes, so the caller must pass the current body with the one line patched — read, patch, write. Nothing about the transport prevents a re-rendered body from replacing the story's acceptance criteria; the caller's own care is the only thing that does. +2. **The write is diff-checked before it is sent.** Compare the patched body against the body just read: the diff must be exactly one line, and that line must differ only in its checkbox marker. A diff of any other shape is a bug in the patch, and the write is **abandoned** (recorded as `patch-rejected` in the batch), never sent hopefully. +3. **One write per checkbox.** The single-line diff is the shape of one *write*, not of one task: a task that ticks its own item **and** a Definition-of-Done box is **two** sequential read-patch-write cycles, each with its own fresh read and its own one-line diff check. Batching both boxes into one body would present a two-line diff, the check would reject it, and the task's own tick would be lost with it — a `patch-rejected` reported over work that was entirely correct. +4. **The patch never unticks.** An item **already ticked** (`[x]`) is already in the target state: no body write is sent at all. Whether it is also *reported* depends on the invocation, never on the checkbox: a task **this invocation** completed and found already ticked is queued as `ticked`; a task this invocation did not attempt is **neither re-written nor queued**. That is what makes a re-run — a resumed story, a re-invocation after a context reset — free of writes, and (when it did no work) free of comment. + +Definition-of-Done boxes are patched by the same rule — its own write, its own diff check — when a task factually satisfies one; boxes that need a reviewer's judgment stay unticked. + +## Batching and the comment format (D22) + +Ticks are silent. The narrative — what was attempted, what landed, what did not — is **one comment per run iteration**, and a **run iteration is one `/pair-process-implement` invocation over one story**: a manual session is one iteration, and a supervised run contributes one per card per pass. Pinning the unit is what keeps the two paths on the same cadence instead of one reading it as "per session" and the other as "per loop pass". + +Each task's outcome is **queued** as it happens and the queue is **flushed exactly once**, at the end of the invocation — including an invocation that ends early. A run that stops at a HALT has the most to report, so the flush is not a success path: it happens on the way out, whatever the way out is. + +The format honours the reading budget (D22): a headline, then **one line per task**, then everything else collapsed. + +```markdown +**Task progress — 2 done, 1 failed of 4 tasks this iteration** · `feature/US-220-breakdown-task-feedback-loop` + +- ✅ T1 — Checklist locator + body patcher +- ✅ T2 — Comment batcher + failure recording +- ❌ T3 — Wiring into implement — quality gate red (unit) + +
+Details + +- T3: `pnpm --filter @pair/knowledge-hub test` failed on 2 assertions; run halted before commit, checklist item left unticked. + +
+``` + +Rules the shape encodes: + +- **The headline counts outcomes, never bare progress.** It reads `Task progress — done, failed, skipped of tasks this iteration`, spelling out only the non-zero groups; `M` is the number of tasks in the story's breakdown, so more lines than `M` is itself a visible tell. `ticked` and the tick-failure outcomes (`not-found`, `ambiguous`, `patch-rejected`, `write-failed`) count as **done** — the tick is attempted only after the task's work completed, so those name a reporting failure over finished work; `failed` and `skipped` count as themselves. A bare `N of M` is forbidden: on a run where T1 lands and T2 and T3 fail it renders `3 of 4`, which a human scanning an unexpanded board notification reads as three tasks done when exactly one did — re-creating, inside this artifact, the "on task 3 of 4 and failed on task 2 look identical" confusion the loop exists to end. +- **The batch is a per-task narrative, not the run's outcome.** It is flushed before the pull-request hand-off, so it cannot know whether the hand-off then succeeded: a run that completes every task and stops on a red gate inside the PR step leaves `4 done of 4` and all ✅ on the item, and **no second comment corrects it**. That is deliberate — run-level success or failure belongs to the artifacts that own it (the pull request's existence and state, the caller's own output, and on a supervised path the loop's card-level note), and buying it here would cost either a second comment per iteration or a flush deferred past the hand-off, leaving the failed hand-off itself unreported. A reader takes this comment for what the tasks did, never for whether the run landed. +- **One line per task, and the line is the whole story for a reader who does not expand.** Outcome glyph, task ID, title, and — for anything other than a plain success — a short reason on the same line. +- **Everything longer goes in `
`**: error output, retry traces, locator mismatch diagnostics. No stack trace, no command transcript, and no diff outside the collapsed block. +- **An empty batch posts nothing.** The queue holds only what **this invocation** attempted: an item already `[x]` for a task this invocation did not attempt is neither re-written nor queued. So an invocation that completed no task leaves no comment — a re-run that finds every task already done is silent, which is what keeps an idempotent re-invocation from accreting one "nothing to report" comment per attempt. +- **Never a second comment.** More detail belongs in `
`, never in another comment; the loop has no verbosity level that turns one comment into several. +- **A batch lost with its session is not recovered.** The queue lives in the invocation that built it. An invocation that ticked T1 and T2 and then died — a context reset before the flush — leaves those two outcome lines unposted on **any** invocation: the next one reports only what **it** attempted. **The ticks stand** (they are body writes, already on the item); the narrative does not, including any `write-failed` / `not-found` diagnostic queued for them. This is the price of the scoping rule above and it is deliberate: a queue outliving its session would have to be reconciled against work the resumed invocation may have redone, and it licenses exactly the accretion — one catch-up comment per resumed run — that the scoping closed. A human reading the item sees the ticks and the tasks still open; what is lost is one paragraph, never a state. + +Verbosity and cadence are the defaults, not a law: a project that wants something else declares it in its own adoption. Absent a declaration, this is what runs. + +## Outcome vocabulary + +Every queued line carries exactly one outcome from this closed set, and the outcome decides the checklist item — never the other way around. + +| Outcome | What happened | Checklist item | +| --- | --- | --- | +| `ticked` | The task completed and the patch landed (or the item was already `[x]`) | `[x]` | +| `failed` | The task did not complete — a red gate, an error, a HALT | stays unticked | +| `skipped` | The task was deliberately not attempted this iteration (blocked, deferred, out of scope) | stays unticked | +| `not-found` | The locator matched no line for this ID | stays unticked | +| `ambiguous` | The locator matched more than one line for this ID | stays unticked | +| `patch-rejected` | The patch diff was not a single checkbox-only line change, so the write was abandoned | stays unticked | +| `write-failed` | The body write was attempted and did not land, retries included | stays unticked | + +`skipped` is produced at **task selection**, not at task completion: the caller queues it when it declines to attempt a task at all — an unmet dependency, a deferral, work put out of scope mid-run — and carries on to the next task. It is **never the outcome of an attempt** that did not land; that is `failed`. Both halves matter: a deliberate deferral reported as `failed` misnames it, and one left out of the batch shrinks the "N of M tasks this iteration" headline without saying why. + +It is queued **at most once per invocation**. Selection scans restart from the top after every task, so a blocked task is reached once per remaining task: a task **already queued** as `skipped` this invocation is passed over silently on every later scan. Otherwise a five-task story blocked on one task reports six lines for five tasks — `Task progress — 4 done, 2 skipped of 5 tasks this iteration`, with that task named twice — breaking both "one line per task" and the headline's per-outcome counts. + +The three write-path outcomes (`not-found`, `ambiguous`, `write-failed`) say nothing about the work: the task may well be done. The line reports the **feedback** failure and names the ID it could not tick, so a human can tick it by hand — and the run carries on. + +## Failure and conflict handling + +A story body is shared: a human can be editing it while a run is ticking it, and a tracker can accept a call and change nothing. Both are ordinary, and neither is allowed to stop the story. + +**A write is confirmed by reading the body back, never by an exit status.** A call that exits 0 while the item's body still shows `- [ ]` is `write-failed`, not `ticked` — a batch that reports a tick the body does not carry is worse than one that reports the failure, because nothing downstream ever contradicts it. + +**Concurrent body edit (the read-modify-write race).** The patch was computed against a body snapshot; between that read and the write, someone else may have changed the body. Compare before writing: if the current body no longer matches the snapshot, **discard the patch** and retry from a **fresh** read — **re-run the locator** on the new body rather than replaying the old patch, because the item may have moved, been renumbered, or already been ticked by the human who was editing. Replaying the stale patch is how the loop would silently revert someone else's edit. + +**Exactly one retry**, per task and per write. A second conflict, or a second failure of any kind, ends the attempt: the retry budget is bounded so a contended body or a broken tracker degrades the feedback instead of stalling the run. + +**Comment-only fallback.** After the retry is spent — a repeated conflict, a rejected patch, a transport error, a read-only body — no further body write is attempted for that task. The outcome is queued (`write-failed`, or the more specific one) and the batched comment carries it, naming the task ID that could not be ticked so a human can do it in one click. The feedback degrades from *tick + comment* to *comment*, and never below. + +**The transport HALTs by default; the tick opts out.** `/pair-capability-write-issue`'s write mode is a HALTing skill — an unresolvable id (its Step 7), a board membership its re-read cannot confirm (Step 7b beat 4, which runs on every write-mode write) and any tracker error (Step 8) each stop it — and **a HALT inside a composed skill propagates to its caller**. Ticking through it unmodified would mean one 5xx or one secondary rate limit while ticking task 1 of 4 ends the run: T2, T3 and T4 never implemented, no pull request, because a checkbox could not be written. So the tick is composed with **`$on-failure: report`** (`/pair-capability-write-issue` Step 8b), which turns those three into returned outcomes: `not-found` and `write-failed` are queued as this vocabulary's outcomes for the task; `membership-unconfirmed` says nothing about the tick — the body write landed — so it rides in the batch's `
` instead of on the task's line. + +**A PM write failure never blocks implementation.** Ticks and comments annotate work that has already happened; the commit is on the branch either way. Whatever this loop fails to write, the run **continues** to the next task and the story to its pull request — the failure is reported, not raised. + +## What never happens + +- **No separate task issues.** Tasks live inline in the story body; nothing here creates, links, or closes a task issue. A tick is an edit to one line. +- **No other section is rewritten.** The acceptance criteria, the classification matrix, the technical analysis and the task titles come back byte-identical from every write this loop makes. Only checkbox markers change. +- **No board state is written.** `In Progress` is set once when the story is activated; a tick is not a transition, and this loop writes no board **state** field. The transport's **membership beats** still run on every write-mode write (Scope, above) — they are `/pair-capability-write-issue`'s contract, not this loop's — and a membership that cannot be confirmed is reported to the caller, never raised. +- **No untick, ever**, and no second comment per iteration. diff --git a/.pair/llms.txt b/.pair/llms.txt index 9ca1efe76..2f53db0b7 100644 --- a/.pair/llms.txt +++ b/.pair/llms.txt @@ -124,7 +124,15 @@ - [Decision: a `pair.config.json` schema guard stays in `#config` and takes its vocabulary INJECTED, never imported from the command layer](.pair/adoption/decision-log/2026-08-24-config-schema-guards-take-their-vocabulary-injected.md) - [Decision: the CLI invocation name is `pair-cli`, not `pair`](.pair/adoption/decision-log/2026-08-25-cli-invocation-canonical-name-is-pair-cli.md) - [Decision: Post-merge cleanup covers local branches and worktrees; PR analyses retire at merge](.pair/adoption/decision-log/2026-08-25-post-merge-cleanup-covers-local-branches-pr-analyses-retire-at-merge.md) +- [Decision: The breakdown-to-task feedback loop is one mechanism owned by /pair-process-implement, batched per invocation](.pair/adoption/decision-log/2026-08-28-task-progress-feedback-is-one-mechanism-owned-by-implement.md) - [Decision: tier 1's `$approval` posture is unconditional, and tier 1 has no declaring composition site yet](.pair/adoption/decision-log/2026-08-28-tier1-approval-posture-is-unconditional-and-has-no-declaring-composition-site-yet.md) +- [Decision: Review re-checks use an immutable baseline and prove provisioned artifacts](.pair/adoption/decision-log/2026-08-31-review-baseline-and-provisioned-artifact-contract.md) +- [Decision: External boundary proof prevents false equivalence](.pair/adoption/decision-log/2026-09-01-external-boundary-proof-prevents-false-equivalence.md) +- [Decision: a workflow guard reads a quoted `echo` argument as data, never as a command](.pair/adoption/decision-log/2026-09-01-quoted-echo-arguments-are-data-not-commands.md) +- [Decision: a repo-wide guard is enforced by a `$TURBO_ROOT$` cache input, not necessarily by a thin CLI + root gate step](.pair/adoption/decision-log/2026-09-01-repo-wide-guard-enforced-by-turbo-root-input.md) +- [Decision: Review contract inventory prevents serial findings](.pair/adoption/decision-log/2026-09-01-review-contract-inventory-prevents-serial-findings.md) +- [Decision: the format-workflow guard PARSES `format.yml` with `yaml@2.8.2`, and every rule is an allow-list over the parsed document](.pair/adoption/decision-log/2026-09-01-workflow-guard-rejects-what-it-cannot-read.md) +- [Decision: the review severity floor defaults to Minor — Questions are carried, never fixed](.pair/adoption/decision-log/2026-09-03-the-review-severity-floor-defaults-to-minor.md) ## How-To Guides @@ -255,6 +263,7 @@ - [Linear - Complete Implementation Guide](.pair/knowledge/guidelines/collaboration/project-management-tool/linear-implementation.md) - [PR State Flow — gate ≠ review](.pair/knowledge/guidelines/collaboration/project-management-tool/pr-states.md) - [Project Management Tool Framework](.pair/knowledge/guidelines/collaboration/project-management-tool/README.md) +- [Task-Progress Feedback — checklist ticks + one batched comment](.pair/knowledge/guidelines/collaboration/project-management-tool/task-progress-feedback.md) - [Azure DevOps Project Tracking](.pair/knowledge/guidelines/collaboration/project-tracking/azure-devops-tracking.md) - [Filesystem Project Tracking](.pair/knowledge/guidelines/collaboration/project-tracking/filesystem-tracking.md) - [GitHub Project Tracking](.pair/knowledge/guidelines/collaboration/project-tracking/github-tracking.md) diff --git a/apps/website/content/docs/developer-journey/execution.mdx b/apps/website/content/docs/developer-journey/execution.mdx index b33ceb41e..66212f7a7 100644 --- a/apps/website/content/docs/developer-journey/execution.mdx +++ b/apps/website/content/docs/developer-journey/execution.mdx @@ -51,6 +51,14 @@ Every new module file must have a corresponding unit test file (1:1 mapping). Mo With commit-per-task, the developer gets a checkpoint between every task — the AI presents a summary and waits for confirmation before committing. +### Progress Feedback: Ticks and One Comment + +While the skill works through the breakdown, the story reports its own progress. As each task completes, its checklist item in the story's **Task Breakdown** is ticked — located by task ID, patched one line at a time, never rewriting another section — and the outcome is queued. At the end of the run, the queue is flushed as **one comment**: a headline, one line per task, everything longer collapsed in `
`. + +The cadence is one comment per run iteration, so a supervised (unattended) run reports at task granularity without turning the story into a comment feed. A task that failed or was skipped is recorded in the comment and its checklist item stays unticked — a story that stopped halfway looks different from one that finished. + +The feedback is annotation, never a gate: a locator mismatch, a body edited concurrently, or a tracker write that does not land is retried once, then reported in the comment, and the run carries on. That holds through the writer too — the tick is composed with `$on-failure: report`, so a tracker error comes back as an outcome to record rather than as a halt that would stop the story it only annotates. Three rules make that safe, and the shipped guideline `task-progress-feedback.md` (installed into your project at `.pair/knowledge/guidelines/collaboration/project-management-tool/`, listed in the [guidelines catalog](/docs/reference/guidelines-catalog)) states them in full: the item is located by **task ID**, never by position or title; the write is **diff-checked** to exactly one changed line, one checkbox per write, or it is abandoned rather than sent; a conflicting body is re-read and re-located, never re-patched from the stale snapshot. + ### Closing Phase: Checkpoint + PR After all tasks are complete, the skill's closing phase splits the handoff from the publish so the PR is always built on a clean context: diff --git a/apps/website/content/docs/reference/guidelines-catalog.mdx b/apps/website/content/docs/reference/guidelines-catalog.mdx index 6bf1a391f..a95d81d54 100644 --- a/apps/website/content/docs/reference/guidelines-catalog.mdx +++ b/apps/website/content/docs/reference/guidelines-catalog.mdx @@ -71,7 +71,7 @@ Process, methodology, and team coordination. | **Estimation** | Story points, T-shirt sizing, estimation techniques | | **Issue Management** | Issue tracking, triage, prioritization | | **Project Tracking** | Velocity, burndown, sprint metrics | -| **Project Management Tools** | GitHub Projects, Linear, Jira configuration; [canonical states + state-mapping schema](/docs/concepts/canonical-states); [PR state flow (gate ≠ review)](/docs/concepts/pr-state-flow); [Definition of Ready & Done](/docs/concepts/definition-of-ready-and-done) | +| **Project Management Tools** | GitHub Projects, Linear, Jira configuration; [canonical states + state-mapping schema](/docs/concepts/canonical-states); [PR state flow (gate ≠ review)](/docs/concepts/pr-state-flow); [Definition of Ready & Done](/docs/concepts/definition-of-ready-and-done); [task-progress feedback (checklist ticks + one batched comment per run iteration)](/docs/developer-journey/execution#progress-feedback-ticks-and-one-comment) | | **Team** | Team structure, roles, communication patterns | | **Templates** | Branch, commit, PR, epic, story, task, initiative, subdomain, bounded context, code review, context map | diff --git a/package.json b/package.json index ddf80d3f8..7c0242325 100644 --- a/package.json +++ b/package.json @@ -35,7 +35,7 @@ "smoke-tests": "./scripts/smoke-tests/run-all.sh --cleanup", "format": "pnpm prettier:fix && pnpm mdlint:fix", "format:check": "pnpm prettier:check; _p=$?; pnpm mdlint:check; _m=$?; exit $((_p > 1 || _m > 1 ? 2 : (_p || _m)))", - "gate:composition": "pnpm --filter @pair/dev-tools pre-push-gate:check", + "gate:composition": "pnpm --filter @pair/dev-tools pre-push-gate:check && pnpm --filter @pair/dev-tools format-workflow:check", "workflows:test": "cd .claude/workflows && node --test" }, "engines": { diff --git a/packages/dev-tools/package.json b/packages/dev-tools/package.json index e6dc5c874..b2b677f9b 100644 --- a/packages/dev-tools/package.json +++ b/packages/dev-tools/package.json @@ -18,7 +18,8 @@ "sync-version": "ts-node src/quality-gates/sync-version-in-docs.ts", "benchmark-update-link": "ts-node src/quality-gates/benchmark-update-link.ts", "determine-version": "ts-node src/release/determine-version.ts", - "pre-push-gate:check": "ts-node src/quality-gates/pre-push-gate-composition.ts" + "pre-push-gate:check": "ts-node src/quality-gates/pre-push-gate-composition.ts", + "format-workflow:check": "ts-node src/quality-gates/format-workflow-composition.ts" }, "devDependencies": { "@pair/eslint-config": "workspace:*", @@ -30,7 +31,8 @@ "vite-tsconfig-paths": "catalog:", "@vitest/coverage-v8": "catalog:", "typescript": "catalog:", - "ts-node": "catalog:" + "ts-node": "catalog:", + "yaml": "catalog:" }, "prettier": "@pair/prettier-config" } diff --git a/packages/dev-tools/src/quality-gates/format-workflow-composition.test.ts b/packages/dev-tools/src/quality-gates/format-workflow-composition.test.ts new file mode 100644 index 000000000..063d59964 --- /dev/null +++ b/packages/dev-tools/src/quality-gates/format-workflow-composition.test.ts @@ -0,0 +1,4070 @@ +import { describe, it, expect } from 'vitest' +import { existsSync, readFileSync } from 'fs' +import { + checkFormatWorkflow, + checkThisRepoFormatWorkflow, + extractRunBlocks, + FORMAT_CHECK_SCRIPT, + FORMAT_WORKFLOW, +} from './format-workflow-composition' + +// #413. `pnpm format:check` was enforced by the husky pre-push hook and NOWHERE +// else: `--no-verify`, or a contributor whose hooks are not installed, landed +// unformatted code with every CI check green. The remedy is a dedicated +// `.github/workflows/format.yml` — and a YAML-only change has nothing asserting +// its shape unless a guard reads the real file. +// +// The failure modes this guard exists for are all one-line edits that keep the +// workflow LOOKING like enforcement: +// - a `paths-ignore:` key (the reason this is not a job inside ci.yml, whose +// workflow-level `paths-ignore: ['.changeset/**']` a job would inherit), or +// its allow-list twin `paths:`, which excludes everything it does not list, +// - a trigger narrowed off the base branch (`pull_request.branches: [release]`, +// or its negative spelling `branches-ignore: [main]`) or off the events that +// matter (`types: [closed]`), +// - `pull_request_target` instead of `pull_request` (a fork PR would then run +// with the base repo's credentials), +// - a write-mode formatter step (`pnpm format`), which would make CI rewrite +// files instead of reporting — the ADL 2026-07-31 ban is repo-wide, not +// hook-specific, +// - dropping the `push: main` trigger, so drift on the base branch goes unseen, +// - dropping `concurrency`, so a superseded run keeps reporting a stale verdict, +// - `continue-on-error: true`, ANY `if:` on the job, ANY `needs:` on the job (the +// same neutralization with no condition written anywhere: a job whose dependency +// fails or is skipped never runs, and a skipped job's required check reads +// SUCCESSFUL), ANY `if:` on the step that runs `format:check` (a `failure()` +// guard included — it is false on a normal PR, so the check is skipped and the +// job ends green), a step `if:` elsewhere that is not a SCOPED `failure()` guard, +// or a write-scoped token — each keeps the check green (or absent) while the +// context still reports, +// - dropping the failure-path remedy, so a red check names the offending file +// and nothing a contributor can act on (AC1) — or widening ANY failure-path +// step past the check step's own outcome, or naming that outcome without +// COMPARING it to `'failure'` (`== 'success'` is false exactly when the check +// failed), so a broken `pnpm install` is annotated "not formatted" or the remedy +// never fires at all, or placing a correctly-SPELLED scope where it cannot +// RESOLVE (a second job, or above the check step), so it is false on every run, +// or leaving the comparison exactly right but letting it DECIDE nothing — `&&` +// binds tighter than `||`, so `failure() && || ` fires the +// remedy on runs the check passed, +// - respelling a trigger as a FLOW mapping, which the block reader sees as an +// empty block and therefore as "no filter at all" — the spelling all four +// trigger holes above walk through untouched — or respelling a STEP the same +// way (`- { uses: creyD/prettier_action@v4 }`), which is a sequence ITEM and so +// is invisible to the key-level sweep, to `usesProblems` and to the write-mode +// scan at once, +// - a formatting action (`uses:`), invisible to a write scan that reads `run:` +// blocks, and needing no permission at all when placed before the check, +// - a checking command that is not THE command (`--filter=`, `-s`, a `cd`), so CI +// checks a strict subset of the tree the developer checks, +// - renaming the job that RUNS the check, which deletes the `format` status +// context branch protection is told to require — or leaving a decoy `format:` +// job behind that only echoes, so that context reports SUCCESS while the real +// check publishes one nobody requires — or renaming it through `name:` (GitHub +// publishes the DISPLAY name, id only when `name:` is absent) or suffixing it +// through a matrix (`format (20)`), or giving another job `name: format`, +// - a `push:` filtered by `tags:` alone, which GitHub fires for tag refs only — the +// workflow never runs on a push to `main` and `branchesOf` read "no filter" as +// "every branch", +// - a `concurrency.group` not keyed on `github.ref`, which puts a PR push and an +// in-progress run on `main` in ONE group and cancels the latter, +// - a `#` inside quotes read as a comment, which cuts an executing command out of +// the guard's view, +// - the checkout's `with:` — `ref: main` checks out `main` instead of the PR merge +// ref (measured: an unformatted PR, `format` SUCCESS), `sparse-checkout` a subset, +// - `working-directory:` on the checking step or `defaults:` anywhere — `cd` spelled +// as a key, so CI runs a package's own `format:check` the moment one declares it, +// - a remedy conjunct that narrows it to zero on the PR path (`&& github.event_name +// == 'push'`: measured, check fails and the remedy is skipped), +// - `cancel-in-progress` / `concurrency.group` matched as SUBSTRINGS, so `!(… == +// 'pull_request')` and `format-${{ github.run_id }}-${{ github.ref }}` passed, +// - an indentless block sequence or a filter-level alias reported as a DIFFERENT, +// false problem ("no branch") instead of read or named, +// - (round 13) the shell of the non-check, non-remedy steps left as the one deny-list +// (`git checkout origin/main -- .` before the check was green), `prettier -w` missing +// from the write list, and three correct spellings misreported (a quoted `run:`, +// CRLF line endings, workflow-level `permissions:`). +// - (round 16) `runs-on:` pointing off GitHub's runners. The job-key allow-list says +// WHICH keys a job may declare; `runs-on` is on it, so its VALUE was free — measured +// `ok=true` on the shipped file for `self-hosted` and for `[self-hosted, linux]`, one +// line relocating the check onto a machine the pull request chose (`container:` spelled +// as a value). +// The module header is the rule inventory; this list is the failure modes the suites +// below are named after. +// +// Structure is asserted, never exact file text: cosmetic YAML edits (comments, +// step names, action versions) must not false-fail this guard. + +/** + * A workflow shaped like the one #413 lands. Each test breaks exactly ONE thing + * in it by `replace`, so a failure names the property that regressed rather than + * "the fixture changed". + */ +const WELL_FORMED = `name: Format + +on: + pull_request: + branches: + - main + push: + branches: + - main + +concurrency: + group: format-\${{ github.ref }} + cancel-in-progress: true + +jobs: + format: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Checkout code + uses: actions/checkout@v4 + - name: Install pnpm + uses: pnpm/action-setup@v4 + with: + version: '10.15.0' + - name: Setup Node.js + uses: actions/setup-node@v4 + with: + node-version: '20' + cache: 'pnpm' + - name: Install dependencies + run: pnpm install + - name: Check formatting + id: format_check + run: pnpm format:check + - name: Explain how to fix it + if: failure() && steps.format_check.outcome == 'failure' + run: echo "::error::Not formatted. Run 'pnpm format' locally and commit the result." +` + +/** + * A mutation asserted to have HAPPENED, for every test that expects the guard to stay + * GREEN through it. + * + * A positive-path `.replace` that matches nothing passes vacuously: rename `id: + * format_check` in the workflow — an edit this guard permits, since `checkStepId` reads + * whatever id is there — and "accepts a quoted `id:`" silently re-runs the guard on the + * UNMUTATED file, still passes, and stops covering the `unquote` path that once made + * this guard RED on a correct workflow. Nothing goes red to say so. + * + * A NEGATIVE-path test cannot fail this way — an unmutated well-formed workflow is + * green, so `expect(ok).toBe(false)` catches the no-op replacement itself — which is why + * only the green ones route through here. + * + * Asserted as "the needle is still there" rather than `mutated !== source`, because one + * accepted spelling IS the shipped one (`uses: actions/checkout@v4` in the version + * table): that row replaces text with itself and must stay covered. + */ +function mutate(source: string, from: string | RegExp, to: string, label = String(from)): string { + expect(source, `the fixture no longer contains ${label}`).toMatch(from) + return source.replace(from, to) +} + +describe('extractRunBlocks reads what the workflow actually executes (#413)', () => { + it('collects an inline `run:` command', () => { + expect(extractRunBlocks(' steps:\n - run: pnpm format:check\n')).toEqual([ + 'pnpm format:check', + ]) + }) + + it('collects a block scalar body, not just its first line', () => { + const blocks = extractRunBlocks( + [ + ' - name: Check', + ' run: |', + ' set -e', + ' pnpm format:check', + ' - name: Next', + ' uses: actions/checkout@v4', + '', + ].join('\n'), + ) + expect(blocks).toEqual(['set -e\npnpm format:check']) + }) + + it('stops the block at the next key of the same or lower indent', () => { + const blocks = extractRunBlocks( + [ + ' - run: |', + ' pnpm format:check', + ' - uses: actions/checkout@v4', + '', + ].join('\n'), + ) + expect(blocks).toEqual(['pnpm format:check']) + }) + + it('ignores commented-out steps, so a comment cannot smuggle a command in', () => { + expect( + extractRunBlocks(' # - run: pnpm format\n - run: pnpm format:check\n'), + ).toEqual(['pnpm format:check']) + }) +}) + +describe('the format workflow closes the trigger-shaped holes (#413)', () => { + it('accepts the well-formed workflow', () => { + const r = checkFormatWorkflow(WELL_FORMED) + expect(r.ok, r.message).toBe(true) + }) + + // AC3. A `paths-ignore` is the hole this whole story exists to avoid: shipping a + // check whose TRIGGER excludes paths reads as enforcement and is not. + it('fails on a workflow-level `paths-ignore`', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' push:\n branches:\n - main', + ' push:\n branches:\n - main\n paths-ignore:\n - .changeset/**', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('paths-ignore') + }) + + it('fails on a `paths-ignore` nested under any trigger', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' pull_request:\n branches:\n - main', + ' pull_request:\n paths-ignore:\n - docs/**\n branches:\n - main', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('paths-ignore') + }) + + // AC3, allow-list twin. `paths:` is the same hole spelled positively: everything + // NOT listed is excluded. A markdown-only or `.changeset`-only PR would then run + // no formatting check at all — identical outcome to the `paths-ignore` above. + it('fails on a `paths:` allow-list under pull_request', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' pull_request:\n branches:', + " pull_request:\n paths: ['**/*.ts']\n branches:", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('paths') + }) + + it('fails on a `paths:` allow-list under push', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' push:\n branches:', + " push:\n paths: ['**/*.ts']\n branches:", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('paths') + }) + + // AC2, base-branch half. `branchesOf` was applied to `push` only, so retargeting + // the PR trigger at a branch nobody opens PRs against silenced the check for + // every real PR while the guard stayed green. + it('fails when `pull_request` no longer covers the base branch', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' pull_request:\n branches:\n - main', + ' pull_request:\n branches:\n - release', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('pull_request') + }) + + // AC2, base-branch NEGATIVE spelling. `branches-ignore` is the same filter written + // the other way round, and a missing filter correctly means "every branch" — so + // reading only `branches:` made this one-line edit invisible: no PR targeting main + // is format-checked and the `format` context simply never reports. + it('fails on a `branches-ignore` under pull_request', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' pull_request:\n branches:\n - main', + ' pull_request:\n branches-ignore:\n - main', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('branches-ignore') + }) + + it('fails on a `branches-ignore` under push', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' push:\n branches:\n - main', + ' push:\n branches-ignore: [main]', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('branches-ignore') + }) + + // Rejected outright rather than pattern-matched against `main`: the values are + // globs, so `ma*` excludes the base branch too and no substring test would see it. + it('fails on a `branches-ignore` that excludes the base branch by glob', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' pull_request:\n branches:\n - main', + ' pull_request:\n branches-ignore:\n - ma*', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('branches-ignore') + }) + + // AC7, the filter GitHub reads as a DIFFERENT ref kind. `branches` and `tags` are + // two independent filters on one event, and the producer's rule (GitHub docs, "events + // that trigger workflows" § push: "If you define only tags/tags-ignore or only + // branches/branches-ignore, the workflow won't run for events affecting the undefined + // Git ref") is measured on this repo: release.yml declares `push: tags: ['v*']` and + // nothing else, and every one of its `push` runs is a tag — none is `main`, while + // ci.yml ran on each of those days' pushes to `main`. So `push: tags:` with no + // `branches:` never runs on any push to `main`, and `branchesOf` returning null — + // "no filter, every branch" — was exactly wrong for it: the guard reported the + // workflow well-formed while post-merge drift went unseen. + it('fails on a `push` filtered by `tags:` alone', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' push:\n branches:\n - main', + " push:\n tags:\n - 'v*'", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('tags') + expect(r.message).toContain('never runs on a push to any branch') + }) + + it('fails on a `push` filtered by `tags-ignore:` alone', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' push:\n branches:\n - main', + " push:\n tags-ignore: ['v*']", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('tags-ignore') + }) + + // `tags:` is not a filter `pull_request` accepts at all; same fail-closed treatment, + // since either way no PR against `main` is checked. + it('fails on a `pull_request` filtered by `tags:` alone', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' pull_request:\n branches:\n - main', + " pull_request:\n tags:\n - 'v*'", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('tags') + }) + + // Both filters defined: the same producer rule says the event fires for EITHER ref + // kind, so `main` is still covered. Kept green so the rule rejects the hole, not the + // word. + it('accepts a `tags:` filter beside a `branches:` filter that covers the base branch', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' push:\n branches:\n - main', + " push:\n branches:\n - main\n tags:\n - 'v*'", + 'the `push:` trigger', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('accepts a `tags-ignore:` filter beside a `branches:` filter that covers the base branch', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' push:\n branches:\n - main', + " push:\n tags-ignore: ['v*']\n branches: [main]", + 'the `push:` trigger', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // A tag filter beside a branch filter that MISSES the base branch is still the + // off-base-branch hole, reported as that and not as a tag problem. + it('still reports the off-base-branch hole when a `tags:` filter sits beside it', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' push:\n branches:\n - main', + " push:\n branches:\n - release\n tags:\n - 'v*'", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('does not cover `main`') + }) + + // AC2, event half. `types: [closed]` runs the check only AFTER the PR is closed — + // never while it is reviewable. + it('fails when a `types:` narrowing drops opened/synchronize', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' pull_request:\n branches:', + ' pull_request:\n types: [closed]\n branches:', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('types') + }) + + it('accepts a `types:` list that still covers opened and synchronize', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' pull_request:\n branches:', + ' pull_request:\n types: [opened, synchronize, reopened, ready_for_review]\n branches:', + 'the `pull_request` trigger block', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // AC2. Without `pull_request` the check does not exist where it matters. + it('fails when the `pull_request` trigger is gone', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace(' pull_request:\n branches:\n - main\n', ''), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('pull_request') + }) + + // AC5. `pull_request_target` runs with the BASE repo's token against the fork's + // head — the classic fork-PR privilege escalation. One word from the safe form. + it('fails on `pull_request_target`, whatever else the file says', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace(' pull_request:', ' pull_request_target:')) + expect(r.ok).toBe(false) + expect(r.message).toContain('pull_request_target') + }) + + // AC7. Drift on the base branch must be visible, not only on pull requests. + it('fails when the `push` trigger is gone', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace(' push:\n branches:\n - main\n', '')) + expect(r.ok).toBe(false) + expect(r.message).toContain('push') + }) + + it('fails when `push` no longer covers the base branch', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' push:\n branches:\n - main', + ' push:\n branches:\n - release/*', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('main') + }) + + it('accepts the flow spelling of the branch filter (`branches: [main]`)', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' push:\n branches:\n - main', + ' push:\n branches: [main]', + 'the block-style `push` branch filter', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // Duplicate-run edge case: `push` + `pull_request` queue two runs of the same + // head, and the mitigation is a one-line block that is equally easy to drop. + it('fails when the concurrency guard is dropped', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + 'concurrency:\n group: format-${{ github.ref }}\n cancel-in-progress: true\n\n', + '', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('concurrency') + }) + + it('fails when concurrency no longer cancels the superseded run', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace('cancel-in-progress: true', 'cancel-in-progress: false'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('cancel-in-progress') + }) + + // Two merges to `main` a minute apart share `format-refs/heads/main`, so an + // unconditional cancel throws away the FIRST commit's verdict — AC7 wanted drift + // on the base branch visible. Cancelling only PR runs keeps both properties. + it('accepts a cancel-in-progress conditioned on the pull_request event', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'cancel-in-progress: true', + "cancel-in-progress: ${{ github.event_name == 'pull_request' }}", + '`cancel-in-progress: true`', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('fails on a cancel-in-progress expression naming no event at all', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + 'cancel-in-progress: true', + "cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('cancel-in-progress') + }) + + // The accepted-expression rule is an ALLOW-list of equality, not a substring test + // for `pull_request`: the NEGATION contains that substring and inverts the + // mitigation, producing BOTH failure modes the rule exists for — three pushes to a + // PR branch keep three runners alive on a stale verdict, and two merges to `main` a + // minute apart leave the first commit with no formatting verdict at all. + it('fails on a cancel-in-progress that NEGATES the pull_request event', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + 'cancel-in-progress: true', + "cancel-in-progress: ${{ github.event_name != 'pull_request' }}", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('cancel-in-progress') + }) + + // The `github.ref` keying is what the whole concurrency argument stands on — "the + // two triggers never meet" is only true because a PR run is `refs/pull//merge` and + // a push to main is `refs/heads/main`. Nothing read `group:`. So `group: format` (or + // `${{ github.workflow }}`) put EVERY run in one group: a `push` run on main in + // progress, any PR push then joins that group with `cancel-in-progress` true (the + // event is `pull_request`) and cancels main's run — that commit ends with no + // formatting verdict, the AC7 loss the conditional cancel exists to prevent — and two + // PRs pushed a minute apart cancel each other's verdict. Same-group cancellation is + // measured, not inferred: runs 33527856271 and 33528146034 of the shipped workflow + // are `cancelled` because a later push to the same PR ref joined their group. + it('fails when the concurrency group is a constant', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace('group: format-${{ github.ref }}', 'group: format'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('group: format`') + expect(r.message).toContain('github.ref') + }) + + it('fails when the concurrency group is keyed on the workflow name only', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace('group: format-${{ github.ref }}', 'group: ${{ github.workflow }}'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('github.ref') + }) + + // The token outside `${{ }}` is the literal string `github.ref`, i.e. a constant. + it('fails when `github.ref` is written outside an expression', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace('group: format-${{ github.ref }}', 'group: format-github.ref'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('github.ref') + }) + + // Allow-list of the canonical spelling, as for `cancel-in-progress`: a different + // context that happens to START with `github.ref` is not the same key. `ref_name` is + // `/merge` for a PR and the bare branch name for a push; `head_ref` is EMPTY on a + // push, so every push to main shares `format-` — and `sha` never groups two runs at + // all, so nothing is ever superseded. + it('fails on the near-miss contexts `github.ref_name`, `github.head_ref` and `github.sha`', () => { + for (const group of [ + 'group: format-${{ github.ref_name }}', + 'group: format-${{ github.head_ref }}', + 'group: format-${{ github.sha }}', + ]) { + const r = checkFormatWorkflow(WELL_FORMED.replace('group: format-${{ github.ref }}', group)) + expect(r.ok, group).toBe(false) + expect(r.message, group).toContain('github.ref') + } + }) + + it('fails when the concurrency block declares no group at all', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace(' group: format-${{ github.ref }}\n', '')) + expect(r.ok).toBe(false) + expect(r.message).toContain('group') + }) + + // Correct spellings — a workflow-distinguishing prefix (a constant or the workflow + // name) followed by the ref key or its documented `head_ref || ref` fallback, quoted + // or not — stay green. + it('accepts every group spelling keyed on github.ref', () => { + for (const group of [ + 'group: ${{ github.workflow }}-${{ github.ref }}', + 'group: format-${{ github.head_ref || github.ref }}', + 'group: "format-${{ github.ref }}"', + ]) { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, 'group: format-${{ github.ref }}', group, 'the concurrency group'), + ) + expect(r.ok, `${group}: ${r.message}`).toBe(true) + } + }) +}) + +// Nothing asserted a single JOB-level property, so the check could be made +// advisory, skipped outright, or handed a write-scoped token while every trigger +// and step rule above stayed green — and the `format` context would still report. +describe('the format job cannot be made advisory, skipped or privileged (#413)', () => { + it('fails on `continue-on-error: true`, which reports SUCCESS on unformatted code', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + ' - name: Check formatting\n continue-on-error: true\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('continue-on-error') + }) + + it('fails on an unconditionally false `if:`', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace(' format:\n runs-on:', ' format:\n if: false\n runs-on:'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + // The rule is an ALLOW-list, and this is why: a deny-list of literal falses waves + // through every never-true EXPRESSION, which is the spelling anyone would write. + // The job never runs on a PR, and a skipped required check reports neutral. + it('fails on a never-true job `if:` EXPRESSION, not just the literal false', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' format:\n runs-on:', + " format:\n if: github.event_name == 'workflow_dispatch'\n runs-on:", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + // `needs:` is the UNGUARDED spelling of the job `if:` above. A job whose dependency + // fails — or is itself skipped — never runs and is reported skipped, and on GitHub a + // skipped job reports its required check as SUCCESSFUL + // (github-implementation.md § Ordering). So once AC8 lists `format`, that context + // reads green and the merge goes through with the formatting check never having + // executed. Every other rule in this module stays green through it, exactly like the + // job-level `if:` it sits beside. + const PRECHECK = ` precheck: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Gate + run: exit 1 +` + + const dependencies: [string, string][] = [ + ['a scalar', ' needs: precheck\n'], + ['a flow sequence', ' needs: [precheck]\n'], + ['a block sequence', ' needs:\n - precheck\n'], + ] + + for (const [label, spelling] of dependencies) { + it(`fails when the job that runs the check is gated by \`needs:\` spelled as ${label}`, () => { + const r = checkFormatWorkflow( + `${WELL_FORMED.replace('jobs:\n', `jobs:\n${PRECHECK}`).replace( + ' format:\n runs-on: ubuntu-latest\n', + ` format:\n runs-on: ubuntu-latest\n${spelling}`, + )}`, + ) + expect(r.ok, `${label}: ${r.message}`).toBe(false) + expect(r.message, label).toContain('needs:') + }) + } + + it('fails on `needs:` even with no failing job to depend on — the gating is the loss', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' format:\n runs-on: ubuntu-latest\n', + ' format:\n runs-on: ubuntu-latest\n needs: setup\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('needs:') + }) + + // The worse half: the JOB runs on every PR, and only the one step that checks + // anything is skipped — so the `format` context reports SUCCESS on unformatted + // code while every other rule in the module stays green. + it('fails on an `if:` that skips the checking STEP while the job still reports', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + " - name: Check formatting\n if: github.event_name == 'push'\n", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + it('fails on an `if:` added to any other step, e.g. the install', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Install dependencies\n', + ' - name: Install dependencies\n if: false\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + // The hole an allow-list keyed on `failure()` ALONE still leaves open: the guard + // permitted any condition containing `failure()`, on ANY step — including the one + // step that checks anything. `if: failure()` there is never true on a normal PR + // (every earlier step succeeded), so `Check formatting` is SKIPPED, the job ends + // successful, and the `format` context reports SUCCESS on unformatted code. The + // check step therefore carries NO condition at all. + it('fails on `if: failure()` on the CHECK step itself', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + ' - name: Check formatting\n if: failure()\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + it('fails when `failure()` is ANDed onto a never-true event test on the check step', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + " - name: Check formatting\n if: github.event_name == 'push' && failure()\n", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + // Worse than the other two: a step referencing its OWN `steps..outcome` reads + // an unpopulated context, so `'' == 'failure'` is false on every event and the + // check never runs at all — while spelling out the exact scoping the guard asks + // the REMEDY for. + it('fails when the check step scopes itself on its own outcome', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + " - name: Check formatting\n if: failure() && steps.format_check.outcome == 'failure'\n", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + // One correctly-scoped step must not license every other one: `failure()` on a + // second annotation step is JOB-scoped, so a `pnpm install` dying on a lockfile + // drift annotates the Checks tab "not formatted" over the real cause — exactly + // the diagnosis the scoping rule exists to prevent, one step further out. + it('fails on a SECOND failure-path step left unscoped beside a scoped remedy', () => { + const r = checkFormatWorkflow( + `${WELL_FORMED} - name: Extra note + if: failure() + run: echo "::error title=Formatting check failed::not formatted" +`, + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('not scoped') + }) + + it('fails on `if: failure()` added to the install step beside a scoped remedy', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Install dependencies\n', + ' - name: Install dependencies\n if: failure()\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('not scoped') + }) + + // AC5 is "safe on fork PRs by construction, not by review". This job runs + // `pnpm install`, i.e. PR-authored lifecycle scripts; a write-scoped token in + // reach of that is the whole exposure. + it('fails on `permissions: write-all`', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' permissions:\n contents: read\n', + ' permissions: write-all\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('permissions') + }) + + it('fails on any write scope inside the permissions block', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace('contents: read', 'contents: write')) + expect(r.ok).toBe(false) + expect(r.message).toContain('permissions') + }) + + it('fails when the permissions block is deleted, so the repo default is inherited', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace(' permissions:\n contents: read\n', ''), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('permissions') + }) + + it('accepts the empty and read-all spellings of "no write scope"', () => { + for (const spelling of ['permissions: {}', 'permissions: read-all']) { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' permissions:\n contents: read', + ` ${spelling}`, + 'the block-style `permissions:`', + ), + ) + expect(r.ok, `${spelling}: ${r.message}`).toBe(true) + } + }) +}) + +// AC1: the failing check must name the offending file AND the remedy. Prettier's +// `--list-different` prints the file and suppresses its own "run with --write" +// line, so without a failure-path step the contributor this story exists for — +// hooks not installed, pushed with `--no-verify` — gets a bare filename. +describe('a failing format check tells the contributor what to run (#413)', () => { + it('fails when no failure-path step names the remedy', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + / {6}- name: Explain how to fix it\n {8}if: failure\(\).*\n {8}run: .*\n/, + '', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('pnpm format') + }) + + it('does not accept `pnpm format:check` as the remedy', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + 'echo "::error::Not formatted. Run \'pnpm format\' locally and commit the result."', + 'echo "::error::Not formatted. Run pnpm format:check locally."', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('pnpm format') + }) + + it('does not accept a remedy printed unconditionally on the success path', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace(/^ {8}if: failure\(\).*\n/m, '')) + expect(r.ok).toBe(false) + expect(r.message).toContain('pnpm format') + }) + + // `if: failure()` is JOB-scoped: it fires when ANY earlier step failed. A `pnpm + // install` broken by a lockfile drift or a registry outage would be annotated + // "not formatted. Run 'pnpm format'" — the contributor runs it, nothing changes, + // and the real cause is buried under a confident wrong diagnosis. + it('fails when the remedy is not scoped to the check step (a bare `if: failure()`)', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + " if: failure() && steps.format_check.outcome == 'failure'\n", + ' if: failure()\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('not scoped') + }) + + it('fails when the check step carries no `id:` to scope the remedy against', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace(' id: format_check\n', '')) + expect(r.ok).toBe(false) + expect(r.message).toContain('id:') + }) + + // A quoted id is valid YAML, prettier-stable, and `steps.format_check.outcome` + // resolves against it exactly the same on GitHub. Reading the scalar raw made the + // guard red on a CORRECT workflow, with a message describing a file that does + // declare a usable id — and a false-positive gate is the kind that gets deleted. + it('accepts a quoted `id:` on the check step', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, 'id: format_check', "id: 'format_check'", '`id: format_check`'), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('accepts `conclusion` as well as `outcome` for the scoping', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'steps.format_check.outcome', + 'steps.format_check.conclusion', + '`steps.format_check.outcome`', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // The reason this rule needs its own scanner: the write-mode guard reads the + // literal `pnpm format`, so the obvious spelling of the remedy was rejected as a + // write-mode STEP. A quoted message is data, not a command. + // + // Asserted on the REMEDY step, not the checking one: since AC4 became an equality on + // the checking step's command, `pnpm format:check || { … }` is rejected there for a + // different reason (it is no longer the one command a developer runs), and since round + // 13 the install step is an allow-list of toolchain commands. The remedy carries the + // required literal inside a quoted message, so the rule under test — quoted arguments + // are DATA — is what decides, and the checking-step spelling is asserted red just below. + it('does not mistake an echoed remedy for a step that writes files', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ` run: echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n`, + ' run: echo "Formatting failed. Run pnpm format and commit."\n', + 'the remedy step', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('rejects the inline `|| { … }` remedy on the checking step, as a command shape', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' run: pnpm format:check\n', + ' run: pnpm format:check || { echo "Formatting failed. Run pnpm format and commit."; exit 1; }\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('one command, two places') + }) + + it('still fails when `pnpm format` is actually RUN rather than quoted', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace(' run: pnpm format:check\n', ' run: pnpm format\n'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier:fix') + }) + + // A quoted string is only inert if it cannot execute: `$( … )` and backticks + // inside double quotes DO run. Those quotes stay in scope for the write scan. + it('still fails on a write hidden in a command substitution inside a quoted message', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' run: pnpm format:check\n', + ' run: echo "$(prettier --write .)" && pnpm format:check\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('does not let a quoted message satisfy the "CI runs format:check" rule', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' run: pnpm format:check\n', + ' run: echo "pnpm format:check"\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain(FORMAT_CHECK_SCRIPT) + }) +}) + +describe('the format workflow runs the same command a developer runs (#413)', () => { + // AC4. Local/CI parity is the point of the story: a CI-only variant (a flag, a + // path list, a re-implementation) recreates the divergence it closes. + it('fails when no step invokes `pnpm format:check`', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace('run: pnpm format:check', 'run: pnpm lint')) + expect(r.ok).toBe(false) + expect(r.message).toContain(FORMAT_CHECK_SCRIPT) + }) + + it('does not accept a step that merely NAMES the script', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace('run: pnpm format:check', 'run: echo format:check'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain(FORMAT_CHECK_SCRIPT) + }) + + it('accepts `pnpm run format:check`, the one other spelling of the same invocation', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'run: pnpm format:check', + 'run: pnpm run format:check', + 'the checking command', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // The checking step's command is an ALLOW-list of two spellings, not "references + // the script". Every row below still satisfies `referencesScript` — the old rule — + // and every row makes CI check a strict SUBSET of what the developer's and the + // hook's whole-repo `pnpm format:check` covers, which is the divergence this story + // exists to close. `-s` belongs here too: it silences the output, i.e. the list of + // offending filenames AC1 requires the contributor to read. + it('fails on any narrowed spelling of the checking command', () => { + // Two rules cover the table. A spelling `referencesScript` still recognises as an + // invocation of the script is caught by the equality ("one command, two places"); + // one whose flag takes a SPACE-separated value is not recognised as an invocation + // at all, so the older "no step RUNS" rule catches it. Both are red, both name the + // local/CI divergence — the fragment column records which fired, so a later change + // that moves a row between them is visible rather than silent. + const narrowed: [string, string][] = [ + ['pnpm --filter=@pair/website format:check', 'one command, two places'], + ['pnpm -F @pair/website format:check', 'no step RUNS'], + ['pnpm -C apps/website format:check', 'no step RUNS'], + ['pnpm -s format:check', 'one command, two places'], + ['cd apps/website && pnpm format:check', 'one command, two places'], + ['npm run format:check', 'one command, two places'], + ['pnpm format:check --ignore-path .prettierignore.ci', 'one command, two places'], + ] + for (const [spelling, fragment] of narrowed) { + const r = checkFormatWorkflow( + WELL_FORMED.replace('run: pnpm format:check', `run: ${spelling}`), + ) + expect(r.ok, `${spelling}: ${r.message}`).toBe(false) + expect(r.message, spelling).toContain(fragment) + expect(r.message, spelling).toContain('divergence') + } + }) + + it('fails when the checking step wraps the command in a multi-line block scalar', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' run: pnpm format:check\n', + ' run: |\n set -e\n pnpm format:check\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('one command, two places') + }) + + // AC6. Check-only holds in CI exactly as it does in the hook (ADL 2026-07-31). + // Reuses the pre-push guard's offender list rather than a second copy of it. + it('fails when a step writes instead of checking (`pnpm format`)', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace('pnpm format:check', 'pnpm format')) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier:fix') + }) + + it('fails on a raw write-mode formatter appended to the check', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace('pnpm format:check', 'pnpm format:check || prettier --write .'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('fails on an auto-commit step, so CI can never repair the branch', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' run: pnpm format:check\n', + ' run: |\n pnpm format\n git commit -am "chore: format"\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier:fix') + }) +}) + +describe('the format workflow is safe on a fork PR by construction (#413)', () => { + // AC5. No secret in the job means a fork run is a FULL-STRENGTH run, and there + // is no credential for PR-authored code to reach. + it('fails when any step reads a secret', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting', + ' - name: Check formatting\n env:\n TOKEN: ${{ secrets.GITHUB_TOKEN }}', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('secret') + }) + + // The classic script-injection sink: attacker-controlled text (a PR title, a + // branch name) expanded by the runner INTO the shell before it runs. + it('fails when an expression is interpolated into a `run:` block', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + 'run: pnpm format:check', + 'run: echo "${{ github.event.pull_request.title }}" && pnpm format:check', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('${{') + }) + + it('leaves an expression OUTSIDE a run block alone (the concurrency group)', () => { + // `group: format-${{ github.ref }}` is not a shell sink — banning it would + // ban the duplicate-run mitigation this same guard requires. + expect(checkFormatWorkflow(WELL_FORMED).ok).toBe(true) + }) +}) + +// YAML has TWO spellings for every mapping, and this guard's block reader only +// understands one of them. `blockUnder` collects the lines indented deeper than a +// key, so a FLOW mapping on the same line yields an EMPTY block — and an empty +// block is read by `listValueOf` as "the key is absent", which for a trigger filter +// means "no filter, therefore every value". Every trigger rule then passes +// vacuously on a workflow whose trigger is exactly as narrow as the rule forbids. +// The complete table below is the decision table for the structural keys this guard +// reads as blocks: each has a flow spelling GitHub honours, and each is rejected +// rather than parsed — a hand-rolled block reader that pretends to understand flow +// style is a second hole, not a fix. +// Rounds 5–6 REJECTED every spelling the hand-rolled line reader could not follow; the +// migration to `yaml@2.8.2` (ADL 2026-09-01, amended 2026-09-03) deletes that whole rule +// family, because the parser resolves those spellings to the same document GitHub runs. +// So the contract inverts: a flow mapping, a JSON step, an anchor, an alias — all legal +// YAML GitHub honours — are READ, and what decides is the SEMANTIC rule on the resolved +// value. Correct workflow, exotic spelling: GREEN. Hole, whatever the spelling: RED, and +// named by its cause. +describe('a flow-style spelling is READ, and the semantic rule decides (#413)', () => { + // Same document as WELL_FORMED, spelled in flow/JSON style throughout — the spelling + // the reader used to reject wholesale, on every structural key at once. + const WELL_FORMED_FLOW = `name: Format +"on": { pull_request: { branches: [main] }, push: { branches: [main] } } +concurrency: { group: "format-\${{ github.ref }}", cancel-in-progress: true } +jobs: + format: + { + runs-on: ubuntu-latest, + permissions: { contents: read }, + steps: + [ + { name: Checkout code, uses: actions/checkout@v4 }, + { name: Install pnpm, uses: pnpm/action-setup@v4, with: { version: "10.15.0" } }, + { name: Install dependencies, run: pnpm install }, + { name: Check formatting, id: format_check, run: pnpm format:check }, + { + name: Explain how to fix it, + if: "failure() && steps.format_check.outcome == 'failure'", + run: "echo \\"::error::Not formatted. Run 'pnpm format' locally and commit the result.\\"", + }, + ], + } +` + + it('accepts a workflow spelled entirely in flow style', () => { + const r = checkFormatWorkflow(WELL_FORMED_FLOW) + expect(r.ok, r.message).toBe(true) + }) + + // Each row is a CORRECT workflow wearing the spelling round 5 rejected by name. + const legalSpellings: [string, string, string][] = [ + [ + '`on:` itself as a flow mapping', + 'on:\n pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + 'on: { pull_request: { branches: [main] }, push: { branches: [main] } }\n', + ], + [ + '`on:` as a flow SEQUENCE of event names (no filter is a SUPERSET of `main`)', + 'on:\n pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + 'on: [pull_request, push]\n', + ], + [ + '`on:` as a block LIST of event names', + 'on:\n pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + 'on:\n - pull_request\n - push\n', + ], + [ + '`pull_request` as a flow mapping', + ' pull_request:\n branches:\n - main\n', + ' pull_request: { branches: [main] }\n', + ], + [ + '`pull_request` as a flow mapping spanning two lines', + ' pull_request:\n branches:\n - main\n', + ' pull_request: {\n branches: [main] }\n', + ], + [ + '`push` as a flow mapping', + ' push:\n branches:\n - main\n', + ' push: { branches: [main] }\n', + ], + [ + '`concurrency` as a flow mapping', + 'concurrency:\n group: format-${{ github.ref }}\n cancel-in-progress: true\n', + 'concurrency: { group: "format-${{ github.ref }}", cancel-in-progress: true }\n', + ], + [ + 'the checking step as a JSON-spelled sequence item', + ' - name: Check formatting\n id: format_check\n run: pnpm format:check\n', + ' - { "name": "Check formatting", "id": "format_check", "run": "pnpm format:check" }\n', + ], + ] + + for (const [label, block, flow] of legalSpellings) { + it(`accepts ${label}`, () => { + const mutated = mutate(WELL_FORMED, block, flow, label) + const r = checkFormatWorkflow(mutated) + expect(r.ok, `${label}: ${r.message}`).toBe(true) + }) + } + + // …and the four holes that spelling used to hide are still RED — each now named by + // the SEMANTIC rule that owns it, not by its punctuation. + const flowHoles: [string, string, string, string][] = [ + [ + 'a flow `paths-ignore`', + ' pull_request:\n branches:\n - main\n', + " pull_request: { branches: [main], paths-ignore: ['**/*.md'] }\n", + 'paths-ignore', + ], + [ + 'a flow trigger off the base branch', + ' pull_request:\n branches:\n - main\n', + ' pull_request: { branches: [release] }\n', + 'does not cover `main` (release)', + ], + [ + 'a flow `types: [closed]`', + ' pull_request:\n branches:\n - main\n', + ' pull_request: { branches: [main], types: [closed] }\n', + 'dropping opened, synchronize', + ], + [ + 'a flow `push` off the base branch', + ' push:\n branches:\n - main\n', + ' push: { branches: [release] }\n', + 'does not cover `main` (release)', + ], + [ + 'a flow `concurrency.group` that is not keyed on the ref', + 'concurrency:\n group: format-${{ github.ref }}\n cancel-in-progress: true\n', + 'concurrency: { group: format, cancel-in-progress: true }\n', + 'github.ref', + ], + ] + + for (const [label, block, flow, cause] of flowHoles) { + it(`still fails on ${label}, naming the cause`, () => { + const r = checkFormatWorkflow(mutate(WELL_FORMED, block, flow, label)) + expect(r.ok, `${label}: ${r.message}`).toBe(false) + expect(r.message, label).toContain(cause) + expect(r.message, label).not.toContain('flow-style') + }) + } + + // A step is a sequence ITEM. Every non-block spelling of one used to walk past + // `usesProblems` AND the write-mode scan at once; the parser resolves each to the + // same mapping, so both rules now see it. Measured `ok=true` on the shipped file + // before the migration, RED here for the reason the step actually is one. + describe('a step spelled as a flow or JSON item is read by every step rule', () => { + const items: [string, string, string][] = [ + [ + 'a flow mapping running a formatter', + '- { name: Fix, run: npx prettier --write . }', + 'prettier --write', + ], + ['a flow mapping with no spaces', '- {run: prettier --write .}', 'prettier --write'], + [ + 'the JSON spelling of the same', + '- { "name": "Fix", "run": "npx prettier --write ." }', + 'prettier --write', + ], + ['a flow mapping using a formatting ACTION', '- { uses: creyD/prettier_action@v4 }', 'uses:'], + [ + 'a flow mapping using an auto-commit action', + '- { uses: stefanzweifel/git-auto-commit-action@v5 }', + 'uses:', + ], + [ + 'an anchored item whose first key shares the line', + '- &fixer { run: npx prettier --write . }', + 'prettier --write', + ], + [ + 'a bare dash, with the node on the NEXT line', + '-\n { run: npx prettier --write . }', + 'prettier --write', + ], + ] + + for (const [label, item, cause] of items) { + it(`fails on ${label}`, () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + ` ${item}\n - name: Check formatting\n`, + ), + ) + expect(r.ok, `${label}: ${r.message}`).toBe(false) + expect(r.message, label).toContain(cause) + }) + } + + // A `steps:` item that is not a mapping carries no step key at all — and GitHub + // refuses to run the file (probe run 33724280781 on PR #477: zero jobs). Rejected + // here rather than filtered out silently. + it('fails on a sequence item that is not a mapping', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + ' - [a, b]\n - name: Check formatting\n', + ), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('not mappings') + }) + + // Over-reach: an extra block-mapping step and a second branch scalar stay green. + it('leaves a block-mapping step and a scalar list item alone', () => { + const extraStep = mutate( + WELL_FORMED, + ' - name: Check formatting\n', + ' - name: Enable corepack\n run: corepack enable\n - name: Check formatting\n', + 'the checking step', + ) + expect(checkFormatWorkflow(extraStep).ok, checkFormatWorkflow(extraStep).message).toBe(true) + + const twoBranches = mutate( + WELL_FORMED, + ' push:\n branches:\n - main\n', + " push:\n branches:\n - main\n - 'release'\n", + 'the `push` branch filter', + ) + expect(checkFormatWorkflow(twoBranches).ok, checkFormatWorkflow(twoBranches).message).toBe( + true, + ) + }) + + // …and shell text inside a `run:` block scalar is not YAML at all: the parser hands + // the body over as a string, so a brace expansion, a `[` test or a `*` glob inside a + // quoted message is shell and never a sequence item or an alias. + it('does not read a `run:` block scalar body as YAML structure', () => { + const shell = mutate( + WELL_FORMED, + ` run: echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n`, + ` run: | + echo "::error::Not formatted. Run 'pnpm format' locally and commit the result." + echo "Lines like - { a,b } and - [ x ] and * are shell here, not YAML." +`, + 'the remedy step', + ) + expect(checkFormatWorkflow(shell).ok, checkFormatWorkflow(shell).message).toBe(true) + expect(extractRunBlocks(shell).some(run => run.includes('- { a,b }'))).toBe(true) + }) + }) + + // Over-reach guard: the spellings the guard always read correctly stay green. + it('leaves the flow spellings the guard does read correctly alone', () => { + const flowBranches = mutate( + WELL_FORMED, + ' pull_request:\n branches:\n - main\n', + ' pull_request:\n branches: [main]\n', + 'the `pull_request` branch filter', + ) + expect(checkFormatWorkflow(flowBranches).ok, checkFormatWorkflow(flowBranches).message).toBe( + true, + ) + + const inlinePermissions = mutate( + WELL_FORMED, + ' permissions:\n contents: read\n', + ' permissions: { contents: read }\n', + 'the block-style `permissions:`', + ) + expect( + checkFormatWorkflow(inlinePermissions).ok, + checkFormatWorkflow(inlinePermissions).message, + ).toBe(true) + }) +}) + +// The parse is the fail-closed boundary the rejection list used to be. `yaml@2.8.2` +// refuses a document it cannot resolve unambiguously, and so does GitHub — measured on +// PR #477: a merge-keyed job (run 33724280781) and an unknown top-level key (run +// 33724281525) each produced a run with ZERO jobs, i.e. "invalid workflow file". +describe('a file the parser refuses is a problem, not a pass (#413)', () => { + const unparseable: [string, string][] = [ + ['a duplicate key', 'name: Format\nname: Format\n'], + ['a tab indent', 'on:\n\tpush:\n'], + ['an alias with no anchor', 'on: *nowhere\n'], + ['unbalanced flow punctuation', 'on: { push: { branches: [main] }\n'], + ] + + for (const [label, text] of unparseable) { + it(`rejects ${label}`, () => { + const r = checkFormatWorkflow(text) + expect(r.ok, `${label}: ${r.message}`).toBe(false) + expect(r.message, label).toContain('not valid YAML') + }) + } + + it('rejects a document that is not a mapping of workflow keys', () => { + const r = checkFormatWorkflow('- pull_request\n- push\n') + expect(r.ok).toBe(false) + expect(r.message).toContain('mapping of workflow keys') + }) + + it('rejects a merge key, which GitHub refuses to run at all', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'jobs:\n format:\n', + 'jobs:\n base: &base\n runs-on: ubuntu-latest\n format:\n <<: *base\n', + 'the jobs block', + ), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('`<<`') + }) +}) + +// AC6, the `uses:` half. The write-mode scan reads `run:` blocks only, so a step +// that writes through an ACTION was invisible to it. Placed before the checking +// step, a formatting action rewrites the runner's checkout and `pnpm format:check` +// then passes on unformatted code with the `format` context green — and it needs no +// permission at all to do it, because it never pushes. +describe('a step may only use an allow-listed action (#413)', () => { + const banned = [ + 'creyD/prettier_action@v4', + 'stefanzweifel/git-auto-commit-action@v5', + 'EndBug/add-and-commit@v9', + './.github/actions/format-fixer', + 'docker://alpine:3', + 'Actions/Checkout-Extra@v1', + ] + + for (const action of banned) { + it(`fails on \`uses: ${action}\``, () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' - name: Check formatting\n', + ` - name: Fix\n uses: ${action}\n with:\n prettier_options: --write .\n - name: Check formatting\n`, + ), + ) + expect(r.ok, `${action}: ${r.message}`).toBe(false) + expect(r.message).toContain('uses:') + }) + } + + it('accepts the three actions the workflow needs, at any version and quoted', () => { + for (const action of [ + 'actions/checkout@v4', + 'actions/checkout@v5', + "'actions/checkout@v4'", + 'actions/checkout@a1b2c3d4e5f6a7b8c9d0a1b2c3d4e5f6a7b8c9d0', + 'ACTIONS/CHECKOUT@v4', + ]) { + // One row here IS the shipped spelling, so this mutation is deliberately an + // identity for it — `mutate` asserts the NEEDLE, not that the text changed. + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'uses: actions/checkout@v4', + `uses: ${action}`, + '`uses: actions/checkout@v4`', + ), + ) + expect(r.ok, `${action}: ${r.message}`).toBe(true) + } + }) +}) + +// `stripComments` cut from the first ` #` to end of line unconditionally. Inside a +// `run:` block scalar — and inside a QUOTED YAML scalar — that `#` may sit inside +// quotes, where neither bash nor YAML treats it as a comment. The truncation was +// documented as "a comment cannot smuggle a banned pattern IN"; it also smuggled a +// real, executing command OUT of the guard's view, which is the direction that costs +// the AC6 ban. +describe('a `#` inside quotes is not a comment (#413)', () => { + it('sees a write-mode formatter hidden behind a quoted `#`', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' run: pnpm install\n', + ' run: |\n pnpm install\n echo "note # here"; prettier --write .\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('sees it behind single quotes too', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' run: pnpm install\n', + " run: |\n pnpm install\n echo 'note # here'; prettier --write .\n", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('still strips a real shell comment inside a block scalar', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' run: pnpm install\n', + ' run: |\n pnpm install # prettier --write .\n', + 'the install step', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('still strips a whole-line YAML comment, so a comment cannot smuggle a command in', () => { + expect( + extractRunBlocks(' # - run: pnpm format\n - run: pnpm format:check\n'), + ).toEqual(['pnpm format:check']) + }) +}) + +// The job's display name (its id when `name:` is absent) IS the status context. Nothing +// asserted it, so renaming `format:` to `fmt:` left the guard green while the context way-of-working documents — and that +// AC8 names for branch protection — silently stopped existing. In advisory mode that +// is no signal at all; once protection lists it, a required context that never +// reports leaves every PR pending with no escape hatch +// (github-implementation.md:857). +describe('the job that reports the `format` context is named (#413)', () => { + it('fails when the job is renamed', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace('\n format:\n', '\n fmt:\n')) + expect(r.ok).toBe(false) + expect(r.message).toContain('format') + expect(r.message).toContain('status context') + }) + + it('fails on a case variant, since the context name is case-sensitive', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace('\n format:\n', '\n Format:\n')) + expect(r.ok).toBe(false) + expect(r.message).toContain('status context') + }) + + it('accepts the quoted spelling of the same job id', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, '\n format:\n', "\n 'format':\n", 'the `format:` job header'), + ) + expect(r.ok, r.message).toBe(true) + }) + + // "Some job is named `format`" is satisfied by a DECOY. Keep `format:` with one + // `run: echo ok` step, move the real steps into `worker:`, and the `format` context — + // the one way-of-working documents and AC8 tells branch protection to list — reports + // SUCCESS after an echo, while the job that actually checks anything publishes a + // `worker` context nobody requires. Same loss as the plain rename, and worse: the + // rename goes red, this used to stay green. So the assertion is on the HOST job. + it('fails on a decoy `format` job while another job runs the check', () => { + const decoy = WELL_FORMED.replace( + '\n format:\n', + ` + format: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Nothing + run: echo ok + worker: +`, + ) + const r = checkFormatWorkflow(decoy) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('is `worker`, not `format`') + }) + + // The same shape without the decoy: the host is renamed and no `format` job exists + // at all. One accurate problem, not the old name-set message. + it('names the host job when it is renamed and nothing else claims the context', () => { + const r = checkFormatWorkflow(WELL_FORMED.replace('\n format:\n', '\n worker:\n')) + expect(r.ok).toBe(false) + expect(r.message).toContain('is `worker`, not `format`') + }) + + // A second job beside a correctly-named host is not the loss — the context still + // belongs to the job that checks. (Its shell is an allow-listed inert echo: the + // toolchain allow-list applies to every non-check, non-remedy step in every job.) + it('accepts an extra job beside a `format` host that runs the check', () => { + const extra = mutate( + WELL_FORMED, + '\n format:\n', + ` + notes: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Nothing + run: echo "ok" + format: +`, + 'the `format:` job header', + ) + expect(checkFormatWorkflow(extra).ok, checkFormatWorkflow(extra).message).toBe(true) + }) + + // No job runs the check at all: `stepProblems` owns that cause, and this rule falls + // back to the name-set assertion so a missing context is still reported. + it('still reports the missing `format` job when no job runs the check', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace('\n format:\n', '\n worker:\n').replace( + ' run: pnpm format:check\n', + ' run: echo skip\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('no job is named `format`') + }) + + // GitHub publishes the job's DISPLAY NAME as the check context, not its id. Measured + // on this repo: version.yml's job id `version` carries `name: Create version commits + // and tags`, and `gh run view 32579550290 --json jobs` reports the job as `Create + // version commits and tags`. So one `name:` line renames the `format` context the + // same way `fmt:` does — rename red, `name:` was green. + it('fails when the host job carries a `name:` that is not `format`', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' format:\n runs-on: ubuntu-latest\n', + ' format:\n name: Formatting\n runs-on: ubuntu-latest\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('name: Formatting') + expect(r.message).toContain('display name') + }) + + it('fails when the host job name is an expression, whatever it evaluates to', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' format:\n runs-on: ubuntu-latest\n', + ' format:\n name: ${{ github.workflow }}\n runs-on: ubuntu-latest\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('display name') + }) + + // A `name:` equal to the id publishes the same context; nothing is lost. + it('accepts a `name:` on the host job that spells the same context', () => { + for (const name of ['name: format', "name: 'format'"]) { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' format:\n runs-on: ubuntu-latest\n', + ` format:\n ${name}\n runs-on: ubuntu-latest\n`, + 'the `format:` job header', + ), + ) + expect(r.ok, `${name}: ${r.message}`).toBe(true) + } + }) + + // A matrix appends its values to the display name: actions/checkout's job id + // `analyze` with `name: Analyze` and `matrix.language: ['javascript']` is published as + // `Analyze (javascript)` (run 33304315280). `format` would become `format (20)` and + // stop existing — every PR pending once protection lists it, with no escape hatch. + it('fails when the host job carries a `strategy:` (matrix)', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' format:\n runs-on: ubuntu-latest\n', + " format:\n strategy:\n matrix:\n node: ['20']\n runs-on: ubuntu-latest\n", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('strategy') + expect(r.message).toContain('format (') + }) + + // `strategy:` without a `matrix:` publishes no suffix today; rejected all the same, + // fail-closed — a strategy block exists to carry a matrix. + it('fails on a `strategy:` block without a matrix, fail-closed', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + ' format:\n runs-on: ubuntu-latest\n', + ' format:\n strategy:\n fail-fast: false\n runs-on: ubuntu-latest\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('strategy') + }) + + // The decoy, spelled through the display name: a second job whose `name:` is + // `format` publishes a SECOND `format` context after an `echo`, beside the real one. + it('fails when another job takes the `format` display name', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + '\n format:\n', + ` + notes: + name: format + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Nothing + run: echo ok + format: +`, + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('`notes`') + expect(r.message).toContain('display name') + }) + + // A step's `name:` sits one level deeper and is not the job's display name. + it('does not mistake a step `name:` for the job display name', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' - name: Check formatting\n', + ' - name: Formatting\n', + 'the check step name', + ), + ) + expect(r.ok, r.message).toBe(true) + }) +}) + +// `steps.` is JOB-LOCAL and populated only for steps that have already run. A +// failure-path step that names it from another job, or from above the checking step, +// carries a condition that is false on every run — so the remedy never fires, and +// AC1's contributor reads a bare filename with the guard reporting the workflow +// well-formed. +describe('a scoped failure-path step must be able to resolve its scope (#413)', () => { + const REMEDY = + " - name: Explain how to fix it\n if: failure() && steps.format_check.outcome == 'failure'\n run: echo \"::error::Not formatted. Run 'pnpm format' locally and commit the result.\"\n" + + it('fails when the remedy lives in a second job', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace( + REMEDY, + ` explain: + runs-on: ubuntu-latest + permissions: + contents: read + steps: +${REMEDY}`, + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('job-local') + }) + + it('fails when the remedy sits ABOVE the checking step in the same job', () => { + const r = checkFormatWorkflow( + WELL_FORMED.replace(REMEDY, '').replace( + ' - name: Check formatting\n', + `${REMEDY} - name: Check formatting\n`, + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('has not run yet') + }) + + it('accepts two correctly-placed scoped remedies', () => { + const r = checkFormatWorkflow(mutate(WELL_FORMED, REMEDY, REMEDY + REMEDY, 'the remedy step')) + expect(r.ok, r.message).toBe(true) + }) +}) + +// Naming `steps..outcome` and COMPARING it are two different things, and only the +// second one scopes anything. `steps..outcome` holds one of four values — `success`, +// `failure`, `cancelled`, `skipped` — so the reference is a substring of every condition +// that reads it, including the ones that are false exactly when the check fails. +// +// The table below is that value domain, both operators, both operand orders and both +// quote styles. The decisive row is `== 'success'`: a PR carries an unformatted file, +// `Check formatting` fails, `failure()` is true — but `outcome` is `'failure'`, so the +// remedy is SKIPPED on the one run that needed it and the contributor reads +// `--list-different`'s bare filename with prettier's own "--write to fix" hint +// suppressed. That is AC1's exact loss from a one-token edit. +describe('a failure-path scope must resolve on the FAILURE path (#413)', () => { + const SCOPE = "steps.format_check.outcome == 'failure'" + + const accepted: [string, string][] = [ + ['the shipped spelling', "steps.format_check.outcome == 'failure'"], + ['`conclusion` instead of `outcome`', "steps.format_check.conclusion == 'failure'"], + ['double quotes around the value', 'steps.format_check.outcome == "failure"'], + ['no spaces around the operator', "steps.format_check.outcome=='failure'"], + ['extra spaces around the operator', "steps.format_check.outcome == 'failure'"], + ['the operands reversed', "'failure' == steps.format_check.outcome"], + [ + 'the whole condition wrapped in an expression', + "${{ failure() && steps.format_check.outcome == 'failure' }}", + ], + ] + + for (const [label, condition] of accepted) { + it(`accepts ${label}`, () => { + // The wrapped row already carries its own `failure()`; the others are ANDed onto one. + const spelled = condition.startsWith('${{') ? condition : `failure() && ${condition}` + const r = checkFormatWorkflow( + mutate(WELL_FORMED, `failure() && ${SCOPE}`, spelled, 'the scoped remedy condition'), + ) + expect(r.ok, `${label}: ${r.message}`).toBe(true) + }) + } + + // Every other value in the domain, both operators, plus the two shapes that name the + // context without comparing it at all. Each leaves the reference intact — the whole + // point: a substring test for `steps..outcome` reports all of them well-formed. + const rejected: [string, string, string][] = [ + [ + "== 'success' — false exactly when the check failed, so the remedy never fires on a red check", + "steps.format_check.outcome == 'success'", + 'AC1', + ], + [ + "== 'skipped' — fires only when the check never ran, i.e. on the broken `pnpm install`", + "steps.format_check.outcome == 'skipped'", + 'wrong diagnosis', + ], + [ + "== 'cancelled' — fires on a superseded run and on nothing else", + "steps.format_check.outcome == 'cancelled'", + 'AC1', + ], + ["!= 'failure' — the scope inverted", "steps.format_check.outcome != 'failure'", 'AC1'], + [ + "!= 'success' — true for `failure`, `skipped` AND `cancelled`, i.e. the unscoped remedy back", + "steps.format_check.outcome != 'success'", + 'wrong diagnosis', + ], + [ + "conclusion == 'success' — the same inversion on the other status field", + "steps.format_check.conclusion == 'success'", + 'AC1', + ], + [ + 'a bare reference with no comparison, which is truthy for all four values', + 'steps.format_check.outcome', + 'no comparison', + ], + [ + '`contains()` instead of the equality this allow-list accepts', + "contains(steps.format_check.outcome, 'failure')", + 'not canonical', + ], + ] + + for (const [label, condition] of rejected) { + it(`fails on ${label}`, () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + `failure() && ${SCOPE}`, + `failure() && ${condition}`, + 'the scoped remedy condition', + ), + ) + expect(r.ok, `${condition}: ${r.message}`).toBe(false) + expect(r.message, condition).toContain("== 'failure'") + }) + } + + // The reference is still there, so the "not scoped at all" message would be the wrong + // cause to report: the author DID scope it, to the wrong value. + it('names the comparison, not the missing reference, when the value is wrong', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + `failure() && ${SCOPE}`, + "failure() && steps.format_check.outcome == 'success'", + 'the scoped remedy condition', + ), + ) + expect(r.message).toContain('compare it to') + expect(r.message).not.toContain('are not scoped to the formatting check') + }) +}) + +// Requiring the equality to be PRESENT is the third leg; requiring it to DECIDE is the +// fourth. `if:` is a boolean expression, and in GitHub's grammar `&&` binds tighter than +// `||`, so `failure() && || ` parses as `(failure() && ) || ` +// — the scope is still there, spelled exactly as shipped, and decides nothing. +describe('a failure-path scope must DECIDE, not merely appear (#413)', () => { + const SCOPE = "steps.format_check.outcome == 'failure'" + const remedyOf = (condition: string) => + mutate(WELL_FORMED, `failure() && ${SCOPE}`, condition, 'the scoped remedy condition') + + // Every one of these keeps `steps.format_check.outcome == 'failure'` intact, so + // `scopesTo` reports them scoped — and every one of them fires the remedy on a run the + // check did not fail on, which is round 3's loss (a broken `pnpm install` annotated + // "not formatted. Run `pnpm format`") restored by adding tokens the guard never reads. + const neutralized: [string, string][] = [ + [ + '`|| true`, which makes the whole condition unconditionally true', + `failure() && ${SCOPE} || true`, + ], + [ + '`|| github.event_name ==` …, true on every push regardless of the check', + `failure() && ${SCOPE} || github.event_name == 'push'`, + ], + [ + "another step's outcome ORed in — the install failure this scope exists to stay quiet about", + `failure() && ${SCOPE} || steps.install.outcome == 'failure'`, + ], + [ + 'the same disjunction parenthesised, so the scope is one disjunct of two', + `failure() && (${SCOPE} || steps.install.outcome == 'failure')`, + ], + ['`always()` ORed in front of the whole guard', `always() || failure() && ${SCOPE}`], + ['the scope negated as a group', `failure() && !(${SCOPE})`], + [ + "the status reference negated, which compares `false` to `'failure'` and is never true", + `failure() && !steps.format_check.outcome == 'failure'`, + ], + ['the neutralised condition wrapped in an expression', `\${{ failure() && ${SCOPE} || true }}`], + ] + + for (const [label, condition] of neutralized) { + it(`fails on ${label}`, () => { + const r = checkFormatWorkflow(remedyOf(condition)) + expect(r.ok, `${condition}: ${r.message}`).toBe(false) + expect(r.message, condition).toContain('decides nothing') + }) + } + + // "A conjunction can only narrow" was the round-8 premise, and narrowing to ZERO on + // the PR path IS the AC1 loss: `&& github.event_name == 'push'` keeps the scope exactly + // right and skips the remedy on every `pull_request` run — the contributor this + // workflow exists for reads `--list-different`'s bare filename with no instruction, + // the identical loss `== 'success'` costs, reached through the conjunct form the guard + // explicitly waved through. So the conjunction is an ALLOW-list too: every `&&` term + // must be `failure()`, a negated status function that is TRUE on the failure path + // (`!cancelled()`, `!success()`), or the scope equality. Anything else — a `github.*` + // context, another step's outcome, a literal, `always()` (a no-op), `!failure()` + // (never true beside `failure()`), a SECOND equality on the same context — is rejected. + const narrowedToZero: [string, string][] = [ + [ + "`&& github.event_name == 'push'`, false on every pull_request run", + `failure() && ${SCOPE} && github.event_name == 'push'`, + ], + ['`&& false`, never true', `failure() && ${SCOPE} && false`], + [ + "`&& steps.install.outcome == 'success'`, another step's outcome ANDed in", + `failure() && ${SCOPE} && steps.install.outcome == 'success'`, + ], + ['`&& !failure()`, false whenever `failure()` is true', `failure() && ${SCOPE} && !failure()`], + ['`&& always()`, a conjunct that decides nothing', `failure() && ${SCOPE} && always()`], + [ + "a second equality on the same context (`== 'success'`), so the conjunction is never true", + `failure() && ${SCOPE} && steps.format_check.outcome == 'success'`, + ], + [ + "the round-8 'narrowing' conjunct itself (`&& github.event_name == 'pull_request'`)", + `failure() && ${SCOPE} && github.event_name == 'pull_request'`, + ], + ] + + for (const [label, condition] of narrowedToZero) { + it(`fails on ${label}`, () => { + const r = checkFormatWorkflow(remedyOf(condition)) + expect(r.ok, `${condition}: ${r.message}`).toBe(false) + expect(r.message, condition).toContain('decides nothing') + }) + } + + // The allow-listed conjuncts, in the spellings a correct workflow may use — a `!` on a + // status FUNCTION that is true on the failure path is not a `!` on the scope, an outer + // `${{ }}` is how GitHub lets any `if:` be written, and a parenthesised term is the same + // term. Rejecting these would fail a correct workflow, which is how a guard gets weakened. + const kept: [string, string][] = [ + [ + '`!cancelled()`, a negated function rather than a negated scope', + `failure() && !cancelled() && ${SCOPE}`, + ], + ['`!success()`, true on the failure path', `failure() && !success() && ${SCOPE}`], + ['the shipped spelling itself', `failure() && ${SCOPE}`], + ['the shipped spelling wrapped in `${{ }}`', `\${{ failure() && ${SCOPE} }}`], + ['each term parenthesised', `(failure()) && (${SCOPE})`], + ['the equality reversed', `failure() && 'failure' == steps.format_check.outcome`], + ] + + for (const [label, condition] of kept) { + it(`accepts ${label}`, () => { + const r = checkFormatWorkflow(remedyOf(condition)) + expect(r.ok, `${condition}: ${r.message}`).toBe(true) + }) + } + + // The three buckets stay disjoint: a neutralised condition DOES name the check and + // DOES carry the equality, so reporting it as "not scoped" or "does not compare it to + // 'failure'" would name a cause the author already got right. + it('names the neutralisation, not the missing reference or the wrong value', () => { + const r = checkFormatWorkflow(remedyOf(`failure() && ${SCOPE} || true`)) + expect(r.message).toContain('decides nothing') + expect(r.message).not.toContain('are not scoped to the formatting check') + expect(r.message).not.toContain('do not compare it to') + }) + + // On the file this repo actually runs, not only the fixture: the mutation is one + // ` || true` appended to line 139, and every other rule in the module stays green. + it('fires on the shipped workflow when the scope is ORed away', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate(shipped, `failure() && ${SCOPE}`, `failure() && ${SCOPE} || true`, SCOPE), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('decides nothing') + }) + + // The same on the conjunct form. Measured on GitHub (probe run on this PR, see the + // working log): with `&& github.event_name == 'push'` appended, a PR carrying an + // unformatted file gets `Check formatting: failure` and the remedy step `skipped`. + it('fires on the shipped workflow when a conjunct narrows the remedy off the PR path', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + `failure() && ${SCOPE}`, + `failure() && ${SCOPE} && github.event_name == 'push'`, + SCOPE, + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('decides nothing') + }) +}) + +// The guard reads the REAL file, exactly like `checkThisRepoGate` does — a +// hand-maintained fixture would keep passing while the shipped workflow drifts. +describe('checkThisRepoFormatWorkflow reads the shipped workflow (#413)', () => { + it('points at .github/workflows/format.yml', () => { + expect(FORMAT_WORKFLOW.endsWith('.github/workflows/format.yml')).toBe(true) + }) + + it('the workflow exists on disk', () => { + expect(existsSync(FORMAT_WORKFLOW), `${FORMAT_WORKFLOW} is missing`).toBe(true) + }) + + it('this repo’s format workflow satisfies every rule above', () => { + const r = checkThisRepoFormatWorkflow() + expect(r.ok, r.message).toBe(true) + }) + + it('reports a missing workflow as a failure, never a vacuous pass', () => { + // The degenerate case the story is about: no workflow at all must be RED. + expect(checkFormatWorkflow('').ok).toBe(false) + }) + + // Asserted against the SHIPPED file, not the fixture: the guard is only worth + // anything if it fires on the workflow this repo actually runs. (The rule is + // "declares no paths-ignore", not "never says the word" — the header comment + // explains why the key is banned, and a comment must not fail the check.) + it('fires on the shipped workflow the moment a paths-ignore is declared in it', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + ' push:\n branches:', + ' push:\n paths-ignore:\n - .changeset/**\n branches:', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('paths-ignore') + }) + + // The same "fires on the SHIPPED file" treatment for the three properties whose + // absence a well-formed fixture cannot demonstrate: they are job-level, and a + // fixture that drifts from the real job would keep passing while the real job + // goes advisory / privileged / silent. + it('fires on the shipped workflow when the job is made advisory or privileged', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const advisory = checkFormatWorkflow( + shipped.replace( + ' - name: Check formatting\n', + ' - name: Check formatting\n continue-on-error: true\n', + ), + ) + expect(advisory.ok).toBe(false) + expect(advisory.message).toContain('continue-on-error') + + const privileged = checkFormatWorkflow(shipped.replace('contents: read', 'contents: write')) + expect(privileged.ok).toBe(false) + expect(privileged.message).toContain('permissions') + }) + + it('fires on the shipped workflow when its failure path stops naming the remedy', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(shipped.replace(/^\s*if: failure\(\).*\n/m, '')) + expect(r.ok).toBe(false) + expect(r.message).toContain('pnpm format') + }) + + it('fires on the shipped workflow when the remedy widens to a bare `if: failure()`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(shipped.replace(/^(\s*)if: failure\(\).*$/m, '$1if: failure()')) + expect(r.ok).toBe(false) + expect(r.message).toContain('not scoped') + }) + + // The two never-true `if:` EXPRESSIONS, on the file this repo actually runs: the + // job-level one skips the job, the step-level one leaves the job green with the + // only checking step skipped. A literal-false deny-list waved both through. + it('fires on the shipped workflow when a never-true `if:` is added to the job', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + 'jobs:\n format:\n', + "jobs:\n format:\n if: github.event_name == 'workflow_dispatch'\n", + 'the shipped job header', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + it('fires on the shipped workflow when a never-true `if:` is added to the check step', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + ' - name: Check formatting\n', + " - name: Check formatting\n if: github.event_name == 'push'\n", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('if:') + }) + + // The three spellings an allow-list keyed on `failure()` alone waved through, on + // the file this repo actually runs. All of them leave `Check formatting` skipped + // on a normal pull request while the job — and the `format` context — end green. + it('fires on the shipped workflow when the check step is given a `failure()` guard', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const conditions = [ + 'failure()', + "github.event_name == 'push' && failure()", + "failure() && steps.format_check.outcome == 'failure'", + ] + for (const condition of conditions) { + const r = checkFormatWorkflow( + shipped.replace( + ' - name: Check formatting\n', + ` - name: Check formatting\n if: ${condition}\n`, + ), + ) + expect(r.ok, `${condition}: ${r.message}`).toBe(false) + expect(r.message).toContain('if:') + } + }) + + // One correctly-scoped remedy must not license a second, job-scoped one: the + // annotation would fire on a broken `pnpm install` and tell the contributor to + // run `pnpm format` against a cause it cannot fix. + it('fires on the shipped workflow when a second unscoped `failure()` step is added', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + `${shipped} - name: Extra note + if: failure() + run: echo "::error title=Formatting check failed::not formatted" +`, + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('not scoped') + }) + + it('fires on the shipped workflow when `if: failure()` is added to the install step', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + ' - name: Install dependencies\n', + ' - name: Install dependencies\n if: failure()\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('not scoped') + }) + + // Quoting the id is valid YAML and resolves identically on GitHub: the shipped + // workflow must stay GREEN through it, or the guard fails a correct file. + it('stays green on the shipped workflow when its `id:` is quoted', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate(shipped, 'id: format_check', "id: 'format_check'", '`id: format_check`'), + ) + expect(r.ok, r.message).toBe(true) + }) + + // One token on the file this repo actually runs. `failure()` is still true when the + // formatting check fails, but `outcome` is `'failure'`, so `== 'success'` is FALSE and + // the remedy step is skipped on precisely the run that needed it: the contributor gets + // `--list-different`'s bare filename with prettier's own "--write to fix" hint + // suppressed, and nothing telling them what to run (AC1). + it('fires on the shipped workflow when the remedy scope compares against the wrong outcome', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + for (const value of ['success', 'skipped']) { + const r = checkFormatWorkflow( + mutate( + shipped, + "steps.format_check.outcome == 'failure'", + `steps.format_check.outcome == '${value}'`, + "the shipped remedy's scope", + ), + ) + expect(r.ok, `${value}: ${r.message}`).toBe(false) + expect(r.message, value).toContain('compare it to') + } + }) + + // The reviewer's measured repro: a second job the `format` job depends on. `precheck` + // fails (or is itself skipped), `format` never runs and is reported skipped, and a + // skipped job reports its required check SUCCESSFUL — so once AC8 lists `format` the + // merge goes through with the formatting check never having executed. + it('fires on the shipped workflow when the format job is gated by `needs:`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const gated = mutate( + shipped, + 'jobs:\n format:\n', + `jobs: + precheck: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Gate + run: exit 1 + format: + needs: precheck +`, + 'the shipped `format` job header', + ) + const r = checkFormatWorkflow(gated) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('needs:') + }) + + // The negative branch filter, on the shipped file: one line under `pull_request` + // and no PR targeting `main` is ever format-checked. + it('fires on the shipped workflow when a trigger gains a `branches-ignore`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + for (const event of ['pull_request', 'push']) { + const r = checkFormatWorkflow( + shipped.replace( + ` ${event}:\n branches:\n - main`, + ` ${event}:\n branches-ignore:\n - main`, + ), + ) + expect(r.ok, `${event}: ${r.message}`).toBe(false) + expect(r.message).toContain('branches-ignore') + } + }) + + // The flow-mapping hole, on the file this repo actually runs. Each of the four is + // ONE line of valid YAML that GitHub honours, and each left `ok=true` before round 5: + // a markdown-only PR never format-checked, no PR targeting `main` ever checked, the + // check running only after the PR closes, and post-merge drift on `main` invisible. + // Since the parser migration the SPELLING is read and the SEMANTIC rule is what fires, + // so each row asserts its own cause — and the correct flow spelling stays green. + it('fires on the shipped workflow when a trigger is respelled as a flow mapping WITH a hole', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const holes: [string, string, string][] = [ + [ + ' pull_request:\n branches:\n - main\n', + " pull_request: { branches: [main], paths-ignore: ['**/*.md'] }\n", + 'paths-ignore', + ], + [ + ' pull_request:\n branches:\n - main\n', + ' pull_request: { branches: [release] }\n', + 'does not cover `main` (release)', + ], + [ + ' pull_request:\n branches:\n - main\n', + ' pull_request: { branches: [main], types: [closed] }\n', + 'dropping opened, synchronize', + ], + [ + ' push:\n branches:\n - main\n', + ' push: { branches: [release] }\n', + 'does not cover `main` (release)', + ], + ] + for (const [block, flow, cause] of holes) { + const mutated = shipped.replace(block, flow) + expect(mutated, `${flow}: the shipped file no longer contains the block spelling`).not.toBe( + shipped, + ) + const r = checkFormatWorkflow(mutated) + expect(r.ok, `${flow}: ${r.message}`).toBe(false) + expect(r.message, flow).toContain(cause) + } + }) + + it('stays green on the shipped workflow when a correct trigger is respelled as a flow mapping', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + ' pull_request:\n branches:\n - main\n', + ' pull_request: { branches: [main] }\n', + 'the shipped `pull_request` trigger', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // The `uses:` writer, on the shipped file. Before the `Check formatting` step this + // needs no permission at all: the action rewrites the runner's checkout, the check + // then passes on unformatted code, and the `format` context goes green. + it('fires on the shipped workflow when a formatting action is added', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + ' - name: Check formatting\n', + ' - name: Fix\n uses: creyD/prettier_action@v4\n with:\n prettier_options: --write .\n - name: Check formatting\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('uses:') + }) + + it('fires on the shipped workflow when a writer hides behind a quoted `#`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + ' - name: Install dependencies\n run: pnpm install\n', + ' - name: Install dependencies\n run: |\n pnpm install\n echo "note # here"; prettier --write .\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('fires on the shipped workflow when the job that carries the context is renamed', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(shipped.replace('\n format:\n', '\n fmt:\n')) + expect(r.ok).toBe(false) + expect(r.message).toContain('status context') + }) + + // Measured on the shipped file before the fix: `ok=true`. `format` reports SUCCESS + // after an `echo`; the job that checks publishes `worker`, which nothing requires. + it('fires on the shipped workflow when a decoy job takes the `format` name', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + '\n format:\n', + ` + format: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - name: Nothing + run: echo ok + worker: +`, + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('is `worker`, not `format`') + }) + + // Each of these was measured `ok=true` on the shipped file before round 6: a step + // spelled as anything but a block mapping was invisible to `usesProblems` AND to the + // write-mode scan, so inserted before the checking step it rewrites the checkout and + // the `format` context goes green on unformatted code (AC6). The parser resolves each + // to the same mapping GitHub runs (probe run 33724282504 on PR #477: the JSON-spelled + // step executed), so every step rule now reads it — and names the real cause. + it('fires on the shipped workflow when a step is spelled as a flow item', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const items: [string, string][] = [ + ['- { name: Fix, run: npx prettier --write . }', 'prettier --write'], + ['- {run: prettier --write .}', 'prettier --write'], + ['- { uses: creyD/prettier_action@v4 }', 'uses:'], + ['- { uses: stefanzweifel/git-auto-commit-action@v5 }', 'uses:'], + ['- { "name": "Fix", "run": "npx prettier --write ." }', 'prettier --write'], + ] + for (const [item, cause] of items) { + const r = checkFormatWorkflow( + shipped.replace( + ' - name: Check formatting\n', + ` ${item}\n - name: Check formatting\n`, + ), + ) + expect(r.ok, `${item}: ${r.message}`).toBe(false) + expect(r.message, item).toContain(cause) + } + }) + + // …and a correct step spelled as a flow item stays green on the shipped file — the + // spelling GitHub accepted in the same probe run. + it('stays green on the shipped workflow when a correct step is spelled as a flow item', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + ' - name: Check formatting\n', + ' - { name: Install dependencies, run: pnpm install }\n - name: Check formatting\n', + 'the shipped checking step', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // The shipped file's own `run: |` block scalar contains shell (`if ! command -v + // pnpm …`), and the workflow is green: the structural rules do not read that body. + it('stays green on the shipped workflow, whose run blocks carry real shell', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + expect(shipped, 'the shipped workflow no longer has a block scalar to prove this on').toContain( + 'run: |', + ) + expect(checkFormatWorkflow(shipped).ok).toBe(true) + }) + + it('fires on the shipped workflow when the remedy is moved above the checking step', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const remedy = + / {6}- name: Explain how to fix a formatting failure\n(?:.*\n)*? {10}echo "::error[^\n]*\n/ + const found = remedy.exec(shipped) + expect(found, 'the shipped remedy step was not found').not.toBeNull() + const moved = shipped + .replace(remedy, '') + .replace( + ' - name: Check formatting\n', + `${found?.[0] ?? ''} - name: Check formatting\n`, + ) + const r = checkFormatWorkflow(moved) + expect(r.ok).toBe(false) + expect(r.message).toContain('has not run yet') + }) + + it('fires on the shipped workflow when the checking command is narrowed', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace('run: pnpm format:check', 'run: pnpm --filter=@pair/website format:check'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('one command, two places') + }) + + it('fires on the shipped workflow when cancel-in-progress negates the PR event', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + "cancel-in-progress: ${{ github.event_name == 'pull_request' }}", + "cancel-in-progress: ${{ github.event_name != 'pull_request' }}", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('cancel-in-progress') + }) + + // Measured `ok=true` on the shipped file before the fix. The workflow never runs on + // a push to any branch — post-merge drift invisible with the guard green. + it('fires on the shipped workflow when `push:` is filtered by tags alone', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + shipped.replace( + ' push:\n branches:\n - main\n', + " push:\n tags:\n - 'v*'\n", + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('tags') + }) + + it('stays green on the shipped workflow when a tags filter is added BESIDE the branch filter', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + ' push:\n branches:\n - main\n', + " push:\n branches:\n - main\n tags:\n - 'v*'\n", + 'the shipped `push:` trigger', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // Measured `ok=true` on the shipped file before the fix, both spellings. Every run in + // one group: a PR push cancels main's in-progress run. + it('fires on the shipped workflow when the concurrency group stops being keyed on the ref', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + for (const group of ['group: format', 'group: ${{ github.workflow }}']) { + const r = checkFormatWorkflow(shipped.replace('group: format-${{ github.ref }}', group)) + expect(r.ok, group).toBe(false) + expect(r.message, group).toContain('github.ref') + } + }) + + // Measured `ok=true` on the shipped file before the fix, both rows: a `name:` renames + // the published context, a matrix suffixes it — `format` stops existing either way. + it('fires on the shipped workflow when the host job is given a display name', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + 'jobs:\n format:\n', + 'jobs:\n format:\n name: Formatting\n', + 'the shipped job header', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('display name') + }) + + it('fires on the shipped workflow when the host job is given a matrix', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + 'jobs:\n format:\n', + "jobs:\n format:\n strategy:\n matrix:\n node: ['20']\n", + 'the shipped job header', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('strategy') + }) +}) + +// `uses:` was matched on the action NAME only, so the step's `with:` was invisible. +// Producer boundary (actions/checkout@v4 action.yml, input `ref`): "The branch, tag or +// SHA to checkout. When checking out the repository that triggered a workflow, this +// defaults to the reference or SHA for that event" — i.e. setting it REPLACES the PR +// merge ref. Measured on GitHub (probe run on this PR, see the working log): with +// `with: ref: main` on the shipped checkout, a PR carrying an unformatted file gets a +// SUCCESSFUL `format` context — CI checked `main`, not the PR. `sparse-checkout` is +// AC4's subset divergence spelled as a checkout input; `repository` and `path` change +// what tree the command runs on at all. So the checkout's `with:` is an ALLOW-list of +// fetch-mechanics inputs that leave the tree as the event's ref. +describe('the checkout step may not choose WHAT is checked out (#413)', () => { + const CHECKOUT = ' - name: Checkout code\n uses: actions/checkout@v4\n' + const withInputs = (source: string, inputs: string) => + mutate(source, CHECKOUT, `${CHECKOUT} with:\n${inputs}`, 'the checkout step') + + const redirected: [string, string, string][] = [ + ['`ref: main`, the base branch instead of the PR merge ref', ' ref: main\n', 'ref'], + [ + '`ref:` set to an expression', + ' ref: ${{ github.event.pull_request.base.sha }}\n', + 'ref', + ], + ['`repository:`, another repository', ' repository: foomakers/other\n', 'repository'], + ['`path:`, a directory the check does not run in', ' path: checkout\n', 'path'], + [ + '`sparse-checkout:`, a subset of the tree', + ' sparse-checkout: packages/dev-tools\n', + 'sparse-checkout', + ], + [ + '`sparse-checkout-cone-mode:`', + ' sparse-checkout-cone-mode: false\n', + 'sparse-checkout-cone-mode', + ], + ['`submodules:`, content the PR does not carry', ' submodules: true\n', 'submodules'], + ['`lfs:`', ' lfs: true\n', 'lfs'], + [ + '`token:`, not an allow-listed input either', + ' token: ${{ github.token }}\n', + 'token', + ], + [ + 'a rejected input BESIDE an accepted one', + ' fetch-depth: 0\n ref: main\n', + 'ref', + ], + ["a quoted key (`'ref'`)", " 'ref': main\n", 'ref'], + ['an input this guard has never heard of', ' new-input: true\n', 'new-input'], + ] + + for (const [label, inputs, key] of redirected) { + it(`fails on checkout ${label}`, () => { + const r = checkFormatWorkflow(withInputs(WELL_FORMED, inputs)) + expect(r.ok, `${inputs}: ${r.message}`).toBe(false) + expect(r.message, inputs).toContain('with:') + expect(r.message, inputs).toContain(`\`${key}\``) + }) + } + + // The recommendation's named inputs are named in the message, whichever one fired — + // the reader learns the whole class, not the one key they happened to write. + it('names `ref`, `repository`, `path`, `sparse-checkout` and `sparse-checkout-cone-mode` in the message', () => { + const r = checkFormatWorkflow(withInputs(WELL_FORMED, ' submodules: true\n')) + expect(r.ok).toBe(false) + for (const key of [ + 'ref', + 'repository', + 'path', + 'sparse-checkout', + 'sparse-checkout-cone-mode', + ]) { + expect(r.message).toContain(`\`${key}\``) + } + }) + + // The `with:` is READ whatever its spelling — the parser resolves a flow mapping and + // an alias to the same inputs GitHub passes the action, so the allow-list decides. + it('fails on a flow-style `with: { ref: main }`, naming the input', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + CHECKOUT, + `${CHECKOUT} with: { ref: main }\n`, + 'the checkout step', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('with:') + expect(r.message).toContain('`ref`') + }) + + it('follows `with: *inputs` to the inputs the alias names', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + CHECKOUT, + ` - name: Anchor\n uses: actions/setup-node@v4\n with: &inputs\n ref: main\n${CHECKOUT} with: *inputs\n`, + 'the checkout step', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('with:') + expect(r.message).toContain('`ref`') + }) + + it('stays green on an aliased `with:` carrying only fetch mechanics', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + CHECKOUT, + `${CHECKOUT} with: &inputs\n fetch-depth: 0\n`, + 'the checkout step', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // Fetch mechanics leave the tree as the event's ref — a guard that rejected `fetch-depth: 0` + // would fail a correct workflow, which is how a guard gets weakened. + const mechanics: [string, string][] = [ + ['`fetch-depth: 0`', ' fetch-depth: 0\n'], + [ + '`fetch-depth` and `persist-credentials: false`', + ' fetch-depth: 0\n persist-credentials: false\n', + ], + ['`fetch-tags: false`', ' fetch-tags: false\n'], + ['`show-progress: false`', ' show-progress: false\n'], + ['`clean: true`', ' clean: true\n'], + ['`set-safe-directory: true`', ' set-safe-directory: true\n'], + ["a quoted accepted key (`'fetch-depth'`)", " 'fetch-depth': 1\n"], + ] + + for (const [label, inputs] of mechanics) { + it(`accepts checkout with ${label}`, () => { + const r = checkFormatWorkflow(withInputs(WELL_FORMED, inputs)) + expect(r.ok, `${inputs}: ${r.message}`).toBe(true) + }) + } + + it('accepts the bare `uses: actions/checkout@v4` (the shipped spelling)', () => { + const r = checkFormatWorkflow(WELL_FORMED) + expect(WELL_FORMED).toContain(CHECKOUT) + expect(r.ok, r.message).toBe(true) + }) + + // The bound: only the checkout decides WHAT is checked out. `pnpm/action-setup` and + // `actions/setup-node` inputs choose tool versions, never the tree, so their `with:` is + // not constrained — the fixture already carries `version:`, `node-version:`, `cache:`. + it('does not constrain the `with:` of the toolchain actions', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + " version: '10.15.0'\n", + " version: '10.15.0'\n run_install: false\n standalone: true\n", + 'the pnpm/action-setup inputs', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('fires on the shipped workflow when its checkout is pointed at `main`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(withInputs(shipped, ' ref: main\n')) + expect(r.ok).toBe(false) + expect(r.message).toContain('`ref`') + }) + + it('stays green on the shipped workflow when its checkout gains `fetch-depth: 0`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(withInputs(shipped, ' fetch-depth: 0\n')) + expect(r.ok, r.message).toBe(true) + }) +}) + +// The command equality reads only the `run:` text. `working-directory:` is `cd` spelled +// as a key, and `defaults.run.working-directory` is the same `cd` one level up (job) or +// two (workflow) — each makes CI run `pnpm format:check` inside a package, i.e. that +// package's OWN `format:check`. Measured (pnpm, the real producer): today no workspace +// package declares the script, so `cd packages/dev-tools && pnpm format:check` exits 254 +// (ERR_PNPM_NO_SCRIPT) — fail-closed by accident; a package.json carrying +// `"format:check": "echo SUBSET-ONLY"` runs it, exit 0. The first package to gain that +// script turns the accident into a silent subset with this guard green — AC4's divergence +// reinstated through a key the `--filter=`/`cd` rule was written to catch. +describe('the checking step runs in the repository root, carrying only the keys it needs (#413)', () => { + const CHECK = + ' - name: Check formatting\n id: format_check\n run: pnpm format:check\n' + const RUN = ' run: pnpm format:check\n' + const checkWith = (source: string, key: string) => + mutate(source, CHECK, CHECK.replace(RUN, `${key}${RUN}`), 'the checking step') + + const foreignKeys: [string, string, string][] = [ + [ + '`working-directory:`', + ' working-directory: packages/dev-tools\n', + 'working-directory', + ], + ['`shell:`', ' shell: bash\n', 'shell'], + ['`env:`', ' env:\n NODE_OPTIONS: --max-old-space-size=4096\n', 'env'], + [ + '`with:` (meaningless on a `run:` step, and not needed)', + ' with:\n x: y\n', + 'with', + ], + ] + + for (const [label, key, name] of foreignKeys) { + it(`fails when the checking step carries ${label}`, () => { + const r = checkFormatWorkflow(checkWith(WELL_FORMED, key)) + expect(r.ok, `${key}: ${r.message}`).toBe(false) + expect(r.message, key).toContain(`\`${name}`) + expect(r.message, key).toContain('may carry only') + }) + } + + it('accepts `timeout-minutes:` on the checking step', () => { + const r = checkFormatWorkflow(checkWith(WELL_FORMED, ' timeout-minutes: 5\n')) + expect(r.ok, r.message).toBe(true) + }) + + // `if:` and `continue-on-error:` on that step are owned by their own rules, which + // name the loss precisely; this rule stays silent on them rather than reporting a + // second, vaguer cause. + it('leaves `if:` and `continue-on-error:` on the checking step to their own rules', () => { + for (const key of [ + " if: github.event_name == 'push'\n", + ' continue-on-error: true\n', + ]) { + const r = checkFormatWorkflow(checkWith(WELL_FORMED, key)) + expect(r.ok, key).toBe(false) + expect(r.message, key).not.toContain('may carry only') + } + }) + + const JOB = ' format:\n runs-on: ubuntu-latest\n' + + it('fails on `defaults:` on the host job, in either spelling', () => { + for (const defaults of [ + ' defaults:\n run:\n working-directory: packages/dev-tools\n', + ' defaults:\n run:\n shell: bash\n', + ]) { + const r = checkFormatWorkflow(mutate(WELL_FORMED, JOB, `${JOB}${defaults}`, 'the host job')) + expect(r.ok, `${defaults}: ${r.message}`).toBe(false) + expect(r.message, defaults).toContain('defaults') + } + }) + + it('fails on a workflow-level `defaults:`', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + '\njobs:\n', + '\ndefaults:\n run:\n working-directory: packages/dev-tools\n\njobs:\n', + 'the jobs key', + ), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('defaults') + }) + + it('fires on the shipped workflow when the checking step gains a `working-directory:`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + checkWith(shipped, ' working-directory: packages/dev-tools\n'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('working-directory') + }) + + it('fires on the shipped workflow when the host job gains `defaults:`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + 'jobs:\n format:\n', + 'jobs:\n format:\n defaults:\n run:\n working-directory: packages/dev-tools\n', + 'the shipped job header', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('defaults') + }) +}) + +// The rule's own comment said a substring test for `pull_request` was not enough — and +// `CANCEL_ON_PULL_REQUEST` was itself an unanchored substring over the value. Measured +// on GitHub (probe run on this PR, see the working log), evaluated on a pull_request +// event: `!(github.event_name == 'pull_request')` → false, `github.event_name == +// 'pull_request' && false` → false, `github.event_name == 'pull_request' || true` → true. +// The first two cancel nothing on a PR (stale verdicts, superseded runs keep burning +// runners); the third cancels on `main` too, so of two merges a minute apart only the +// second commit carries a verdict of its own. +// +// The allow-list has TWO members, and a bare `true` is the weaker of them, not a defect: +// it supersedes, which is what AC7 asks, and `main` is linear, so the surviving run's tree +// still contains the cancelled commit's changes — drift is caught one commit later. What it +// costs is the per-commit `format` history, which is why the shipped file spells the +// conditional. `|| true` is rejected not because it is weak but because it is NEITHER +// member: the rule is anchored to the WHOLE value, so a spelling that merely CONTAINS one +// of the two is a third value nobody argued for. +describe('cancel-in-progress is an anchored allow-list, not a substring (#413)', () => { + const cancelOf = (source: string, value: string) => + mutate(source, 'cancel-in-progress: true', `cancel-in-progress: ${value}`, 'cancel-in-progress') + + const rejected: string[] = [ + "${{ !(github.event_name == 'pull_request') }}", + "${{ github.event_name == 'pull_request' && false }}", + "${{ github.event_name == 'pull_request' || true }}", + "${{ github.event_name == 'push' }}", + "${{ github.event_name == 'pull_request' }}-x", + "${{ contains(github.event_name, 'pull_request') }}", + '${{ true }}', + ] + + for (const value of rejected) { + it(`fails on \`cancel-in-progress: ${value}\``, () => { + const r = checkFormatWorkflow(cancelOf(WELL_FORMED, value)) + expect(r.ok, `${value}: ${r.message}`).toBe(false) + expect(r.message, value).toContain('cancel-in-progress') + }) + } + + const accepted: string[] = [ + "${{ github.event_name == 'pull_request' }}", + "${{ 'pull_request' == github.event_name }}", + '${{ github.event_name == "pull_request" }}', + '"${{ github.event_name == \'pull_request\' }}"', + "${{github.event_name=='pull_request'}}", + // The deliberate weaker member — see the block comment above this describe. + 'true', + ] + + for (const value of accepted) { + it(`accepts \`cancel-in-progress: ${value}\``, () => { + const r = checkFormatWorkflow(cancelOf(WELL_FORMED, value)) + expect(r.ok, `${value}: ${r.message}`).toBe(true) + }) + } + + it('fires on the shipped workflow when the PR equality is negated as a group', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + "cancel-in-progress: ${{ github.event_name == 'pull_request' }}", + "cancel-in-progress: ${{ !(github.event_name == 'pull_request') }}", + 'the shipped cancel-in-progress', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('cancel-in-progress') + }) +}) + +// Round 11, carried forward. `GROUP_KEYED_ON_REF` was a substring test too: a group that +// CONTAINS `github.ref` and also `github.run_id` (or `sha`, `run_number`) is unique per +// run, so nothing is ever superseded — the "concurrency dropped" loss with the block still +// present. And a bare `${{ github.ref }}` with no workflow-distinguishing prefix shares +// its group with any other workflow keyed the same way (latent today; one token from a +// cross-workflow cancel). The accepted shape is `-`, anchored. +describe('the concurrency group is a prefixed ref key and nothing more (#413)', () => { + const GROUP = 'group: format-${{ github.ref }}' + const groupOf = (source: string, value: string) => + mutate(source, GROUP, `group: ${value}`, 'the concurrency group') + + const unique: string[] = [ + 'format-${{ github.run_id }}-${{ github.ref }}', + 'format-${{ github.ref }}-${{ github.sha }}', + 'format-${{ github.ref }}-${{ github.run_number }}', + 'format-${{ github.ref }}-${{ github.run_attempt }}', + ] + + for (const value of unique) { + it(`fails on \`group: ${value}\`, unique per run so nothing supersedes`, () => { + const r = checkFormatWorkflow(groupOf(WELL_FORMED, value)) + expect(r.ok, `${value}: ${r.message}`).toBe(false) + expect(r.message, value).toContain('github.ref') + }) + } + + it('fails on a bare `${{ github.ref }}` with no workflow-distinguishing prefix', () => { + const r = checkFormatWorkflow(groupOf(WELL_FORMED, '${{ github.ref }}')) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('prefix') + }) + + it('fires on the shipped workflow when the group gains a per-run token', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(groupOf(shipped, 'format-${{ github.run_id }}-${{ github.ref }}')) + expect(r.ok).toBe(false) + expect(r.message).toContain('github.ref') + }) +}) + +// Two VALID spellings the reader could not follow were reported as a DIFFERENT, false +// problem. (a) An indentless block sequence — YAML permits `- ` at the parent key's +// indent, and `yaml@2.8.2` parses `branches:\n- main` and `branches:\n - main` to the +// same value (measured); GitHub honours it (probe run on this PR, see the working log). +// `pnpm format:check` runs prettier over ts/tsx/js/jsx/json/html only, so nothing in this +// repo normalizes YAML indentation and an editor default produces exactly this shape. +// It was reported as "does not cover `main` (no branch)" / "no failure-path step names the +// remedy". (b) `branches: *shared` was reported as the same "no branch", while the ADL +// states aliases are rejected anywhere with a message naming the spelling. A guard that +// names the wrong cause on a correct workflow is the kind that gets weakened. +describe('an indentless block sequence is read, and an alias is named as one (#413)', () => { + /** Every `steps:` item and its body moved two columns left — the indentless spelling. */ + function indentlessSteps(source: string): string { + const out: string[] = [] + let inSteps = false + for (const line of source.split('\n')) { + if (/^ {4}steps:\s*$/.test(line)) { + inSteps = true + out.push(line) + continue + } + if (inSteps && line.trim() !== '' && line.length - line.trimStart().length <= 4) { + inSteps = false + } + out.push(inSteps && line.startsWith(' ') ? line.slice(2) : line) + } + expect(out.join('\n'), 'the steps did not move').not.toBe(source) + return out.join('\n') + } + + const indentlessBranches = (source: string) => + mutate(source, /^ {6}- main$/gm, ' - main', 'the indented branch items') + + it('accepts indentless `branches:` items on both triggers', () => { + const r = checkFormatWorkflow(indentlessBranches(WELL_FORMED)) + expect(r.ok, r.message).toBe(true) + }) + + it('accepts indentless `steps:` items', () => { + const r = checkFormatWorkflow(indentlessSteps(WELL_FORMED)) + expect(r.ok, r.message).toBe(true) + }) + + it('accepts the shipped workflow with both sequences written indentless', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(indentlessSteps(indentlessBranches(shipped))) + expect(r.ok, r.message).toBe(true) + }) + + it('accepts indentless `types:` items that keep opened and synchronize', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' pull_request:\n branches:\n - main\n', + ' pull_request:\n branches:\n - main\n types:\n - opened\n - synchronize\n - reopened\n', + 'the pull_request trigger', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // Read, not merely tolerated: the rules see through the spelling to the real cause. + it('reads an indentless branch filter that misses `main` and names THAT cause', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' pull_request:\n branches:\n - main\n', + ' pull_request:\n branches:\n - release\n', + 'the pull_request branch filter', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('does not cover `main` (release)') + }) + + it('reads an indentless `types:` narrowing and names THAT cause', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' pull_request:\n branches:\n - main\n', + ' pull_request:\n branches:\n - main\n types:\n - closed\n', + 'the pull_request trigger', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('dropping opened, synchronize') + }) + + it('follows indentless steps into the write-mode scan', () => { + const r = checkFormatWorkflow( + indentlessSteps(WELL_FORMED).replace( + ' - name: Check formatting\n', + ' - name: Fix\n run: npx prettier --write .\n - name: Check formatting\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('follows indentless steps into the `uses:` allow-list', () => { + const r = checkFormatWorkflow( + indentlessSteps(WELL_FORMED).replace( + ' - name: Check formatting\n', + ' - name: Fix\n uses: creyD/prettier_action@v4\n - name: Check formatting\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('uses:') + }) + + it('reads an indentless `needs:` list and rejects it as `needs:`', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' format:\n runs-on: ubuntu-latest\n', + ' precheck:\n runs-on: ubuntu-latest\n steps:\n - run: exit 1\n format:\n needs:\n - precheck\n runs-on: ubuntu-latest\n', + 'the host job', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('needs: precheck') + }) + + // `on:` as a LIST of events is valid YAML and a valid workflow, and it carries NO + // filter — every branch, GitHub's default activity types — which is a SUPERSET of what + // this guard requires. Read, therefore, not rejected; what is still RED is a list that + // drops an EVENT. + it('accepts `on:` written as an event list', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'on:\n pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + 'on:\n- pull_request\n- push\n', + 'the on block', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('reads an event list that drops `push` and names THAT cause', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'on:\n pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + 'on:\n- pull_request\n', + 'the on block', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('has no `push` trigger') + }) + + it('reads a scalar `on: pull_request` and names the missing `push`', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'on:\n pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + 'on: pull_request\n', + 'the on block', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('has no `push` trigger') + }) +}) + +// (b) An ANCHOR names a node; an ALIAS is that node. GitHub resolves both — measured on +// PR #477, probe run 33724282535: a workflow whose `push.branches` is `*b`, anchored on +// `pull_request.branches`, RAN on a push to that branch — so a guard that reported the +// alias as an unreadable spelling was reporting a CORRECT workflow red. The parser +// resolves them and the semantic rule decides on the resolved value. Rows are the +// decision table over that domain: the same key, aliased to a value that is fine, and +// aliased to a value that is a hole. +describe('an alias resolves to the value it names, and the rule decides (#413)', () => { + const aliased: [string, string, string, string | null][] = [ + [ + '`branches: *shared` anchored on a filter that names `main`', + ' pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + ' pull_request:\n branches: &shared\n - main\n push:\n branches: *shared\n', + null, + ], + [ + '`branches: *shared` anchored on a filter that misses `main`', + ' pull_request:\n branches:\n - main\n push:\n branches:\n - main\n', + ' pull_request:\n branches: &shared\n - release\n push:\n branches: *shared\n', + 'does not cover `main` (release)', + ], + [ + '`types:` narrowed through an anchored/aliased list item', + ' pull_request:\n branches:\n - main\n', + ' pull_request:\n branches:\n - main\n types: [&t closed, *t]\n', + 'dropping opened, synchronize', + ], + [ + '`tags: *tags` with no `branches:` beside it', + ' push:\n branches:\n - main\n', + " push:\n tags: &tags\n - 'v*'\n", + 'never runs on a push to any branch', + ], + [ + '`needs: *deps` anchored on a job list', + ' format:\n runs-on: ubuntu-latest\n', + ' precheck:\n runs-on: ubuntu-latest\n steps:\n - name: Gate\n run: pnpm install\n format:\n needs: &deps\n - precheck\n runs-on: ubuntu-latest\n', + 'needs: precheck', + ], + [ + '`permissions: *perms` anchored on a WRITE scope', + ' permissions:\n contents: read\n', + ' permissions: &perms\n contents: write\n', + 'WRITE', + ], + [ + '`group: *group` anchored on a constant', + 'concurrency:\n group: format-${{ github.ref }}\n', + 'concurrency:\n group: &group format\n', + 'github.ref', + ], + [ + '`cancel-in-progress: *cancel` anchored on `true`', + 'concurrency:\n group: format-${{ github.ref }}\n cancel-in-progress: true\n', + 'concurrency:\n group: format-${{ github.ref }}\n cancel-in-progress: &cancel true\n', + null, + ], + [ + 'an anchor on a value (`group: &g format-${{ github.ref }}`)', + 'group: format-${{ github.ref }}', + 'group: &g format-${{ github.ref }}', + null, + ], + [ + 'an anchored trigger key (`pull_request: &filters`)', + ' pull_request:\n', + ' pull_request: &filters\n', + null, + ], + ] + + for (const [label, from, to, cause] of aliased) { + it(`${cause === null ? 'accepts' : 'fails on'} ${label}`, () => { + const r = checkFormatWorkflow(mutate(WELL_FORMED, from, to, label)) + if (cause === null) { + expect(r.ok, `${label}: ${r.message}`).toBe(true) + return + } + expect(r.ok, `${label}: ${r.message}`).toBe(false) + expect(r.message, label).toContain(cause) + }) + } + + // An aliased STEP is a whole step the write-mode scan used to lose. The parser puts it + // back, so the alias is scanned exactly like the node it names. + it('follows an aliased step into the write-mode scan', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' - name: Check formatting\n', + ' - &fixer\n name: Fix\n run: npx prettier --write .\n - *fixer\n - name: Check formatting\n', + 'the checking step', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('accepts an aliased step that is an allow-listed one', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' - name: Checkout code\n uses: actions/checkout@v4\n', + ' - &checkout\n name: Checkout code\n uses: actions/checkout@v4\n - *checkout\n', + 'the checkout step', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // Shell is not YAML: the parser hands a `run:` body over as a string, so a `*` at the + // start of a line inside it (a `case` pattern, a glob) is never an alias. + it('leaves a `*` inside a run block scalar alone', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ` run: echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n`, + ` run: |\n echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n echo "* and & are shell here, not YAML."\n`, + 'the remedy step', + ), + ) + expect(r.ok, r.message).toBe(true) + }) +}) + +// Round 13. Every OTHER surface of the workflow is an allow-list; the shell of the +// non-check, non-remedy steps was the one deny-list left (write-mode formatters, `${{`, +// `secrets.`). Measured on the shipped file: `- name: Sync / run: git fetch origin main && +// git checkout origin/main -- .` before `Check formatting` → ok=true; `run: pnpm install +// && find . -name '*.ts' -not -path './node_modules/*' -delete` → ok=true. Both make +// `pnpm format:check` run on a tree that is not the PR's — the identical AC2 loss measured +// with `with: ref: main` (run 33635537234) and closed by an allow-list there — spelled in +// shell, where no formatter list can name it. The workflow has exactly two such commands: +// `pnpm install` (flags only) and the corepack fallback. +describe('a step outside the check and its remedy runs only the toolchain install (#413)', () => { + const CHECK = ' - name: Check formatting\n' + const INSTALL = ' - name: Install dependencies\n run: pnpm install\n' + const before = (source: string, step: string) => + mutate(source, CHECK, `${step}${CHECK}`, 'the checking step') + const install = (source: string, run: string) => + mutate( + source, + INSTALL, + ` - name: Install dependencies\n run: ${run}\n`, + 'the install step', + ) + + const foreign: [string, (source: string) => string, string][] = [ + [ + 'a sync step that replaces the PR tree with `main`', + source => + before( + source, + ' - name: Sync\n run: git fetch origin main && git checkout origin/main -- .\n', + ), + 'git checkout origin/main -- .', + ], + [ + 'an install line that deletes files afterwards', + source => + install( + source, + "pnpm install && find . -name '*.ts' -not -path './node_modules/*' -delete", + ), + '-delete', + ], + [ + 'an install followed by a second command on the same line', + source => install(source, 'pnpm install; git checkout origin/main -- .'), + 'git checkout', + ], + [ + 'an install with a positional argument (`pnpm install ` edits package.json)', + source => install(source, 'pnpm install left-pad'), + 'left-pad', + ], + [ + 'a bare echo (unquoted words are not an inert message)', + source => before(source, ' - name: Say hello\n run: echo hello\n'), + 'echo hello', + ], + [ + 'a block scalar whose lines are not all toolchain commands', + source => + install( + source, + '|\n pnpm install\n curl -sSf https://example.com/fix.sh | sh', + ), + 'curl', + ], + [ + 'an unconditional step AFTER the check (position does not license it)', + source => + mutate( + source, + ' - name: Explain how to fix it\n', + ' - name: Tests\n run: pnpm test\n - name: Explain how to fix it\n', + 'the remedy step', + ), + 'pnpm test', + ], + [ + 'a step in a second job', + source => + mutate( + source, + '\njobs:\n', + '\njobs:\n other:\n runs-on: ubuntu-latest\n permissions:\n contents: read\n steps:\n - run: git checkout origin/main -- .\n', + 'the jobs key', + ), + 'git checkout', + ], + ] + + for (const [label, edit, needle] of foreign) { + it(`fails on ${label}`, () => { + const r = checkFormatWorkflow(edit(WELL_FORMED)) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain(needle) + expect(r.message).toContain('toolchain install') + }) + } + + // The message names the loss in its shell spelling, whichever foreign command fired. + it('names `git checkout origin/main -- .` and `find … -delete` as the loss', () => { + const r = checkFormatWorkflow(install(WELL_FORMED, 'pnpm test')) + expect(r.ok).toBe(false) + expect(r.message).toContain('git checkout origin/main -- .') + expect(r.message).toContain('-delete') + expect(r.message).toContain('AC2') + }) + + // The two commands the workflow has, in every spelling this repo uses or may need. + const toolchain: [string, string][] = [ + ['`pnpm install` with a flag', 'pnpm install --frozen-lockfile'], + [ + '`pnpm install` with several flags, one valued', + 'pnpm install --prefer-offline --reporter=append-only', + ], + [ + 'the corepack fallback block, as shipped', + [ + '|', + ' if ! command -v pnpm >/dev/null 2>&1; then', + ' echo "pnpm not found in PATH; attempting to enable via corepack"', + ' corepack enable || true', + ' corepack prepare pnpm@10.15.0 --activate || true', + ' fi', + ].join('\n'), + ], + ['`corepack enable` alone', 'corepack enable'], + ['`corepack prepare` without the `|| true`', 'corepack prepare pnpm@10.15.0 --activate'], + ['a trailing shell comment on the install line', 'pnpm install # frozen by CI=true'], + ] + + for (const [label, run] of toolchain) { + it(`accepts ${label}`, () => { + const r = checkFormatWorkflow(install(WELL_FORMED, run)) + expect(r.ok, r.message).toBe(true) + }) + } + + // Allow-listing the toolchain does not retire the write scan: a write-mode formatter on + // the install line is reported as BOTH — the reader learns the formatter, not only "foreign". + it('still names a write-mode formatter beside the toolchain rejection', () => { + const r = checkFormatWorkflow(install(WELL_FORMED, 'pnpm install && npx prettier --write .')) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + expect(r.message).toContain('toolchain install') + }) + + // The remedy's shell stays deny-list scanned (it runs after the check, on the failure + // path): its message may say what it needs to. + it('leaves the remedy step outside the TOOLCHAIN allow-list (it has its own)', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ` run: echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n`, + ` run: |\n echo "::error::Not formatted. Run 'pnpm format' locally."\n echo "See the step above for the files."\n`, + 'the remedy step', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + // The two reviewer probes, on the SHIPPED file. + it('fires on the shipped workflow when a sync step precedes the check', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + before( + shipped, + ' - name: Sync\n run: git fetch origin main && git checkout origin/main -- .\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('git checkout origin/main -- .') + }) + + it('fires on the shipped workflow when the install line deletes files', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + install(shipped, "pnpm install && find . -name '*.ts' -not -path './node_modules/*' -delete"), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('-delete') + }) +}) + +// `-w` is prettier's documented short form of `--write` (`prettier --help`: "-w, --write +// Edit files in-place"). Measured: `run: pnpm install && npx prettier -w .` → ok=true while +// `--write` was red — the one list the AC6 ban reuses had the long spelling only. +describe('prettier `-w` is `--write` (#413)', () => { + const INSTALL = ' - name: Install dependencies\n run: pnpm install\n' + + it('fails on `npx prettier -w .` in the install step, naming the formatter', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + INSTALL, + ' - name: Install dependencies\n run: pnpm install && npx prettier -w .\n', + 'the install step', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('fails on `prettier -w` in the remedy step too, where no allow-list applies', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ` run: echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n`, + ` run: npx prettier -w . && echo "::error::Not formatted. Run 'pnpm format' locally."\n`, + 'the remedy step', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) + + it('fires on the shipped workflow', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + INSTALL, + ' - name: Install dependencies\n run: pnpm install && npx prettier -w .\n', + 'the install step', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('prettier --write') + }) +}) + +// Three reader-incompleteness false-fails on CORRECT workflows (the failure class ADL +// 2026-09-01-workflow-guard-rejects-what-it-cannot-read names as the parser flip trigger). +// Each is a spelling GitHub resolves identically to the shipped one — measured on run +// 33676806439 (probe C on PR #477): `run: "pnpm format:check"` logged as `Run pnpm +// format:check`; the whole file with CRLF line endings parsed and ran; workflow-level +// `permissions:` was INHERITED by a job without its own (token: Contents+Issues: read) and +// REPLACED by the job with its own (token: Contents: read only). +describe('a quoted `run:` scalar, CRLF line endings and workflow-level permissions are read (#413)', () => { + const CHECK_RUN = ' run: pnpm format:check\n' + + it('accepts `run: "pnpm format:check"` (double-quoted YAML scalar)', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, CHECK_RUN, ' run: "pnpm format:check"\n'), + ) + expect(r.ok, r.message).toBe(true) + }) + + it("accepts `run: 'pnpm format:check'` (single-quoted)", () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, CHECK_RUN, " run: 'pnpm format:check'\n"), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('still rejects a quoted command that is not THE command', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, CHECK_RUN, ' run: "pnpm format:check --filter=@pair/website"\n'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('not `pnpm format:check`') + }) + + // The same quoted scalar on the INSTALL step. yaml@2.8.2 (the catalog entry) parses + // `run: "pnpm install"` and `run: 'pnpm install'` to the string `pnpm install`; GitHub + // resolves them the same way (run 33676806439). Round 13 unquoted the check command + // but not the setup allow-list, so a correct spelling was red with a misleading cause + // ("runs `"pnpm install"`: … the only shell this workflow runs is the toolchain install"). + const INSTALL_RUN = ' run: pnpm install\n' + const quotedInstall: [string, string][] = [ + ['`run: "pnpm install"` (double-quoted)', ' run: "pnpm install"\n'], + ["`run: 'pnpm install'` (single-quoted)", " run: 'pnpm install'\n"], + ['a quoted install with a flag', ' run: "pnpm install --frozen-lockfile"\n'], + ] + + for (const [label, run] of quotedInstall) { + it(`accepts ${label} on the install step`, () => { + const r = checkFormatWorkflow(mutate(WELL_FORMED, INSTALL_RUN, run, 'the install run')) + expect(r.ok, r.message).toBe(true) + }) + } + + it('accepts the shipped workflow with its install line quoted', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(mutate(shipped, INSTALL_RUN, ' run: "pnpm install"\n')) + expect(r.ok, r.message).toBe(true) + }) + + // Quotes do not launder a foreign command: what is inside them is still read. + it('still rejects a quoted install with a positional argument', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, INSTALL_RUN, ' run: "pnpm install left-pad"\n', 'the install run'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('pnpm install left-pad') + expect(r.message).toContain('toolchain install') + }) + + it('still rejects a quoted install chained to a tree rewrite', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + INSTALL_RUN, + ' run: "pnpm install && git checkout origin/main -- ."\n', + 'the install run', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('git checkout origin/main -- .') + expect(r.message).toContain('toolchain install') + }) + + it('accepts the well-formed workflow with CRLF line endings', () => { + const crlf = WELL_FORMED.replace(/\n/g, '\r\n') + expect(crlf).toContain('\r\n') + const r = checkFormatWorkflow(crlf) + expect(r.ok, r.message).toBe(true) + }) + + it('still sees a hole through CRLF line endings', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' push:\n branches:', + ' push:\n paths-ignore:\n - .changeset/**\n branches:', + 'the push trigger', + ).replace(/\n/g, '\r\n'), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('paths-ignore') + expect(r.message).not.toContain('list of events') + }) + + const JOB_PERMISSIONS = ' permissions:\n contents: read\n' + const atWorkflowLevel = (source: string, scope: string, job: string | null) => { + const withoutJob = mutate(source, JOB_PERMISSIONS, job ?? '', 'the job permissions') + return mutate(withoutJob, '\njobs:\n', `\npermissions:\n${scope}\njobs:\n`, 'the jobs key') + } + + it('accepts `permissions: contents: read` at workflow level with none on the job', () => { + const r = checkFormatWorkflow(atWorkflowLevel(WELL_FORMED, ' contents: read\n', null)) + expect(r.ok, r.message).toBe(true) + }) + + it('accepts `permissions: read-all` at workflow level', () => { + const r = checkFormatWorkflow( + mutate( + mutate(WELL_FORMED, JOB_PERMISSIONS, '', 'the job permissions'), + '\njobs:\n', + '\npermissions: read-all\n\njobs:\n', + 'the jobs key', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('rejects a WRITE scope at workflow level inherited by the job, naming the level', () => { + const r = checkFormatWorkflow(atWorkflowLevel(WELL_FORMED, ' contents: write\n', null)) + expect(r.ok).toBe(false) + expect(r.message).toContain('WRITE') + expect(r.message).toContain('workflow-level') + }) + + it('rejects `permissions: write-all` at workflow level', () => { + const r = checkFormatWorkflow( + mutate( + mutate(WELL_FORMED, JOB_PERMISSIONS, '', 'the job permissions'), + '\njobs:\n', + '\npermissions: write-all\n\njobs:\n', + 'the jobs key', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('WRITE') + }) + + it("accepts a job's own `contents: read` under a workflow-level write (the job's block REPLACES it)", () => { + const r = checkFormatWorkflow( + atWorkflowLevel(WELL_FORMED, ' contents: write\n', JOB_PERMISSIONS), + ) + expect(r.ok, r.message).toBe(true) + }) + + it("rejects a job's own write under a workflow-level read", () => { + const r = checkFormatWorkflow( + atWorkflowLevel( + WELL_FORMED, + ' contents: read\n', + ' permissions:\n contents: write\n', + ), + ) + expect(r.ok).toBe(false) + expect(r.message).toContain('WRITE') + expect(r.message).not.toContain('workflow-level') + }) + + it('still rejects no `permissions:` at either level, naming both', () => { + const r = checkFormatWorkflow(mutate(WELL_FORMED, JOB_PERMISSIONS, '', 'the job permissions')) + expect(r.ok).toBe(false) + expect(r.message).toContain('declares no `permissions:`') + expect(r.message).toContain('workflow level') + }) + + it('stays green on the shipped workflow under each of the three spellings', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const quoted = mutate(shipped, CHECK_RUN, ' run: "pnpm format:check"\n') + expect(checkFormatWorkflow(quoted).ok, checkFormatWorkflow(quoted).message).toBe(true) + const crlf = shipped.replace(/\n/g, '\r\n') + expect(checkFormatWorkflow(crlf).ok, checkFormatWorkflow(crlf).message).toBe(true) + const inherited = atWorkflowLevel(shipped, ' contents: read\n', null) + expect(checkFormatWorkflow(inherited).ok, checkFormatWorkflow(inherited).message).toBe(true) + }) +}) + +// Every key that decides WHAT or HOW the check runs was allow-listed on the checking +// STEP — and a job-level or workflow-level key reaches that step anyway, so the step +// rule was bypassable by RELOCATION, the same shape as the `working-directory:` / +// `defaults:` pair. `env:` is the measured case, end to end: +// node prettier/bin/prettier.cjs --list-different bad.ts → prints `bad.ts`, exit 1 +// NODE_OPTIONS=--require=$PWD/shim.js node … --list-different bad.ts → prints `bad.ts`, exit 0 +// (`shim.js` is one line: `process.on('exit', () => { process.exitCode = 0 })`; prettier +// is the repo's own pinned 3.6.2.) And a job-level `env:` DOES reach a step that declares +// none of its own — GitHub probe run 33724282486 on PR #477 logged `D5-JOB-ENV=from-job-env`. +// So `pnpm format:check` still NAMES the offending file and the `format` context reports +// SUCCESS on unformatted code: the `with: ref: main` loss class (run 33635537234), spelled +// as a job key. +describe('the workflow and its jobs carry only allow-listed keys (#413)', () => { + const JOB_HEADER = ' format:\n runs-on: ubuntu-latest\n' + + const jobKeys: [string, string][] = [ + [ + '`env:` carrying NODE_OPTIONS', + ' format:\n env:\n NODE_OPTIONS: --require ./scripts/shim.js\n runs-on: ubuntu-latest\n', + ], + [ + '`container:`, third-party code as a job key', + ' format:\n container: node:20\n runs-on: ubuntu-latest\n', + ], + [ + '`services:`', + ' format:\n services:\n redis:\n image: redis\n runs-on: ubuntu-latest\n', + ], + ['`environment:`', ' format:\n environment: production\n runs-on: ubuntu-latest\n'], + ['`outputs:`', ' format:\n outputs:\n files: x\n runs-on: ubuntu-latest\n'], + [ + '`concurrency:` on the job', + ' format:\n concurrency: format\n runs-on: ubuntu-latest\n', + ], + ] + + for (const [label, header] of jobKeys) { + it(`fails on a job declaring ${label}`, () => { + const r = checkFormatWorkflow(mutate(WELL_FORMED, JOB_HEADER, header, label)) + expect(r.ok, `${label}: ${r.message}`).toBe(false) + expect(r.message, label).toContain('a job may carry only') + }) + } + + it('fires on the SHIPPED workflow when the job gains an `env:`', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + 'jobs:\n format:\n', + 'jobs:\n format:\n env:\n NODE_OPTIONS: --require ./scripts/shim.js\n', + 'the shipped job header', + ), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('`env`') + }) + + it('fails on a workflow-level `env:`, which every job and step inherits', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'jobs:\n', + 'env:\n NODE_OPTIONS: --require ./scripts/shim.js\njobs:\n', + 'the jobs key', + ), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('at workflow level') + expect(r.message).toContain('`env`') + }) + + // GitHub itself refuses an unknown top-level key — probe run 33724281525 on PR #477 + // (`x-base:` beside `on:`/`jobs:`) produced a run with ZERO jobs, i.e. invalid workflow + // file. The allow-list agrees with the producer rather than tolerating it. + it('fails on an unknown top-level key, which GitHub rejects outright', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, 'jobs:\n', 'x-base:\n runs-on: ubuntu-latest\njobs:\n', 'the jobs key'), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('at workflow level') + expect(r.message).toContain('`x-base`') + }) + + // Over-reach: every key the workflow legitimately carries stays green, and the keys + // owned by a rule of their own keep reporting THAT rule's message, not the generic one. + it('accepts the keys the workflow and its job actually need', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' format:\n runs-on: ubuntu-latest\n permissions:\n contents: read\n', + ' format:\n name: format\n runs-on: ubuntu-latest\n timeout-minutes: 10\n permissions:\n contents: read\n', + 'the job header', + ), + ) + expect(r.ok, r.message).toBe(true) + }) + + it('accepts a workflow-level `permissions:` and `name:`', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, 'jobs:\n', 'permissions:\n contents: read\njobs:\n', 'the jobs key'), + ) + expect(r.ok, r.message).toBe(true) + }) + + const ownedElsewhere: [string, string, string][] = [ + ['if', " format:\n if: github.event_name == 'push'\n runs-on: ubuntu-latest\n", 'if:'], + [ + 'strategy', + ' format:\n strategy:\n matrix:\n node: [20]\n runs-on: ubuntu-latest\n', + 'strategy', + ], + [ + 'defaults', + ' format:\n defaults:\n run:\n working-directory: packages/dev-tools\n runs-on: ubuntu-latest\n', + 'defaults', + ], + ] + + for (const [key, header, cause] of ownedElsewhere) { + it(`reports \`${key}:\` through its own rule, not the key allow-list`, () => { + const r = checkFormatWorkflow(mutate(WELL_FORMED, JOB_HEADER, header, key)) + expect(r.ok, `${key}: ${r.message}`).toBe(false) + expect(r.message, key).toContain(cause) + expect(r.message, key).not.toContain('a job may carry only') + }) + } +}) + +// The remedy's shell was the module's LAST deny-list (write-mode formatters, `${{`, +// `secrets.`), and it fell to the argument that retired the `uses:` and `with:` ones: a +// deny-list of formatters waves through the next formatter published. Measured on the +// shipped file before this rule: a remedy body of `echo "::error…pnpm format"` + +// `npx dprint fmt` + `git commit -am style && git push` → ok=TRUE. The remedy SAYS what +// to run; it does not run anything. +describe('a failure-path step only says what to run (#413)', () => { + const REMEDY = ` run: echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n` + const remedyBody = (body: string) => + mutate(WELL_FORMED, REMEDY, ` run: |\n${body}`, 'the remedy step') + const KEEP = ` echo "::error::Not formatted. Run 'pnpm format' locally and commit the result."\n` + + const banned: [string, string][] = [ + ['a formatter no offender list names', ' npx dprint fmt\n'], + ['an auto-commit', ' git commit -am style && git push\n'], + ['a checkout rewrite', ' git checkout origin/main -- .\n'], + ['a command substitution inside the message', ' echo "$(prettier --write .)"\n'], + ['a backtick substitution', ' echo "`prettier --write .`"\n'], + ['an unquoted echo argument', ' echo pnpm format && rm -rf .\n'], + ] + + for (const [label, extra] of banned) { + it(`fails on a remedy that also runs ${label}`, () => { + const r = checkFormatWorkflow(remedyBody(`${KEEP}${extra}`)) + expect(r.ok, `${label}: ${r.message}`).toBe(false) + }) + } + + it('names the remedy allow-list as the cause of the dprint row', () => { + const r = checkFormatWorkflow(remedyBody(`${KEEP} npx dprint fmt\n`)) + expect(r.ok).toBe(false) + expect(r.message).toContain('REMEDY_COMMAND_LINES') + }) + + it('fires on the SHIPPED workflow when the remedy gains a formatter', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow( + mutate( + shipped, + ' echo "::error title=Formatting check failed::The files listed in the previous step are not formatted. Run \'pnpm format\' locally, commit the result, and push again. CI never rewrites your branch."\n', + ' echo "::error title=Formatting check failed::The files listed in the previous step are not formatted. Run \'pnpm format\' locally, commit the result, and push again. CI never rewrites your branch."\n npx dprint fmt\n', + 'the shipped remedy body', + ), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('REMEDY_COMMAND_LINES') + }) + + const accepted: [string, string][] = [ + ['a single-quoted echo', ` echo '::error::Not formatted. Run pnpm format locally.'\n`], + ['two quoted echoes', `${KEEP} echo "See the step above for the file list."\n`], + [ + 'an echo with a flag', + ` echo -e "::error::Not formatted. Run 'pnpm format' locally."\n`, + ], + ['a printf', ` printf '%s\\n' "::error::Not formatted. Run 'pnpm format' locally."\n`], + ] + + for (const [label, body] of accepted) { + it(`accepts a remedy that is ${label}`, () => { + const r = checkFormatWorkflow(remedyBody(body)) + expect(r.ok, `${label}: ${r.message}`).toBe(true) + }) + } + + it('accepts the shipped remedy, which is one quoted echo', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + expect(checkFormatWorkflow(shipped).ok).toBe(true) + }) + + // The remedy still has to NAME the remedy — the allow-list narrows the shell, it does + // not replace AC1's content rule. + it('still fails when the allow-listed echo says nothing about `pnpm format`', () => { + const r = checkFormatWorkflow(remedyBody(' echo "::error::Something went wrong."\n')) + expect(r.ok).toBe(false) + expect(r.message).toContain('names the remedy') + }) +}) + +// `jobKeyProblems` allow-lists the KEYS a job may declare and `runs-on` is on that list, +// so until `runsOnProblems` the VALUE was free: measured on the SHIPPED file, `runs-on: +// self-hosted` was `ok=true`, and so was `[self-hosted, linux]` — one line relocating the +// `format` check onto a machine the pull request chose, with every other rule green. It is +// the `container:` argument spelled as a value: the machine decides what `pnpm` and +// `prettier` are, and on a PUBLIC repo a `pull_request` run executes the PR's OWN copy of +// this file. +// +// Boundary proof, GitHub itself (probe workflow on this PR, run 33729726089, reverted): +// runs-on: ubuntu-24.04 -> job `d7-ubuntu-2404` completed success (x86_64, 24.04) +// runs-on: ubuntu-22.04 -> job `d7-ubuntu-2204` completed success (x86_64, 22.04) +// runs-on: 'ubuntu-latest' -> job `d7-quoted-ubuntu` completed success, RUNNER_ENVIRONMENT=github-hosted +// runs-on: [ubuntu-latest] -> job `d7-list-ubuntu` completed success, RUNNER_ENVIRONMENT=github-hosted +// runs-on: self-hosted -> job `d7-self-hosted` still `queued`, never started +// runs-on: [self-hosted, …] -> job `d7-self-hosted-list` still `queued`, never started +// (`gh api repos/foomakers/pair/actions/runners` -> `total_count: 0`.) So the quoted and +// one-element-list spellings are the SAME machine, not a weaker one; and self-hosted reads +// today as a BLOCKED merge — the context stays PENDING, never SUCCESS — which lasts exactly +// as long as no self-hosted runner is registered on this public repo. +// +// A LIST IS ANDed. Boundary proof, GitHub itself (probe workflow on this PR, run +// 33782665948, reverted), 20 minutes after the run started: +// runs-on: [ubuntu-latest] -> `d8-one-ubuntu` success +// runs-on: [ubuntu-latest, ubuntu-latest] -> `d8-two-same-label` success (GitHub dedupes) +// runs-on: [ubuntu-latest, ubuntu-22.04] -> `d8-two-ubuntu` still `queued`, never started +// runs-on: [ubuntu-latest, ubuntu-24.04, ubuntu-22.04]-> `d8-three-ubuntu` still `queued`, never started +// and, in a second probe workflow (run 33782655630): +// runs-on: [[ubuntu-latest]] -> the RUN failed outright, no job scheduled +// So a value every element of which is on the allow-list can still name NO machine: the +// rule counts labels (exactly one), it does not merely allow-list each of them. The +// duplicate spelling is measured GREEN on GitHub and rejected here anyway — one label is +// the contract, and the direction of that narrowing is a false red, never a false green. +describe('runs-on is an allow-list of GitHub-hosted Ubuntu labels (#413)', () => { + const runsOn = (source: string, value: string) => + mutate(source, ' runs-on: ubuntu-latest\n', ` runs-on: ${value}\n`, 'the runs-on line') + + const accepted: [string, string][] = [ + ['the pinned 24.04 image', 'ubuntu-24.04'], + ['the pinned 22.04 image', 'ubuntu-22.04'], + ['a single-quoted label', "'ubuntu-latest'"], + ['a double-quoted label', '"ubuntu-24.04"'], + ['a one-element flow sequence', '[ubuntu-latest]'], + ['a one-element block sequence', '\n - ubuntu-latest'], + ] + + for (const [label, value] of accepted) { + it(`accepts ${label}`, () => { + const r = checkFormatWorkflow(runsOn(WELL_FORMED, value)) + expect(r.ok, `${value}: ${r.message}`).toBe(true) + }) + } + + // The parser resolves the alias, so the rule reads the LABEL, not the `*runner` text. + it('accepts a label reached through an anchor and an alias', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + ' runs-on: ubuntu-latest\n', + ' runs-on: &runner ubuntu-latest\n', + 'the runs-on line', + ) + + ' helper:\n runs-on: *runner\n permissions:\n contents: read\n steps:\n - run: pnpm install\n', + ) + expect(r.ok, r.message).toBe(true) + }) + + const rejected: [string, string][] = [ + ['a bare self-hosted label', 'self-hosted'], + ['a self-hosted label list', '[self-hosted, linux]'], + ['a list that only CONTAINS an allowed label', '[ubuntu-latest, self-hosted]'], + ['a runner group mapping', '\n group: ubuntu-runners'], + ['a runner group with labels', '\n group: ubuntu-runners\n labels: [ubuntu-latest]'], + ['another GitHub-hosted OS', 'macos-latest'], + ['a Windows image', 'windows-latest'], + ['a larger runner label', 'ubuntu-latest-4-cores'], + ['an arm image', 'ubuntu-24.04-arm'], + ['a label chosen by a repository variable', '${{ vars.RUNNER }}'], + ['a label chosen by a matrix', '${{ matrix.os }}'], + ['an empty value', ''], + ['an empty list', '[]'], + ['a nested sequence', '[[ubuntu-latest]]'], + // A LIST IS ANDed BY GITHUB (probe run 33782665948, table above): every extra label + // narrows the runner set, and no GitHub-hosted image carries two image labels, so a + // list of allow-listed labels reaches NO runner — the job queues and the `format` + // context stays PENDING, the same blocked-merge loss as `self-hosted`. + ['a two-label list of allow-listed images', '[ubuntu-latest, ubuntu-22.04]'], + ['a three-label list of allow-listed images', '[ubuntu-latest, ubuntu-24.04, ubuntu-22.04]'], + ['a two-label block sequence', '\n - ubuntu-latest\n - ubuntu-22.04'], + // MEASURED GREEN on GitHub (`d8-two-same-label` completed success — GitHub dedupes the + // label set): rejected anyway. The contract is ONE label, and a spelling nobody needs + // is not worth a second acceptance path; the direction is safe (a false red, never a + // false green). + ['the same allow-listed label twice', '[ubuntu-latest, ubuntu-latest]'], + ['a sequence carrying a mapping', '[{group: ubuntu-runners}]'], + ] + + for (const [label, value] of rejected) { + it(`fails on ${label}`, () => { + const r = checkFormatWorkflow(runsOn(WELL_FORMED, value)) + expect(r.ok, `${value}: ${r.message}`).toBe(false) + expect(r.message, value).toContain('runs-on') + }) + } + + // The rejection QUOTES the value back, so a non-scalar item must not flatten into the + // spelling the rule accepts: `[[ubuntu-latest]]` rendered as `[ubuntu-latest]` reads + // "you wrote `[ubuntu-latest]`, which is rejected; use `[ubuntu-latest]`". + const rendered: [string, string, string][] = [ + ['a nested sequence', '[[ubuntu-latest]]', '`runs-on: [["ubuntu-latest"]]`'], + ['a sequence carrying a mapping', '[{group: x}]', '`runs-on: [{"group":"x"}]`'], + [ + 'a sequence carrying a null item', + '[ubuntu-latest, null]', + '`runs-on: [ubuntu-latest, null]`', + ], + [ + 'a two-label list', + '[ubuntu-latest, ubuntu-22.04]', + '`runs-on: [ubuntu-latest, ubuntu-22.04]`', + ], + ] + + for (const [label, value, quoted] of rendered) { + it(`quotes ${label} back distinguishably`, () => { + const r = checkFormatWorkflow(runsOn(WELL_FORMED, value)) + expect(r.ok, `${value}: ${r.message}`).toBe(false) + expect(r.message, value).toContain(quoted) + }) + } + + it('never quotes a REJECTED value back as the accepted one-label spelling', () => { + const r = checkFormatWorkflow(runsOn(WELL_FORMED, '[[ubuntu-latest]]')) + expect(r.message).not.toContain('`runs-on: [ubuntu-latest]`') + }) + + it('fails when the job declares no `runs-on` at all', () => { + const r = checkFormatWorkflow( + mutate(WELL_FORMED, ' runs-on: ubuntu-latest\n', '', 'the runs-on line'), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('no `runs-on:`') + }) + + // The rule is per JOB, not per host job: a second job on a machine the PR chose runs + // `pnpm install` — PR-authored lifecycle scripts — on that machine just the same. + it('fails on a SECOND job pinned to a self-hosted runner', () => { + const r = checkFormatWorkflow( + mutate( + WELL_FORMED, + 'jobs:\n', + 'jobs:\n helper:\n runs-on: self-hosted\n steps:\n - run: pnpm install\n', + 'the jobs key', + ), + ) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('runs-on') + expect(r.message).toContain('helper') + }) + + it('fires on the SHIPPED workflow when it is relocated to a self-hosted runner', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(runsOn(shipped, 'self-hosted')) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('runs-on') + }) + + it('fires on the SHIPPED workflow when it is relocated through a label list', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(runsOn(shipped, '[self-hosted, linux]')) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('runs-on') + }) + + it('fires on the SHIPPED workflow when it is widened to a two-label list', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(runsOn(shipped, '[ubuntu-latest, ubuntu-22.04]')) + expect(r.ok, r.message).toBe(false) + expect(r.message).toContain('`runs-on: [ubuntu-latest, ubuntu-22.04]`') + }) + + it('accepts the shipped workflow, which runs on a GitHub-hosted Ubuntu image', () => { + const shipped = readFileSync(FORMAT_WORKFLOW, 'utf-8') + const r = checkFormatWorkflow(shipped) + expect(r.ok, r.message).toBe(true) + }) + + // The message is the entire user-facing product of a RED gate, and its two failure modes + // have no other guard: it asserts things about GitHub that no rule re-measures, and + // prettier does not reflow a string literal, so a spliced sentence leaves a ragged line + // nothing in the toolchain catches. + describe('the rejection message says only what was measured, at one column width', () => { + const flat = (m: string) => m.replace(/\s+/g, ' ') + const rejectionFor = (value: string) => { + const r = checkFormatWorkflow(runsOn(readFileSync(FORMAT_WORKFLOW, 'utf-8'), value)) + expect(r.ok, `${value}: ${r.message}`).toBe(false) + return r.message + } + + // MEASURED AT THE PRODUCER. Probe run 33782665948, re-read this round with + // `gh api repos/foomakers/pair/actions/runs/33782665948/jobs`: job `d8-two-same-label` + // (`runs-on: [ubuntu-latest, ubuntu-latest]`) got runner `GitHub Actions 1000003379`, + // ran `Set up job` / its `echo` / `Complete job` and completed `success` — GitHub + // dedupes the label SET. Only the DIFFERENT-label rows sat unscheduled: + // `d8-two-ubuntu` and `d8-three-ubuntu` show `runner_name: ""`, zero steps, still + // queued 23 minutes later. So "the job never gets a runner and the context stays + // PENDING" is true of two DIFFERENT labels and FALSE of a repeated one — and this + // message tags it `(measured)`, on a value a contributor can disprove in one push. + it('scopes the never-starts claim to DIFFERENT labels', () => { + const m = flat(rejectionFor('[ubuntu-latest, ubuntu-latest]')) + expect(m).toMatch(/two DIFFERENT allow-listed labels are ANDed[^.]*never starts/) + expect(m).not.toMatch(/A list of TWO allow-listed labels[^.]*never gets a runner/) + }) + + it('names the repeated label as its own case, not as a job that never runs', () => { + const m = flat(rejectionFor('[ubuntu-latest, ubuntu-latest]')) + expect(m).toMatch(/EXACTLY ONE label/) + expect(m).toMatch(/repeated label is rejected too/) + expect(m).toMatch(/GitHub dedupes/) + }) + + // The same paragraph is printed for every rejected value, so the two rows that DO + // queue forever must still read as measured fact. + it('keeps the PENDING claim for the values that really never start', () => { + for (const value of ['self-hosted', '[ubuntu-latest, ubuntu-22.04]']) { + expect(flat(rejectionFor(value)), value).toMatch(/PENDING, never SUCCESS \(measured\)/) + } + }) + + // Line 0 is the file header and line 1 carries the quoted value (its width is the + // contributor's, not ours); the last line closes the paragraph. Everything between is + // body text and must render as one column (the paragraph is wrapped at 95; the floor + // is 80, which is where greedy wrapping lands when the next word is a long one) — a + // 48-char line beside 95-char neighbours is what a mid-line splice leaves behind. + it('wraps every body line to one column width', () => { + for (const value of [ + 'self-hosted', + '[ubuntu-latest, ubuntu-latest]', + '[ubuntu-latest, ubuntu-22.04]', + ]) { + for (const line of rejectionFor(value).split('\n').slice(2, -1)) { + expect(line.length, `${value}: ${line.length} chars — ${line}`).toBeGreaterThanOrEqual(80) + expect(line.length, `${value}: ${line.length} chars — ${line}`).toBeLessThanOrEqual(99) + } + } + }) + }) +}) diff --git a/packages/dev-tools/src/quality-gates/format-workflow-composition.ts b/packages/dev-tools/src/quality-gates/format-workflow-composition.ts new file mode 100644 index 000000000..7d1b0aef7 --- /dev/null +++ b/packages/dev-tools/src/quality-gates/format-workflow-composition.ts @@ -0,0 +1,1967 @@ +/** + * format-workflow-composition — keeps CI's formatting check real (story #413). + * + * Since #394 the pre-push hook CHECKS formatting instead of applying it, which is + * the right shape for a hook — but it left the local hook as the ONLY enforcement + * point. `--no-verify`, or a contributor whose hooks are not installed, lands + * unformatted code with every CI check green. `.github/workflows/format.yml` is + * the fix; this module is what stops that fix from quietly decaying. + * + * THE FILE IS PARSED, NOT READ LINE BY LINE (ADL 2026-09-01 + * `workflow-guard-rejects-what-it-cannot-read`, amended 2026-09-03). `yaml@2.8.2` — this + * repo's adopted parser — resolves the workflow to the same document GitHub runs, once, + * at the top of `checkFormatWorkflow`. Every rule below reads NODES. The hand-rolled + * line reader this module used between rounds 5 and 14 (`blockUnder`, `listValueOf`, + * `scalarAt`, `keysAt`, `stepsOf`, `withoutBlockScalars`) is retired, and with it the + * four rule families that existed only to reject the spellings it could not follow: + * flow mappings, JSON-spelled steps, anchors/aliases, indentless sequences, CRLF and + * quoted scalars are now READ, and their resolved values are subject to every rule + * here. The reader failed OPEN on those spellings (round 5: `pull_request: { branches: + * [main], paths-ignore: ['**\/*.md'] }` left every trigger rule passing vacuously) and + * then failed CLOSED on five correct ones (rounds 12–14); a parser does neither. + * + * Two properties the parse gives for free and the rules rely on: + * + * - **fail-closed on an unreadable file**: a parse error is itself a reported problem, + * and so is a duplicate key (`yaml@2.8.2` rejects both) — the direction the rejection + * list used to hold. + * - **`run:` bodies stay shell, never YAML**. The parser hands each `run:` scalar over + * as a string; shell comments inside it are stripped quote-aware (`stripLineComment`), + * because `#` inside quotes is not a comment to bash and cutting there once removed an + * executing `prettier --write .` from view. + * + * Every regression this guards against is a one-line edit that leaves the + * workflow LOOKING like enforcement: + * + * - a `paths-ignore:` key. This is why the check is a dedicated workflow and not + * a job in `ci.yml`: that workflow carries `paths-ignore: ['.changeset/**']` at + * WORKFLOW level, which a job inherits — so a PR touching only `.changeset/**` + * would run no formatting check at all. Trigger coverage is part of check + * coverage. + * - `paths:`, the same hole spelled positively — an allow-list excludes everything + * it does not name, so a markdown-only PR runs no check at all. + * - a trigger narrowed off the base branch (`pull_request.branches: [release]`, or + * its negative spelling `branches-ignore: [main]`) or off the events that matter + * (`types: [closed]` runs the check only once the PR is closed). Whatever shapes + * the trigger shapes the coverage. + * - `pull_request_target` instead of `pull_request`, which hands the base repo's + * credentials to a fork's head commit. + * - a write-mode formatter, or a formatting auto-commit. The ADL 2026-07-31 ban + * ("the gate reports, the developer fixes deliberately") is repo-wide, not + * hook-specific — CI repairing the branch is the same defect one layer up. Held by + * the SHELL ALLOW-LISTS below, not by the formatter deny-list: every `run:` in this + * workflow is one of three things and each is allow-listed (the checking command as + * an equality, `SETUP_COMMAND_LINES` for the toolchain install, `REMEDY_COMMAND_LINES` + * for the failure-path message). `findWriteModeFormatters` still runs across all of + * them, as the repo-wide statement of the ban rather than as the surface that holds it. + * - the same write, spelled `uses:` instead of `run:`. The write-mode scan reads + * `run:` blocks, so a formatting ACTION was invisible to it — and placed before the + * checking step it needs no permission at all, because it never pushes. Hence an + * allow-list of the three actions this job needs (`usesProblems`). + * - a checking command that is not THE command. "Some run block references the + * script" was satisfied by `pnpm --filter= format:check` and by `cd && + * pnpm format:check`, each making CI check a strict subset of the tree the developer + * and the hook check — AC4's divergence, reinstated green. The command is an + * equality (`checkCommandProblems`). + * - the job that RUNS the check renamed, or displaced by a decoy. The job's DISPLAY + * NAME — its id when `name:` is absent — is the status context GitHub publishes: + * `format` is what way-of-working requires and what AC8 tells branch protection to + * list, so `fmt:` deletes the context without touching a rule — and a `format:` job + * that only echoes, beside a `worker:` job carrying the real steps, reports SUCCESS + * in that context after an echo. Asserted on the HOST job, never on the set of job + * names (`jobIdentityProblems`). And the id is not the whole story: a `name:` line + * renames the published context as surely as the id does (this repo's version.yml, + * job id `version`, is published as `Create version commits and tags`), and a + * `strategy.matrix` suffixes its values (`format (20)`), so `name:` other than + * `format` and any `strategy:` on the host are rejected, and `name: format` on any + * OTHER job is the decoy spelled through the display name. + * - a `push:` trigger filtered by `tags:` alone. `branches` and `tags` are independent + * filters and GitHub fires the event only for the ref kinds a filter names, so with no + * `branches:` the workflow never runs on a push to any branch — measured on this + * repo's release.yml, whose `push` runs are all tags. "No filter ⇒ every branch" was + * the wrong reading for it (`tagFilterProblems`). + * - a `concurrency.group` that is not keyed on `github.ref`. Every claim below about + * the two triggers "never meeting" holds only because of that key; `group: format` + * puts every run in one group, so a PR push cancels an in-progress run on `main` + * (the cancel is conditioned on the PR event, which the PR push is) and that commit + * ends with no verdict (`groupProblems`). + * - dropping `push: main`, so drift on the base branch is invisible. + * - dropping `concurrency`, so a superseded run keeps burning a runner and + * reporting a stale verdict for a ref that has already moved on. + * - `continue-on-error: true`, ANY `if:` on the job, ANY `needs:` on the job, ANY `if:` + * on the step that runs `format:check`, a step `if:` elsewhere that is not a SCOPED + * `failure()` guard, or + * a write-scoped `permissions:`. None of these touch a trigger or a step COMMAND, + * yet each turns the `format` context into one that cannot fail, never runs, or + * hands a write-scoped token to a job that executes PR-authored lifecycle scripts + * (`pnpm install`). AC5's "safe on fork PRs by construction, not by review" is only + * construction if the construction is asserted. `if:` is an ALLOW-list on purpose: + * a deny-list of literal falses (`if: false`) waves through every never-true + * EXPRESSION — `if: github.event_name == 'workflow_dispatch'` on the job, or + * `if: github.event_name == 'push'` on the checking step, both leave a SUCCESS + * `format` context on unformatted code. And `failure()` is not the allow-list + * either: on the CHECKING step it is false on a normal PR, so the check is skipped + * and the job ends green wearing the remedy's own spelling — hence no condition at + * all there, and everywhere else `failure()` AND the check step's own outcome. + * `needs:` sits in the same bullet because it is that neutralization with no + * condition written anywhere: a job whose dependency fails or is skipped never runs, + * is reported skipped, and a skipped job's required check reads SUCCESSFUL. + * - dropping the failure-path remedy, or widening it. `--list-different` prints the + * offending file and suppresses prettier's own "run with --write to fix" line, so + * a red check without that step hands the contributor this story exists for — + * hooks not installed, pushed with `--no-verify` — a bare filename and no + * instruction (AC1). And an unscoped `if: failure()` is JOB-scoped, so a failed + * `pnpm install` is annotated "not formatted, run `pnpm format`" — a confident + * wrong diagnosis over the real cause. EVERY failure-path step is therefore + * conditioned on the checking step's own `outcome`: one scoped step does not + * license a second, unscoped one beside it. And that scope must RESOLVE, not + * merely be spelled — `steps` is job-local and empty before the step has run, so a + * remedy in a SECOND JOB, or ABOVE the checking step in the same job, carries a + * condition false on every run: the remedy fires never and AC1's contributor reads + * the bare filename anyway. Nor is naming the context enough: the condition must + * COMPARE it to `'failure'`. `outcome` holds one of four values, so + * `steps..outcome == 'success'` still mentions the check and is false exactly + * when the check fails — the same never-firing remedy from a one-token edit. Nor is + * the comparison being PRESENT enough: it must DECIDE. `&&` binds tighter than + * `||`, so `failure() && || steps.install.outcome == 'failure'` keeps the + * scope spelled exactly right and puts the broken-install annotation back; and a + * CONJUNCT can narrow the remedy to zero — `&& github.event_name == 'push'` is false + * on every pull_request run (measured: check fails, remedy skipped). The condition + * is a conjunction of allow-listed terms (`failure()`, `!cancelled()`/`!success()`, + * the scope equality); `||`, a negated scope and any other conjunct are rejected. + * - the checkout's `with:`. `uses:` was matched on the action NAME; `with: ref: main` + * makes actions/checkout check out `main` instead of the PR merge ref, so a PR + * carrying an unformatted file gets a SUCCESSFUL `format` context (measured on + * GitHub). `repository`, `path`, `sparse-checkout` are the same loss; the checkout's + * inputs are an allow-list of fetch mechanics (`checkoutInputProblems`). + * - `working-directory:` on the checking step, or `defaults:` on a job or the + * workflow — `cd` spelled as a key, invisible to the command equality, so CI runs a + * package's own `format:check` (a subset) the moment one declares the script. The + * checking step carries only `name`, `id`, `run`, `timeout-minutes` + * (`checkStepKeyProblems`, `defaultsProblems`). + * - `env:` or `container:` on the JOB — the same relocation one level up. Every key + * that decides WHAT or HOW the check runs is allow-listed on the checking STEP, and + * a job-level key reaches that step anyway, so the step-level rule was bypassable by + * relocation exactly as `working-directory:`/`defaults:` were. Measured end-to-end + * against the repo's own pinned prettier 3.6.2: `node bin/prettier.cjs + * --list-different bad.ts` prints `bad.ts` and exits 1, while + * `NODE_OPTIONS=--require=./shim.js node bin/prettier.cjs --list-different bad.ts` + * prints `bad.ts` and exits 0 (`shim.js` being one line: `process.on('exit', () => { + * process.exitCode = 0 })`). A job-level `env: NODE_OPTIONS:` reaches the checking + * step — measured on GitHub, probe run 33724282486 on PR #477, `D5-JOB-ENV=from-job-env` + * logged from a step that declares no `env:` of its own — so `pnpm format:check` still + * NAMES the offending file and the job goes GREEN. `container:` is the `uses:` + * third-party-code argument spelled as a job key (the image decides what `pnpm` and + * `prettier` even are), and `services:` with it. Hence the workflow's own keys and + * every job's keys are allow-lists (`workflowKeyProblems`, `jobKeyProblems`). + * - `runs-on:` pointing off GitHub's runners. Those two rules allow-list which KEYS may + * appear; `runs-on` is on the job's list because the job needs a machine, and its VALUE + * picks which one — the same "what are `pnpm` and `prettier` here" question `container:` + * asks, and a machine also decides who watches the run. On a PUBLIC repo a + * `pull_request` run executes the PR's own version of this file, so the value is chosen + * by the pull request. `runs-on: self-hosted` (and `[self-hosted, linux]`) was `ok=true` + * with every other rule green, so the value is an allow-list too: GitHub-hosted Ubuntu + * labels, alone or as a one-label list (`RUNNER_LABELS`, `runsOnProblems`). On this repo + * today that regression reads as a BLOCKED merge rather than a green check — no + * self-hosted runner is registered (`total_count: 0`), so the job queues and `format` + * stays pending (measured, probe run 33729726089) — which lasts exactly as long as that + * stays true. ONE label, counted, not "every label allow-listed": GitHub ANDs a label + * list, so `[ubuntu-latest, ubuntu-22.04]` names no machine at all (measured, probe run + * 33782665948) and its job queues forever — the same blocked-merge loss as `self-hosted`, + * reached by a value every element of which is on the allow-list. + * - `cancel-in-progress` and `concurrency.group` read as SUBSTRINGS. `!(github.event_name + * == 'pull_request')` contains the accepted equality and inverts it; `format-${{ + * github.run_id }}-${{ github.ref }}` contains the ref key and is unique per run. + * Both values are anchored allow-lists over the WHOLE value now. + * - the shell of a step that is neither the check nor its remedy. `run:` was the last + * deny-list surface in this module, and the `with: ref: main` loss has a shell + * spelling no formatter list names: `run: git fetch origin main && git checkout + * origin/main -- .` before the check, or `pnpm install && find . -name '*.ts' + * -delete` — each `ok=true` on the shipped file, each running `pnpm format:check` on a + * tree that is not the PR's (AC2). The toolchain install is the whole allow-list: `pnpm + * install` with flags, and the corepack fallback line by line (`setupCommandProblems`). + * - the shell of the REMEDY. It is the last surface a deny-list guarded, and the same + * argument retires it: `npx dprint fmt` (a formatter no offender list names) and + * `git commit -am style && git push` beside the required `pnpm format` message were + * `ok=true`. The remedy says something; it does not do anything. Its shell is an + * allow-list of quoted `echo`/`printf` lines (`REMEDY_COMMAND_LINES`) — which is all + * it has ever contained. + * - `prettier -w`, the documented short form of `--write`, missing from the one offender + * list the AC6 ban reuses (`WRITE_MODE_FORMATTERS`, next door): `pnpm install && npx + * prettier -w .` was green while `--write` was red. + * + * Structure is asserted, never exact file text: comments, step names, action + * versions and YAML spelling must be editable without false-failing this guard. + * + * WRITE-MODE DETECTION IS NOT RE-IMPLEMENTED. `findWriteModeFormatters` and the + * transitive `expandScriptReferences` come from the pre-push guard next door — the + * workflow says `pnpm format:check`, so scanning its literal text would miss the + * likeliest regression (`pnpm format`, which only resolves to `prettier:fix` + * through the root scripts). Two copies of that offender list would drift, and the + * one that drifts is always the one guarding the newer surface. + * + * Per the gate-tooling ADL (2026-07-13) the logic lives here as a tested module, with a + * thin `main()` CLI behind a `require.main` guard (`format-workflow:check`, run by the + * root `gate:composition` — AC6). Turbo's cache is handled by the `$TURBO_ROOT$` input + * entry on `@pair/dev-tools#test` (turbo.json) — the same treatment + * `@pair/knowledge-hub#test` already uses for repo-wide artifacts — so `pnpm test` is the + * other enforcement point and is never a stale PASS. + */ +import { readFileSync } from 'fs' +import { resolve } from 'path' + +import { parseDocument } from 'yaml' + +import { + expandScriptReferences, + findWriteModeFormatters, + referencesScript, + REMEDY_SCRIPT, + ROOT_PACKAGE_JSON, + type GateCheckResult, +} from './pre-push-gate-composition' +import { REPO_ROOT } from './repo-root' + +/** The workflow this guard reads — the real file, never a fixture copy. */ +export const FORMAT_WORKFLOW = resolve(REPO_ROOT, '.github/workflows/format.yml') + +/** The one command CI must run. Same script a developer runs — that is the point. */ +export const FORMAT_CHECK_SCRIPT = 'format:check' + +/** The branch whose formatting drift must be visible post-merge, not only on PRs. */ +export const BASE_BRANCH = 'main' + +/** + * The job id — and, with no `name:` on the job, its display name, which is what GitHub + * publishes as the status context. way-of-working documents `format` as the required + * check and AC8 names it as the context branch protection must list, so renaming the + * job, giving it a `name:`, or giving it a matrix deletes that context without touching + * a rule. + */ +export const FORMAT_JOB = 'format' + +/** + * The complete set of actions this workflow may use. An ALLOW-list, like `if:`, and + * for the same reason: the write-mode scan reads `run:` blocks, so a step that writes + * through an ACTION is invisible to it, and a deny-list of formatting actions waves + * through the next one published. + */ +export const ALLOWED_USES = ['actions/checkout', 'pnpm/action-setup', 'actions/setup-node'] as const + +/** + * The `with:` inputs `actions/checkout` may carry — fetch mechanics only, i.e. inputs + * that leave the checked-out tree as the event's ref. An ALLOW-list for the same reason + * `uses:` is one: the action's `ref` input "defaults to the reference or SHA for that + * event" (action.yml) and setting it REPLACES the PR merge ref, so a `with:` the guard + * never read decided WHAT `pnpm format:check` ran on. Measured on GitHub (probe run on + * PR #477): `with: ref: main` on the shipped checkout, a PR carrying an unformatted file + * → `Check formatting: success`, `format` context SUCCESS — AC2 defeated by one line. + * `repository` and `path` change which tree the command runs on at all; `sparse-checkout` + * / `sparse-checkout-cone-mode` check out a SUBSET (AC4's `--filter=` divergence spelled + * as an input). The toolchain actions' inputs choose versions, never the tree, and are + * not constrained. + */ +export const ALLOWED_CHECKOUT_INPUTS = [ + 'fetch-depth', + 'fetch-tags', + 'show-progress', + 'persist-credentials', + 'clean', + 'set-safe-directory', +] as const + +/** + * The `pull_request` activity types the check must keep covering. GitHub's default + * is `[opened, synchronize, reopened]`; a narrowing that keeps `reopened` but drops + * either of these two runs the check where it cannot influence a review. + */ +export const REQUIRED_PR_TYPES = ['opened', 'synchronize'] as const + +/** + * The keys the WORKFLOW may declare. `env:` is the reason this is an allow-list: it is + * inherited by every job and every step, so it reaches the checking step no matter what + * the step-level allow-list says (see `jobKeyProblems`). `defaults:` has its own rule, + * which names the loss precisely, so it is not reported twice here. + */ +export const WORKFLOW_KEYS = ['name', 'on', 'concurrency', 'permissions', 'jobs'] as const +const WORKFLOW_KEYS_OWNED_ELSEWHERE = ['defaults'] as const + +/** + * The keys a JOB may declare. Same allow-list discipline as `CHECK_STEP_KEYS`, one level + * up: `env:` on the job reaches the checking step (measured on GitHub, probe run + * 33724282486 — a step declaring no `env:` printed the job-level value), and + * `NODE_OPTIONS=--require=` makes the repo's own prettier 3.6.2 print the offending + * filename and exit 0 (measured against `prettier/bin/prettier.cjs`), i.e. `format` GREEN + * on unformatted code. `container:`/`services:` decide what `pnpm` and `prettier` even + * ARE — third-party code chosen by a job key, which is the `uses:` argument one level up. + * `if`, `needs`, `strategy`, `defaults` and `continue-on-error` have their own rules and + * are not reported twice here. + * + * This is an allow-list of KEYS. `runs-on` is on it because the job needs a machine, and + * its VALUE picks which — the same question `container:` asks — so that value has an + * allow-list of its own (`runsOnProblems`). + */ +export const JOB_KEYS = ['name', 'runs-on', 'permissions', 'timeout-minutes', 'steps'] as const +const JOB_KEYS_OWNED_ELSEWHERE = [ + 'if', + 'needs', + 'strategy', + 'defaults', + 'continue-on-error', +] as const + +/** + * The machines this workflow may run on: GitHub-hosted Ubuntu images, ephemeral and + * owned by GitHub. Every other label — `self-hosted`, a runner `group:`, another + * GitHub-hosted OS (`macos-*`, `windows-*`), a larger or arm runner — is a deliberate + * edit to this list, because each changes what `pnpm` and `prettier` are on the machine + * that publishes the `format` context. Measured on GitHub (probe run 33729726089, PR + * #477): `ubuntu-24.04`, `ubuntu-22.04`, a quoted `'ubuntu-latest'` and the one-element + * list `[ubuntu-latest]` all ran on a GitHub-hosted x86_64 runner + * (`RUNNER_ENVIRONMENT=github-hosted`), so the list spelling is the same value, not a + * weaker one. + */ +export const RUNNER_LABELS = ['ubuntu-latest', 'ubuntu-24.04', 'ubuntu-22.04'] as const + +/** + * Removes `#` comments from a `run:` body so a comment can neither smuggle a banned + * pattern in nor trip the guard by merely mentioning one — but only a `#` that is + * OUTSIDE quotes. + * + * YAML's own comments never reach here: the parser drops them. This is the SHELL's + * comment rule, applied to shell text. The unconditional cut this replaces ran the wrong + * way: `#` inside quotes is not a comment to bash, so `echo "note # here"; prettier + * --write .` EXECUTES in full while the guard saw only `echo "note` — the write-mode + * formatter stripped out of view, the AC6 ban silently gone, and `pnpm format:check` + * passing on a tree CI had already rewritten. + * + * Bias is deliberate on the one ambiguity left: an UNBALANCED quote earlier in the + * line leaves a later `#` unstripped, so the guard scans more text than the shell + * runs. Scanning too much can only add a problem, never hide one. + */ +function stripLineComment(line: string): string { + let quote: string | null = null + for (let index = 0; index < line.length; index++) { + const char = line[index] + if (quote !== null) { + if (char === quote) quote = null + continue + } + if (char === '"' || char === "'") { + quote = char + continue + } + if (char === '#' && (index === 0 || /\s/.test(line[index - 1] ?? ''))) + return line.slice(0, index) + } + return line +} + +/** A `run:` body as the shell sees it: comments gone, blank edges trimmed. */ +function shellOf(run: string): string { + return run.split('\n').map(stripLineComment).join('\n').trim() +} + +type Mapping = Record + +function isMapping(value: unknown): value is Mapping { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +/** + * The values of a list-valued key. `null` means the key is ABSENT — for a trigger filter + * that means "no filter", which GitHub reads as "every value", so callers treat `null` as + * covering everything. A key present with an empty or null value is `[]`, which covers + * nothing: the two are not the same and the parser is what finally tells them apart. + */ +function filterOf(block: Mapping, key: string): string[] | null { + if (!(key in block)) return null + const value = block[key] + if (value === null || value === undefined) return [] + return Array.isArray(value) ? value.map(item => String(item)) : [String(value)] +} + +/** A scalar key's value as text, or `undefined` when the key is absent. */ +function textAt(block: Mapping, key: string): string | undefined { + if (!(key in block)) return undefined + const value = block[key] + return value === null || value === undefined ? '' : String(value) +} + +/** Every key and string scalar in the document, so a textual scan reads the whole file. */ +function everyText(node: unknown): string[] { + if (typeof node === 'string') return [node] + if (Array.isArray(node)) return node.flatMap(everyText) + if (isMapping(node)) + return Object.entries(node).flatMap(([key, value]) => [key, ...everyText(value)]) + return [] +} + +/** Every `run:` scalar in the document, as shell text. */ +function collectRuns(node: unknown): string[] { + if (Array.isArray(node)) return node.flatMap(collectRuns) + if (!isMapping(node)) return [] + const own = typeof node['run'] === 'string' ? [shellOf(node['run'])] : [] + return [ + ...own, + ...Object.entries(node).flatMap(([key, value]) => (key === 'run' ? [] : collectRuns(value))), + ] +} + +/** + * Every shell command the workflow executes: inline `run: cmd` and block scalars + * (`run: |`) alike, in document order. What a step DOES lives here — the security and + * check-only rules are asserted against these, not against the whole file, so an + * expression in `concurrency.group` is not confused with one expanded into a shell. + */ +export function extractRunBlocks(yamlText: string): string[] { + const document = parseDocument(yamlText) + if (document.errors.length > 0) return [] + try { + return collectRuns(document.toJS() as unknown) + } catch { + return [] + } +} + +/** The `run:` of a single parsed step, as shell text — `[]` when the step runs nothing. */ +function runsOf(step: Mapping): string[] { + return typeof step['run'] === 'string' ? [shellOf(step['run'])] : [] +} + +/** The root scripts a `pnpm