Skip to content

[US-482] feat: skill-local scripts ship with their skill and mirror byte-identically - #483

Open
rucka wants to merge 10 commits into
feature/US-479-delivery-workflow-to-befrom
feature/US-482-skill-local-scripts-conformance
Open

[US-482] feat: skill-local scripts ship with their skill and mirror byte-identically#483
rucka wants to merge 10 commits into
feature/US-479-delivery-workflow-to-befrom
feature/US-482-skill-local-scripts-conformance

Conversation

@rucka

@rucka rucka commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

PR Information

PR Title: [US-482] feat: skill-local scripts ship with their skill and mirror byte-identically
Story/Epic: #482 (epic #212 — Supervised automation)
Type: Feature
Priority: P1 (Should-Have)
Assignee: rucka
Labels: user story, risk:green, pr-state:to-be-reviewed
Base: feature/US-479-delivery-workflow-to-beSTACKED on #479, not main. That branch's commits are already in this history and are not this story's work.

Summary

What Changed

pnpm skills:conformance gains one failure class, checkSkillLocalScripts, wired into runChecks against the real installed root and named in the PASS summary:

  • Link half — every […](./scripts/x) / […](scripts/x) a SKILL.md links must resolve to a file inside that skill's OWN scripts/ directory. Delegates to extractLinkTargets / isCheckableTarget and the same #fragment strip checkLinks does, so a fenced authoring example, a <placeholder> and an adr-NNN- pattern path stay examples instead of becoming phantom missing scripts.
  • Mirror half — every dataset skill-local script must have a BYTE-identical twin at the path the registry's bounded flatten installs it to (<category>/<name>/scripts/<sub-path>pair-<category>-<name>/scripts/<sub-path>), sub-directories preserved.

Why This Change

A skill must be portable as ONE folder. Today a script edited in the dataset but not in .claude/skills/ (or the reverse) is a runtime surprise at the moment an agent runs the stale copy; after this it is a red gate at pnpm skills:conformance.

Story Context

User Story: As a pair maintainer shipping skills that carry their own scripts, I want the conformance check to prove every linked script ships beside its SKILL.md and every shipped script has a byte-identical installed twin, so a skill is portable as one folder and a one-sided edit is a red gate rather than a runtime surprise.

Acceptance Criteria: AC 1 (linked script exists), AC 2 (installed twin byte-identical), AC 3 (PASS on the real corpus, summary names the check) — all three covered by the sealed acceptance contract below.

Changes Made

Decisions

  • Directional, like the SKILL.md mirror guard: dataset canonical, installed derived. An installed script with no dataset source is an accepted residual of a rename, never a violation. Drift is REPORTED here; pair update repairs it.
  • A dataset-only checkout (no installed root) skips the twin half rather than reporting the whole corpus missing.
  • Equality is by content bytes, not size or mtime'ab' vs 'ba' is drift.
  • The containment boundary is the skill's OWN scripts/, not the skill folder. scripts/../helper.mjs resolving to <skill>/helper.mjs is refused even though it is portable with the folder; scripts/lib/../helper.mjs stays legal. Decided on the RESOLVED path, never by banning .. in the spelling. Authority: AC 1 ("in the skill's scripts/ directory") and the story's business rule ("files under a skill's own scripts/ directory").
  • Symlinks are FOLLOWED, and what cannot be followed is NAMED. A symlinked script or sub-directory really ships with the skill, so it is compared like any other; an entry that resolves to nothing is reported by path. The invariant is that no dataset entry is ever answered with silence.
  • statSync never escapes a try in this module. It throws ENOENT on a dangling entry, and an unhandled throw there kills the whole conformance run with no report — contradicting the module's own stated principle that an unreadable twin is never assumed identical.
  • Nested scripts are guarded, a depth-1 meta skill owning scripts/ is refused. Both read the same flatten authority (installedArtifactPath, skill-md-mirror.ts) in opposite directions, verified by running it: workflow/alpha/scripts/lib/util.mjspair-workflow-alpha/scripts/lib/util.mjs (ships inside the skill ⇒ guard it), next/scripts/router.mjspair-next-scripts/router.mjs (a SEPARATE top-level skill dir, and the corpus walk then stops finding next at all ⇒ refuse it).
  • Function split for the lint gate (max 50 lines): checkMirroredLocalScriptscompareScriptsTree.

Decision added in this revision (finding r3-9) — the walk bounds by its ANCESTOR CHAIN, not a walk-global visited set

The walk kept ONE visited set for the whole descent, keyed on realpathSync(dir). A directory reachable under two names — a real scripts/zzz-lib/ and, beside it, a sibling symlink aliasing it — really ships under BOTH, because the bounded flatten installs a twin at each mirrored path. The shared key dropped whichever name readdirSync yielded SECOND, so a drifted twin of a genuinely shipped path was answered with silence, and which half went silent was decided by directory order: one corpus, two answers.

The fix records the resolved path of the directories currently being descended and pops on exit. A sibling ALIAS is then walked once per relative path (each compared against its own twin); a true CYCLE (scripts/lib/loop -> scripts, whose target is an ancestor of the descent that reached it) is still refused, so the walk still terminates.

Two alternatives were rejected on evidence, not taste:

  1. Drop the visited set — the cheapest way to emit both names, and a true cycle then recurses until the stack dies, taking the whole gate down with no report. Row R32 is the trap that catches it.
  2. Keep the global set, order real directories before symlinked ones — satisfies the letter of "the answer must not depend on readdir order" and makes R30/R31/R32 green, but the alias is then always second and always dropped: the drift-on-alias half falls from 1 error to 0. It buys one silent half with the other. Rows R33/R34 are what catch it.

A third resolution (keep the global set but re-emit an already-visited subtree under its second relative path) is equally admissible and yields an identical entry list; the ancestor chain was chosen because it needs no second bookkeeping structure. The rows assert the ANSWER at each mirrored path, never a walk strategy.

Boundary probe at the real producer (isolated, discarded after the run): a tree with scripts/lib/util.mjs, two independent aliases scripts/a1 -> lib and scripts/a2 -> lib, and a self-alias cycle scripts/lib/self -> lib returns exactly three errors — scripts/a1/util.mjs, scripts/a2/util.mjs, scripts/lib/util.mjs — in 2 ms, without throwing. Both aliases emitted, the cycle bounded, no blowup.

Files Changed

  • Modified: packages/knowledge-hub/src/tools/skills-conformance-check.ts — the whole check (the only production path in the contract's fixScope.allowedPaths)
  • Added (test, sealed by the acceptance contract): 34 rows appended to packages/knowledge-hub/src/tools/skills-conformance-check.test.ts

Nothing else in the repository is touched: git diff feature/US-479-delivery-workflow-to-be...HEAD --name-only is exactly those two files.

Testing

Test Coverage

  • Unit: packages/knowledge-hub/src/tools/skills-conformance-check.test.ts124 passed / 124, of which 34 are this story's acceptance rows R1–R34 (witnesses, boundaries, controls, interactions).
  • End-to-end (CLI): R15 spawns the real pnpm skills:conformance (PASS, exit 0); R18 spawns the module copied into a temp ROOT over a violating fixture (FAIL — 1 violation, exit 1).
  • Wiring: R13 proves runChecks resolves the REAL installed root, so a drifted script fails the gate end to end.
  • The r3-9 cross-product is asserted in full — all four cells of {which name readdirSync yields first} × {which twin drifted}: R30 and R33 were RED at the base head (each expected [] to have a length of 1), R31 and R34 are the already-correct controls that a naive ordering fix would have silently regressed, and R32 is the true-cycle termination control.

Test Results

Sealed acceptance suite:  124 passed (124)      exit 0
pnpm test  (full turbo):  10/10 tasks           exit 0
pnpm lint:                                       exit 0
pnpm ts:check:                                   exit 0
pnpm build:                                      exit 0
pnpm quality-gate:                               exit 0
  (workflows:test, format:check, gate:composition, hygiene:check,
   smoke-modes:check, docs:staleness, skills:conformance, dup:check)
skills:conformance:       PASS — 50 skills conformant (… entrypoint depth,
                          skill-local scripts, catalog counts …)

Pre-merge tiering is disabled in this project's adoption, so the FULL adopted suite was run for CI parity rather than the risk:green base subset. No gate was bypassed; the pre-push hook (pnpm quality-gate) ran on the push that produced this head.

Testing Strategy

  • Happy path: twin present and identical → no error; the real 10-script / 6-skill corpus stays clean (R14, R15).
  • Edge cases: empty scripts/, absent installed root, orphan installed script, nested sub-directory (both outcomes), non-checkable targets (placeholder / pattern / fragment), fenced-block-only link, equal-length byte swap, a directory aliased by a sibling symlink in either readdir order.
  • Error handling: installed twin occupied by a directory (EISDIR), symlinked file, symlinked sub-directory, dangling entry inside scripts/, dangling scripts entry itself, a true symlink cycle — every one reported by path or bounded, none throwing.

Reviewer Guide

Review Focus Areas

  1. The walk's boundcollectLocalScriptEntries. The ancestor chain must be popped on exit (that is what makes a sibling alias walkable) while still refusing a directory already open in the current descent (that is what makes a cycle terminate). R30/R33 fail if the pop is missing; R32 hangs if the refusal is.
  2. The containment boundaryisWithin(<skill>/scripts, resolved). R26 (escapes both boundaries), R29 (inside the folder, outside scripts/) and R27 (normalizes back inside) are the three rows that separate the two candidate readings; only a resolved-path test against the skill's own scripts/ passes all three.
  3. The lstat/stat disagreementresolvedKind (following statSync, in a try) and entryExists (lstatSync). R28 is the trap: a statSync inside the walk with no try passes R23/R24/R25 and crashes the gate.
  4. The flatten derivationinstalledSkillDirName must keep agreeing with skill-md-mirror.ts's real transform.

Testing the Changes

git checkout feature/US-482-skill-local-scripts-conformance
pnpm install
pnpm --filter @pair/knowledge-hub exec vitest run src/tools/skills-conformance-check.test.ts
pnpm --filter @pair/knowledge-hub skills:conformance   # PASS — 50 skills conformant

Dependencies & Related Work

Blocking Dependencies

  • PR Dependency: stacked on feature/US-479-delivery-workflow-to-be; that branch merges first.

Follow-up Work

Three gaps fall outside this story's sealed fixScope and are deliberately NOT patched around here:

  1. The convention this gate now enforces — a skill is portable as ONE folder; a scripts/-prefixed link must resolve inside the skill's own scripts/; a depth-1 meta skill cannot own scripts/ — is stated nowhere in .pair/knowledge/.../skill-conventions/. The check is currently its only statement, so an author meets the rule for the first time as a red gate.
  2. red-snapshot.mjs seal writes JSON.stringify(…, null, 2), which prettier would reformat, and no .prettierignore covers it — so a tree that keeps the seal manifest fails format:check. Answered here by removing the transient manifest above the snapshot (the workflow's own Step 3.1), but the collision itself lives in the workflow script and is out of scope.
  3. This revision records no ADL, and that is a contract gap rather than a skipped decision. The a0-rev3 fixScope added .pair/adoption/decision-log/ to allowedPaths precisely so the implement process could record its decisions there, but kept mode: behavioral, under which the custody check breaches behavioral-adds-or-moves-module on any diff entry whose status is not M. A decision-log entry is always a NEW file, hence always status A — so the contract permits the path and forbids the only way to use it. Reproduced, not inferred: with the ADL committed, red-snapshot.mjs verify returned {"verified":false,"contractBreach":true,"breaches":[{"code":"behavioral-adds-or-moves-module","path":".pair/adoption/decision-log/2026-09-10-…md","status":"A"}]}. Independently, adding any file under .pair/adoption/decision-log/ also forces .pair/llms.txt to be regenerated — apps/pair-cli/src/registry/llms-index-conformance.test.ts pins the committed index byte-for-byte against the generator over the real tree — and .pair/llms.txt is not in allowedPaths at all, so even a mode fix alone would leave the contract unsatisfiable. The walk-strategy rationale therefore lives in the doc comment on collectLocalScriptEntries (in scope) and in the "Decision added in this revision" section above.

@rucka rucka added user story Work item representing a user story risk:green Classification: low risk tier labels Sep 9, 2026
@rucka rucka self-assigned this Sep 9, 2026
@rucka rucka added the pr-state:to-be-reviewed PR state: awaiting review / gate label Sep 9, 2026
@rucka

rucka commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator Author

Verdict

risk:green · cost:greenCHANGES-REQUESTED — AC 1 and AC 3 are proven and the producer is correct, custody-clean and green on the real corpus; the twin half still demands installed files that pair update provably never creates, and one of the three divergences needs no symlink at all.

Open findings: 2 (1 Major, 1 Minor). 3 Questions, informational. 1 prior finding resolved.

PR: #483 · Author: rucka · Reviewer: pair-reviewer 1/1 (independent, clean context) · Date: 2026-09-10 · Story: US-482 · Type: feature

Reviewed head 4b1192f0463ae85b4ea1dd6bc97ae44ef0ec7667 · Remote head 4b1192f0463ae85b4ea1dd6bc97ae44ef0ec7667 (equal) · Cycle canary-479-v4 r0 · Tier risk:green (pass: general)

Classification matrix — per dimension
Dimension Tier Source Note
Service/domain criticality green Criticality Table packages/knowledge-hub is listed Low.
Change/diff risk green diff footprint 2 files, +1070/−5, one module + its vitest file. No shared code, schema or infra.
Business impact green subdomain class Development Tooling Standards (Generic).
Security relevance green path heuristic Read-only filesystem comparison over a repo-controlled corpus; no untrusted input.
Coupling balance green not assessed (risk:green review set) Consumes the dataset and installed trees it already reads.

Tier = max(assessed) = green. Confirmed, not raised.

Custody and evidence

Custody: verified, no breach. red-snapshot.mjs verify-chain --pr 483 --base 421441faverified: true, breaches: [], three sealed snapshots in one chain: a0 (a3772abe, base 421441fa) → a0-rev2 (2c5bba41) → a0-rev3 (e0475a76). No sealed test blob changed, no unlisted test touched, fixScope respected.

Evidence re-run on this exact head — every row reproduced, none taken on trust
Check Command Result
Sealed witnesses + controls pnpm --filter @pair/knowledge-hub exec vitest run src/tools/skills-conformance-check.test.ts 124/124 pass, R1–R34 included
Full package suite pnpm --filter @pair/knowledge-hub exec vitest run 4933/4933 pass, 48/48 files
CLI oracle (AC 3) pnpm skills:conformance exit 0 — PASS — 50 skills conformant (… entrypoint depth, skill-local scripts, catalog counts …)
Type gate pnpm --filter @pair/knowledge-hub run ts:check exit 0
Lint gate pnpm --filter @pair/knowledge-hub run lint exit 0
Format gate pnpm --filter @pair/knowledge-hub run prettier:check exit 0
Link gate pnpm --filter @pair/knowledge-hub run check:links exit 0
Twin identity, independent of the checker shasum pairwise over all 10 dataset scripts/ files vs their installed twins 0 missing, 0 drifted

The corpus is 6 workflow skills carrying 10 skill-local scripts, all .mjs, all flat, all byte-identical to their installed twins. Verified with my own shell loop, not by calling the code under review.

Assessments

Security — Input validation

Verdict: green — the only inputs are repo-controlled dataset paths; no untrusted data reaches this gate.

Security — Output handling

Verdict: green — output is diagnostic strings to stdout; no encoding surface.

Security — Authentication / Authorization

Verdict: not applicable — a local dev-time filesystem gate, no auth surface.

Security — Introduced vulnerabilities

Verdict: green — 0 introduced, 0 pre-existing.

Details

Path traversal was considered and is handled deliberately: isWithin decides containment on the resolved path (relative()), never by scanning the spelling for .., so scripts/lib/../helper.mjs stays legal and scripts/../helper.mjs is refused. statSync/lstatSync/realpathSync are each wrapped in try — a dangling entry can no longer take the whole run down. No shell, no network, no writes.

Cost

Verdict: cost:green — no new dependency, no runtime service, one extra filesystem walk over ~10 files in a dev gate.

Architecture (Coupling)

Verdict: not assessed — outside the risk:green review set (general pass only).

One coupling observation, recorded rather than assessed

INSTALLED_PREFIX = 'pair' and ENTRY_DEPTH duplicate SKILL_COPY_OPTS from skill-md-mirror.ts rather than importing it. The doc comment justifies this well — the module must stay runnable as a SINGLE file via ts-node before any build, and row R18 spawns a lone copy of it — and pins the derivation behaviourally through the literal installed paths asserted in R12/R21/R22. This is a sound trade, not a finding. It is nonetheless the same duplication that finding r5-11 below turns into a defect at a different level: the prefix is duplicated correctly, while the copy semantics are re-implemented incorrectly.

Bug fix — Red test before fix

Verdict: not applicable — feature PR. (The two in-cycle fixes above the seals, 9735d2da and 7ab4240b, each landed above a sealed RED snapshot — a0-rev2/a0-rev3 — that carries the failing rows, so the test-first order is enforced mechanically by custody rather than by convention.)

Details

Findings by severity

Critical (must fix before merge)

None.

Major (must fix before merge)

  • packages/knowledge-hub/src/tools/skills-conformance-check.ts:975-1010 (collectLocalScriptEntries) and :1049-1080 (compareScriptsTree)r5-11 — the twin half decides "what ships" by walking the dataset itself, but the thing it must agree with is the installer. Where the two disagree the gate demands an installed file pair update never produces, so a correctly installed tree is red and the remediation the message prescribes ("re-run pair update") is the very operation that created the difference. Three divergences, all reproduced on this head — and the third needs no symlink at all, so it is reachable by an ordinary edit:

    1. A .md under scripts/ is byte-rewritten on install. applySkillReferenceRewrites (packages/content-ops/src/ops/copy/copy-directory-transforms.ts:449-455) collects every .md in the copied tree — scripts/ included — and runs rewriteSkillReferencesInFiles over it. Ran the real pipeline (copyDirectoryWithTransforms with skillCopySyncOptions()): dataset scripts/README.md containing Run the `/red-verify` skill first. installs as Run the `/pair-workflow-red-verify` skill first., and the gate then emits
      workflow/alpha/scripts/README.md: skill-local script has DRIFTED from its installed twin pair-workflow-alpha/scripts/README.md (35 vs 49 bytes, first difference at byte 10). The dataset copy is canonical: re-run 'pair update'.
      Adding a README.md beside a skill's scripts is a plausible edit, not a contrived one — this is the realistic trigger.
    2. A directory symlink under scripts/ is skipped on install. entryIsCopyable (packages/content-ops/src/file-system/file-operations.ts:174) returns false for one — "a symlink to a directory is not followed" — while collectLocalScriptEntries follows it and emits every name it ships under, so the gate demands a twin at each. Sealed rows R33/R34 currently require that DRIFTED answer, on the premise that the installer creates the alias twin; the premise is false, and the fixtures hand-build the twin instead of deriving it, which is why no approved row caught it.
    3. A symlink escaping the root is skipped on install. resolvesWithin refuses it; the gate walks it and compares (scripts/ext -> <outside> returned a DRIFTED error in my probe).

    Blast radius today is zero — the corpus carries 10 flat .mjs files, no .md under any scripts/, and no symlinks — so the gate is green and no shipped path is unguarded. The defect is latent, and it is the item the maintainer's r5 status comment left open for exactly this cycle.

    VERIFY: run the REAL pipeline (copyDirectoryWithTransforms + skillCopySyncOptions()) over a fixture and compare the gate's expectation against the tree the installer actually produced, for all three shapes: a .md under scripts/ containing a /skill reference, a directory symlink, and a symlink escaping the root. Hand-built twins are what hid this. ORACLE: entryIsCopyable + applySkillReferenceRewrites — the installer is the authority on what ships and in what bytes, never the dataset walk. ASSERT: on a freshly and correctly installed tree checkSkillLocalScripts returns [] for each of the three shapes; where a layout genuinely cannot be installed (a directory symlink), the message names it as an unsupported layout rather than prescribing re-run pair update; and each rule is asserted against a twin derived from the pipeline, not written by the fixture.

    Secondary, same root cause: with the walk bounded by an ancestor chain instead of a global visited set, sibling aliases fan out combinatorially (a 6-level tree with two aliases per level yields 127 entries). Refusing directory symlinks removes this too.

Minor (must fix before merge — same bar as Major, just lower impact)

  • packages/knowledge-hub/src/tools/skills-conformance-check.ts:1233r0-2 — one missing linked script is counted as two violations. Reproduced: a fixture whose SKILL.md links ./scripts/absent.mjs with no such file yields both workflow/alpha/SKILL.md: broken relative reference "./scripts/absent.mjs" (from checkLinks) and workflow/alpha/SKILL.md: links "./scripts/absent.mjs" but no such skill-local script exists — expected workflow/alpha/scripts/absent.mjs. … (from checkLinkedLocalScripts). The CLI's FAIL — N violations therefore double-counts, and a maintainer fixing one file sees the count drop by two. VERIFY: runChecks over that fixture. ORACLE: the set of distinct problems in the corpus — one absent file is one problem. ASSERT: exactly one error mentions absent.mjs, and control R16 still holds (checkLinks keeps owning both link spellings for targets outside scripts/, and keeps reporting a target under scripts/ when checkSkillLocalScripts is not the one reporting it).

Questions (informational, never blocking)

  • Story Conformance: skill-local scripts ship with their skill and mirror byte-identically #482, Business Rules / .pair/knowledge/r3-10 — the convention this gate now enforces is documented nowhere in the knowledge base: grep -rli 'skill-local script' .pair/knowledge .pair/adoption returns no files. An author learns the rule by tripping the gate. Worth a line in the skills-authoring guideline; outside this PR's fixScope.
  • packages/knowledge-hub/src/tools/skills-conformance-check.ts:989-1010r0-4 — the walk-strategy decision (ancestor chain vs. global visited set, and the alternatives weighed) has no ADL, which AGENTS.md requires for a project decision. Deliberately and correctly reverted in 4b1192f0: a0-rev3 widened allowedPaths to .pair/adoption/decision-log/ but kept mode: behavioral, under which any added file breaches behavioral-adds-or-moves-module — the contract permitted the path and forbade the only way to use it — and a new entry would also force .pair/llms.txt to be regenerated, a path not in scope at all. The rationale currently lives in the doc comment. Recording it belongs to a follow-up after merge.
  • PR [US-482] feat: skill-local scripts ship with their skill and mirror byte-identically #483 (repository CI configuration)r2-8 — no CI runs on this stacked PR and the required pair-review check is absent: gh api repos/foomakers/pair/commits/4b1192f0.../check-runs --jq .total_count0. Every mechanical assurance in this report rests on my local re-run in a clean detached worktree, not on the code host. Owner is US-479, not this story.
Prior findings — transitions verified against the producer, not against the claim
id severity transition how it was verified on this head
r3-9 Major resolved The alias-order cross-product is closed: R30/R31/R33/R34 all pass, and the 2x2 is genuinely discriminating (a fix that merely orders real dirs before symlinks would silence R34).
r0-1 Major resolved Symlinked scripts are followed and compared, dangling ones named (R23R25, R28); my own probe of an EISDIR twin returned the UNREADABLE error, never a silent pass.
r2-6 Major resolved scripts/../helper.mjs is refused on the resolved path while scripts/lib/../helper.mjs stays legal (R26, R27, R29).
r0-3, r1-5 resolved Confirmed by the sealed suite passing in full.
r2-7 Minor resolved The cold-cache flake in packages/dev-tools/.../run-format.test.ts did not reproduce: 4933/4933 green on a cold worktree with a fresh pnpm install.
r5-11 Major open Still present, and broadened — see severityEvidence in the finding above: the prior evidence required a directory symlink, whereas a plain .md under scripts/ triggers it with no symlink at all. Severity stays Major (dev-time gate, zero blast radius on the current corpus).
r0-2, r0-4, r2-8, r3-10 open Re-verified individually; each reproduces exactly as described above.

Definition of Done

Criterion State
AC 1 — linked script exists, error names skill + script, exit non-zero met (R1R4, R19, R20, R26, R27, R29; exit branch by R18)
AC 2 — installed twin missing/differing is an error naming both paths partially met — correct for regular files (R5R8, R11, R21, R22); r5-11 is the gap
AC 3 — PASS on the real corpus, summary names the check met — pnpm skills:conformance exit 0, summary reads … entrypoint depth, skill-local scripts, catalog counts …
pnpm --filter @pair/knowledge-hub test green met — 4933/4933
No new dependency; format, lint, hygiene green met
Regression: every existing conformance test still passes met — 48/48 files
Merged via PR into feature/US-479-delivery-workflow-to-be pending (human merge gate; this stage never merges)

What happens next

r5-11 and r0-2 are blocking and both are fixable inside this repository, so this cycle routes them to a remediation group under pr=483 rather than escalating: preparation writes discriminating witnesses that derive the expected installed tree from the real pipeline, an independent validator seals them, the fix lands above that seal, and this stage re-verifies the same head. Round 0 of maxFixRounds: 3 — budget is not exhausted. Nothing merges automatically.

rucka pushed a commit that referenced this pull request Sep 9, 2026
… bare one

`collectSkillFiles` returned early on a dir's own SKILL.md marker, so a dir
holding BOTH its marker and a SKILL.md-bearing sub-dir yielded only the bare
skill — moving the silent drop instead of closing it. The copy pipeline
(`datasetSkillDirs`) collects every */SKILL.md, so the dropped nested
entrypoint still installs invocable as `pair-<dir>-<sub>`: wholly unchecked
(portability, size, links, approval signal) and `skillCount` short by one,
which then fails the catalog/KB prose counts somewhere unrelated.
`checkEntrypointDepth` cannot be the backstop — `<dir>/<sub>/SKILL.md` sits at
exactly ENTRY_DEPTH.

The two markers are now independent: collect the bare SKILL.md, then keep
walking the sub-dirs. Set parity with the copy pipeline is pinned on the real
corpus (55 = 55, difference empty both ways) and per domain row R1-R6.

Closes the r1-g1 RED contract for PR #483.
@rucka
rucka force-pushed the feature/US-482-skill-local-scripts-conformance branch from e8194d0 to b3efa04 Compare September 9, 2026 10:44
rucka pushed a commit that referenced this pull request Sep 9, 2026
…coordinator, handed on verbatim

Canary run 3 (#482, PR #483, 26 agents): three of four groups had their first `red` rejected by
hasRedContractReady for an absolute contractPath; the relative retry then did not resolve from the
story worktree (the contract lives in the main checkout's run directory) and cost one failed seal;
group g4 (test mode) ended the card as failed-fix on the same rejection. The coordinator now
accepts an absolute path under /.pair/working/runs/ (no `..`, no shell syntax) and passes it to
red-verify and red-seal unchanged; red-spec returns it absolute by contract. ADR-024 §11.

Run 3 also proved seal → GREEN → P3 live for the first time (three groups green, no breach).

Refs: #479

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rucka pushed a commit that referenced this pull request Sep 9, 2026
…ndoned attempt instead of refusing stale

Canary run 4 (#482, PR #483, 5 agents): the RED author found HEAD at base but the tree dirty at
one test file — the unsealed edit run 3's g4 author left when the coordinator rejected its result
— and refused `stale`, correctly under the old rule. New rule (ADR-024 §12): with HEAD at base, a
tree dirty ONLY at test artifacts, no $repair and no snapshot for the PR, the author records each
path + sha256 as `discarded` in its handoff, restores the tree, and proceeds; a moved head or a
dirty production path is still `stale`.

Refs: #479

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rucka pushed a commit that referenced this pull request Sep 9, 2026
…un keeps its runId and builds on its own prior attempts

Canary run 5 (#482, PR #483, 8 agents): GREEN had appended the cycle log under the MAIN checkout's
.pair/working/reviews/ while the probe looked for it in the worktree — no resume was ever a
continuation. cycle-comments/green-fix now resolve $reviewLog against the main checkout, like the
handoffs (ADR-024 §13).

Convergence across runs: a resume passes the SAME runId; red-spec treats the verifier's earlier
rejection for its phase (<phase>-red-verify.json in the run directory) as mandatory rows, and a
`fresh` review re-validates the previous review's findings before hunting for new ones, so two
attempts of one cycle build on each other instead of re-sampling. Repair budget unchanged.

Refs: #479

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rucka pushed a commit that referenced this pull request Sep 9, 2026
…e merge gate instead of grouping them

Canary run 7 (#482, PR #483, 5 agents, severityFloor Major): the review raised a Major on the
story CARD (business rule 3 described a layout the gate refuses); the planner had no place for a
finding with no repository path, put it in a `structural` group with `allowedPaths: []`, and the
plan was rejected twice → failed-plan. PLAN_SCHEMA gains `carried[]` ({ finding, disposition });
every index must be in exactly one group or carried; carried findings are accepted as
"Outside the repository — <disposition>" and a plan with no groups converges with them on the
record. remediation-plan skill updated; ADR-024 amendment §7. Story #482's rule 3 corrected on
the card as the review asked.

Incident noted: during run 7 an agent flipped the main checkout's `core.bare` to true (restored
by hand); no transcript names the command.

Refs: #479

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rucka pushed a commit that referenced this pull request Sep 9, 2026
…h instead of joining it onto the worktree

Canary run 8 (#482, PR #483, 8 agents): RED authored and verified, then the sealer refused twice
with ENOENT — join(cwd, <absolute contractPath>) produced <worktree>/Users/.../red-contract.json.
resolve(cwd, contractPath) keeps an absolute path and resolves a relative one against cwd; the
dirty-tree filter compares resolved paths. Regression test on a contract stored outside the
worktree; all four copies synced.

Refs: #479

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rucka pushed a commit that referenced this pull request Sep 9, 2026
… too

`checkInstallableLayout` implemented only the SHALLOW half of the installer's
bounded-flatten rule (`validateNoShallowEntryWithSubdir`). The DEEP half
(`validateNoDeepEntry`) had no owner in the gate: `checkEntrypointDepth` is
marker-BOUND (`basename(file) !== 'SKILL.md'`), so a marker-LESS directory below
the entry depth whose ancestor at that depth holds no files tripped the
installer and nothing in the gate — the whole corpus installed as NOTHING behind
a green gate.

Measured at the real boundary, injecting `workflow/shared/lib/util.mjs` (the
plausible trigger: `red-snapshot.mjs` is already duplicated across two skills)
into the live corpus:
- before: `skills:conformance` PASS (exit 0) while the producer REFUSES
- after:  FAIL — 1 violation, naming `workflow/shared/lib`, the same directory
  the producer's refusal blames

The rule is marker-BLIND on both sides, mirroring the producer: a dir deeper
than the entry depth is CONTENT iff its ancestor at EXACTLY that depth holds
files of its own. Additive — `checkEntrypointDepth` is untouched. Each rule is
its own helper so the lint ceilings (50 lines / complexity 10) still hold.

Refs: #483 review round 1, finding 0

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rucka added a commit that referenced this pull request Sep 9, 2026
…emental resume, upstream acceptance contract; way-of-working delta

- replaces §1/3/4/6/7 and amendment-a §3–7 with the rules in force, stage owners, retired dispatches (rejected at parse time), typed statuses, finding-id policy
- freezes the baseline identities (engine 2.0.0 @ 8b8b260, #482/#483, runs 1–8 cost) before optimizing
- Task: T-10 — Freeze the delta contract and replace the contradictory rules

Refs: #479
@rucka rucka closed this Sep 9, 2026
@rucka
rucka force-pushed the feature/US-482-skill-local-scripts-conformance branch from c2e5527 to ad81780 Compare September 9, 2026 18:51
T added 4 commits September 9, 2026 20:54
…Workflow harness refused the script (canary run 11 launch); the NUL delimiter is spelled as an escape, guarded by test

Refs: #479
… in the Workflow sandbox and aborted canary run 11 at launch; wall time is read from the harness summary, guarded by test

Refs: #479
Pair-RED-Snapshot: pr=0; phase=a0; base=421441fa0a8e2a8a4c02bbf4d00bcd99feb129eb; manifest=.pair/red-snapshots/pr-0-a0.json
…yte-identically

checkSkillLocalScripts(skillsDir, installedSkillsDir), wired into runChecks
against the real installed root and named in the CLI PASS summary.

- linked ./scripts/x and scripts/x resolve inside the skill's own scripts/;
  delegated to extractLinkTargets + isCheckableTarget + the #fragment strip, so
  a fenced example, a <placeholder> and an adr-NNN- pattern stay examples
- every dataset skill-local script has a byte-identical twin at the bounded
  flatten's path: <cat>/<name>/scripts/<sub> -> pair-<cat>-<name>/scripts/<sub>,
  recursive, sub-directories preserved (verified against the real
  installedArtifactPath); missing / drifted / unreadable reported distinctly
- directional like the SKILL.md mirror guard: orphans ignored, drift reported
  never repaired; absent installed root (dataset-only checkout) skips the twin half
- a meta skill owning scripts/ is refused: the flatten maps next/scripts/router.mjs
  to pair-next-scripts/router.mjs, a separate top-level dir, and the entry walk
  then loses the skill entirely
- own corpus walk, not collectSkillFiles, precisely so that layout stays visible
- no import of skill-md-mirror: the gate runs under ts-node with no build and its
  exit branch is exercised by spawning a copy of this file alone

Sealed contract green: 112/112 (18 previously-red rows R1-R13, R15, R19-R22;
controls R14, R16, R18 still pass).

Refs: #482
@rucka rucka reopened this Sep 9, 2026
T and others added 3 commits September 9, 2026 22:37
… snapshot

The seal's record lives in Git history at a3772ab; left in the working
tree it is the ONE path failing prettier (format:check) and it blocks the
pre-push gate. Custody verify expects it gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pair-RED-Snapshot: pr=0; phase=a0-rev2; base=a5f4da500a1ff5f511ea335a3acf97e5cf0cab12; manifest=.pair/red-snapshots/pr-0-a0-rev2.json
… leave the skill's own scripts/

r0-1: readdirSync-withFileTypes reports lstat semantics and existsSync follows, so a
symlinked script, a symlinked sub-directory, a dangling entry and a dangling `scripts`
link were all dropped in silence. Entry type is now resolved by a following statSync
inside a try (a dangling entry is `unresolvable` and gets named, never an ENOENT that
takes the run down); the walk is cycle-guarded by resolved path.

r2-6: a scripts/-prefixed link is refused when its RESOLVED path leaves the skill's own
scripts/ directory — the boundary is scripts/, not the skill folder, so <skill>/helper.mjs
is refused too, while scripts/lib/../helper.mjs stays legal.

Also drops the transient seal manifest, whose record lives in Git history.
@rucka rucka changed the title [US-482] feat: conformance — skill-local scripts ship with their skill and mirror byte-identically [US-482] feat: skill-local scripts ship with their skill and mirror byte-identically Sep 10, 2026
@rucka

rucka commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

Human decision required — round 5 of 5 (budget exhausted)

Reviewed head 4b1192f0463ae85b4ea1dd6bc97ae44ef0ec7667 · Verdict CHANGES-REQUESTED · Tier risk:green (pass: general)

Custody is clean (verify-chain --base 421441faverified: true, three sealed snapshots a0, a0-rev2, a0-rev3, no breach). The full adopted gate is green on this head (pnpm quality-gate, exit 0, 22/22 turbo tasks, tree clean) and the whole sealed suite passes — 124/124, including this round's R30–R34.

The round-5 finding closed, and a new one opened

r3-9 → resolved. Verified against the producer, not the claim: the exact r4 repro (real scripts/zzz-lib/util.mjs, alias scripts/aaa-alias -> zzz-lib sorting first, drift on the real twin) now returns exactly one error naming workflow/alpha/scripts/zzz-lib/util.mjs, where it returned [] at 9735d2da. The ancestor-chain walk is the right shape and terminates on true cycles.

r5-11 (Major, blocking, contract-gap on group a0) — the gate now demands an installed twin that pair update provably never creates.

collectLocalScriptEntries follows a symlinked sub-directory under scripts/ and emits every name the directory ships under. The installer does the opposite: entryIsCopyable in packages/content-ops/src/file-system/file-operations.ts:174 skips directory symlinks ("a symlink to a directory is not followed"), and the skill-install path shares that predicate (collectFilescollectInstallableFilescopyDirectoryWithTransforms).

Proved end to end by running the real install pipeline (copyDirectoryWithTransforms with skillCopySyncOptions(), the same one skill-md-mirror uses) over a fixture, then the gate on its output:

Q1 installed scripts/ contains: ["flat.mjs","lib"]          # alias skipped, WARN logged
Q1 gate errors on a FRESH install:
  workflow/alpha/scripts/alias/util.mjs: skill-local script is MISSING from the
  installed mirror — expected a byte-identical twin at
  pair-workflow-alpha/scripts/alias/util.mjs. The dataset copy is canonical:
  re-run `pair update`.

A freshly and correctly installed tree is red, and the remediation the message prescribes cannot clear it — re-running pair update skips the alias again, forever. Controls: a symlinked file installs and the gate is silent (Q2 → []), and a symlink-free tree is clean (Q3 → []), so the contradiction is specific to directory symlinks.

Why this needs a contract revision rather than a fix. R24 licensed either answer — compared or refused as an unsupported layout — and refusing is the answer consistent with the installer. But R33/R34, sealed this round, now require the alias path to produce a DRIFTED error, i.e. they mandate the wrong answer, on the rationale that "pair update installs a twin at pair-workflow-alpha/scripts/zzz9-alias/util.mjs". That premise is false; the fixtures hand-build the twin instead of deriving it from the installer, which is why no approved row caught it. The rows cannot be satisfied and corrected at the same time.

Secondary, same root cause: with the walk-global set replaced by an ancestor chain, aliases fan out combinatorially — a 6-level tree with two aliases per level yields 127 entries (measured, 70 ms). Refusing directory symlinks removes this too.

Blast radius today: none. The corpus carries zero symlinks (find packages/knowledge-hub/dataset/.skills -type l → 0, same for .claude/skills), so the gate is green and no shipped path is unguarded. The defect is latent, and it predates this round — a symlinked sub-directory already demanded an uncreatable twin at 9735d2da (r4's own r0-1 evidence records exactly that error), so it is marked missedUpstream.

The decision

The fix budget is spent (round 5 of maxFixRounds: 5). One of:

  1. Grant a revision a0-rev4 that rewrites R30/R31/R33/R34 around what the installer actually does — derive the expected twin by running the real pipeline rather than hand-building it — and makes collectLocalScriptEntries refuse a directory symlink under scripts/ as an unsupported layout, with a message that says so instead of re-run pair update.
  2. Accept and merge as is, recording r5-11 as known latent behaviour, on the ground that the corpus has no symlinks and the gate is green.

Carried to the merge gate (not actionable inside this contract)

id severity why it cannot be closed here
r0-2 Minor A missing linked script is reported twice; deduplicating breaks sealed control R16. Needs a contract owning both producers.
r0-4 Questions The walk-strategy decision has no ADL. a0-rev3 widened allowedPaths to .pair/adoption/decision-log/ but kept mode: behavioral, under which any added file breaches behavioral-adds-or-moves-module — so the contract permits the path and forbids the only way to use it. Correctly reverted in 4b1192f0 and reported as CG-3 rather than patched around; .pair/llms.txt would also need to be in scope.
r2-7 Minor Cold-cache flake in packages/dev-tools/.../run-format.test.ts:155, outside fixScope. Did not reproduce this pass.
r2-8 Questions No CI runs on a stacked PR and the required pair-review check is absent (check-runstotal_count: 0); the mechanical verification rests entirely on this local gate re-run. Owner is US-479.
r3-10 Questions The convention the gate enforces is documented nowhere in the KB (grep -rli 'skill-local script' .pair/knowledge .pair/adoption → no files). Outside fixScope.

Resolved and verified this cycle: r0-1, r2-6, r0-3, r1-5, r3-9.

@rucka

rucka commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

Human decision on the r3 escalation (maintainer, 2026-09-10): option 1 — extend the fix-round budget by one round for a0-rev3.

  • r3-9 stays blocking and is closed by a revision of group a0 (alias-directory row, both readdir orders) plus a fix of the cycle guard inside the sealed scope — never by lowering the floor or carrying the regression.
  • The budget is raised from 3 to 4 rounds for this cycle only (pipeline.maxFixRounds: 4, same runId canary-479-v3); reviewer independence, RED repair budget and gates unchanged.
  • Questions r0-3, r0-4, r3-10, r2-8 and carried r0-2, r2-7 remain for the merge gate. No automatic merge.

Technical note (author, 2026-09-10): engine 3.0.9 removed the fix-round budget from the effective-inputs digest (a budget change must not invalidate review evidence). The formula change itself invalidates the pre-3.0.9 review digests once, so the resume performs one migration re-review (r4, same head, no fix) before a0-rev3. To keep exactly ONE extra fix round as decided, the run is launched with pipeline.maxFixRounds: 5: round 4 = migration re-review, round 5 = the granted a0-rev3 round. Floor, RED budget, reviewer independence and gates unchanged.

T added 3 commits September 10, 2026 10:56
Pair-RED-Snapshot: pr=0; phase=a0-rev3; base=9735d2da1d354f60171afbe2c80fe703d8a7f9d3; manifest=.pair/red-snapshots/pr-0-a0-rev3.json
…, in either readdir order

The scripts/ walk kept one visited set for the whole descent, keyed on realpathSync.
A directory reachable under two names — a real scripts/lib and a sibling symlink
aliasing it — really ships under both, so pair update installs a twin at each
mirrored path. The shared key dropped whichever name readdirSync yielded second, so
a drifted twin of a genuinely shipped path was answered with silence and the corpus
gave two answers depending on directory order.

Bound the descent by its own ancestor chain instead: a sibling ALIAS is walked once
per relative path, a true CYCLE (scripts/lib/loop -> scripts) is still refused and
the walk still terminates.

The transient seal manifest leaves the tree above the snapshot.

Refs: #482
… recorded inside this contract

The a0-rev3 fixScope widened allowedPaths to include .pair/adoption/decision-log/
"because the implement process records its decisions there", but kept mode
"behavioral", under which the custody check flags any added file (status != M) as
behavioral-adds-or-moves-module. A decision-log entry is always a NEW file, so the
contract permits the path and forbids the only way to use it. Adding one also
forces .pair/llms.txt to be regenerated — the committed index is pinned
byte-for-byte by llms-index-conformance — and that path is not in allowedPaths at
all.

Both are reported as contractGaps rather than patched around. The walk-strategy
rationale and its alternatives stay where they are in scope: the doc comment on
collectLocalScriptEntries.

Refs: #482
@rucka

rucka commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

Human decision on the r5 escalation (maintainer, 2026-09-10): grant a0-rev4.

  • r5-11 stays blocking and is closed by a revision of group a0: rows R30/R31/R33/R34 rewritten around what the installer actually does (derive the expected installed tree by running the real pipeline, never a hand-built twin); collectLocalScriptEntries refuses a directory symlink under scripts/ as an unsupported layout, naming it, instead of demanding a twin pair update never creates.
  • Budget raised from 5 to 6 rounds for this cycle only (pipeline.maxFixRounds: 6, same runId canary-479-v3, engine 3.0.9 e5211994). Floor, RED repair budget, reviewer independence and gates unchanged. No automatic merge.

@rucka

rucka commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

Cycle status after a0-rev4failed-contract (RED repair budget spent)

The granted revision a0-rev4 (39 → 41 rows, directory symlinks refused as an unsupported layout, expected installed tree derived by running the real pipeline) was rejected twice by the independent validator, each time on a real, reproduced gap in the same class (r5-11, installer byte-rewriting of .md under scripts/): first entryIsCopyable as the sole oracle, then only one of the two rewriters covered (RV-12 on R40). The RED repair budget is 1 by policy and was not raised, so the cycle is terminal failed-contract; nothing was sealed or implemented, head stays 4b1192f0.

What holds on this head: AC-1 and AC-3 proven and reviewed clean across r0–r5; r0-1, r2-6, r3-9, r0-3, r1-5 resolved; full gate green; custody chain a0 → a0-rev2 → a0-rev3 verified. Open: r5-11 (blocking, latent — the corpus carries zero symlinks), carried r0-2, r2-7, questions r0-4, r3-10, r2-8.

Human decision: (1) accept r5-11 as known latent behaviour and treat this PR as ready for the human merge gate as is; or (2) a NEW cycle on this PR (entry: pr) that lets the first review route r5-11 through a remediation group (red-spec remediation → seal under pr=483 → green-fix → re-review → synthesis) — the path the maintainer already asked to exercise live for AC-10 of #479. No automatic merge either way.

@rucka

rucka commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

Cycle v4 status — failed-preparation (split-required) on r1-g1

The remediation cycle requested by the maintainer ran on engines 3.0.10 → 3.0.13 (six launches, four engine defects found and fixed in between). The first review upserted in place, ids stable across cycles, and the two blocking findings were grouped correctly: r1-g1 = r5-11, r1-g2 = r0-2.

Preparation then refused split-required on r1-g1, with the reason stated in full: r5-11's directory-symlink divergence cannot be given a witness at 4b1192f0 because satisfying it necessarily breaks two SEALED, currently-passing rows of the a0-rev3 acceptance contract — R33 and R34 — whose blob a fixer may not touch. No contract was written, no file modified, the worktree is clean.

The reviewer also widened r5-11's evidence this round: the same unclearable DRIFT reproduces with no symlink at all — a plain README.md beside a skill's scripts/ is byte-rewritten on install (applySkillReferenceRewrites), so the gate demands a twin pair update never writes. Blast radius today is still zero (10 flat .mjs files, no .md under any scripts/, no symlinks).

The cycle cannot converge while R33/R34 stand. The options are structural, not another round:

  1. Retire the false rows: a revision of a0 that deletes/rewrites R33/R34 (they encode the wrong answer) and re-derives the expected installed tree by running the real pipeline. The previous cycle spent its RED repair budget attempting exactly this (a0-rev4, rejected twice on real gaps) — it needs a fresh cycle, not a retry.
  2. Carry r5-11 and merge: record it as known latent behaviour, close the canary, and open a follow-up card owning the installer-vs-walk authority question.

Open on this PR: r5-11 (Major, blocking), r0-2 (Minor, blocking), questions r0-4, r3-10, r2-8. Resolved and verified across the cycles: r0-1, r2-6, r3-9, r0-3, r1-5, r2-7. No automatic merge; the PR stays at 4b1192f0.

Durable evidence: .pair/working/runs/canary-479-v4/482/ (handoffs, maintainer-log.md, per-run journals under logs/) and the evidence gist linked from #480.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-state:to-be-reviewed PR state: awaiting review / gate risk:green Classification: low risk tier user story Work item representing a user story

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant