Skip to content

[US-321] feat: wire setup-pm to Azure DevOps end to end - #481

Open
rucka wants to merge 1 commit into
mainfrom
feature/US-321-azure-devops-setup-pm
Open

[US-321] feat: wire setup-pm to Azure DevOps end to end#481
rucka wants to merge 1 commit into
mainfrom
feature/US-321-azure-devops-setup-pm

Conversation

@rucka

@rucka rucka commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

PR Information

PR Title: [US-321] feat: wire setup-pm to Azure DevOps end to end
Story/Epic: #321 — standalone follow-up of #235 / PR #317
Type: Feature
Priority: Medium
Labels: documentation, user story, risk:green

Summary

What Changed

/pair-capability-setup-pm now presents, accepts and fully configures Azure DevOps: Notes names all 4 shipped adapters, $tool resolves by-reference against the canonical token table, Step 3 routes through the adapter's Detection/HALT before any write, Step 4 writes team + area path and derives ## State Mapping from real work item states, and the Azure adapter snippet + website doc both declare Team + Area path (byte-identical). Guarded by a new data-driven conformance test (packages/knowledge-hub/src/conformance/setup-pm.test.ts) so a future 5th adapter cannot land unenrolled.

Why This Change

#235 shipped the Azure DevOps KB guidelines but never wired them into /pair-capability-setup-pm's config flow — the adapter existed but the skill couldn't drive it end to end.

Story Context

User Story: Wire setup-pm to Azure DevOps (KB guidelines shipped in #235, config flow not wired)
Acceptance Criteria: AC-1 through AC-8, all covered — RED-first (34 cases, 12 failing before content edits), then GREEN per AC, see task list below.

Changes Made

Implementation Details

  • RED conformance guard: 34 cases, 12 failing pre-edit (packages/knowledge-hub/src/conformance/setup-pm.test.ts)
  • AC-1/2/8: Notes names all 4 adapters; $tool points at canonical token table; edge cases add invalid-token fall-through + inherited-process caveat
  • AC-3: Step 3 Azure walkthrough routes through Detection/HALT before any config write
  • AC-4/6: Step 4 writes team + area path, derives ## State Mapping from real states, reports unresolvable fields instead of omitting
  • AC-5: Azure adapter ### Adoption Configuration + apps/website/content/docs/pm-tools/azure-devops.mdx § Configuration declare Team + Area path, byte-identical
  • Canonical token table completed with linear + filesystem rows
  • Mirrors re-synced via the real pair update transform
  • Injection sweep: 20 mutations, all reddened; 2 vacuous assertions found and fixed (AC-4/AC-6, now scoped per act-step)
  • Full quality gate green (tiering disabled ⇒ full suite)

Files Changed

  • Modified: .claude/skills/pair-capability-setup-pm/SKILL.md, .pair/knowledge/guidelines/collaboration/project-management-tool/azure-devops-implementation.md, .pair/knowledge/guidelines/technical-standards/ai-development/skill-conventions/way-of-working-pm-resolution.md, apps/website/content/docs/pm-tools/azure-devops.mdx, mirrored dataset copies under packages/knowledge-hub/dataset/
  • Added: packages/knowledge-hub/src/conformance/setup-pm.test.ts

Testing

Test Results

turbo ts:check test lint:  PASS — 22/22 tasks
pnpm workflows:test:       PASS — 320 pass / 0 fail
pnpm format:check:         PASS
pnpm gate:composition:     PASS
pnpm hygiene:check:        PASS
pnpm smoke-modes:check:    PASS
pnpm docs:staleness:       PASS — 44 skills, 11 commands in sync
pnpm skills:conformance:   PASS — 44 skills conformant
pnpm dup:check:            PASS — new file not among 18 pre-existing clones
setup-pm.test.ts:          34/34 pass (12 failed before content edits)

Testing Strategy

  • Happy Path: Azure walkthrough end to end — detect/HALT, then team + area path config write, state mapping derived from real work item states.
  • Edge Cases: invalid token fall-through, inherited process caveat, unresolvable field reported not omitted.
  • Injection sweep: 20 mutations, every one reddened; caught 2 vacuous assertions, fixed.

Reviewer Notes (deviations from story wording)

  • AC-7 guard placement. AC-7 says "when pnpm skills:conformance runs"; the guard actually lives in the vitest conformance corpus (packages/knowledge-hub/src/conformance/, run by pnpm test), matching where the existing ADL (2026-07-18-conformance-test-per-file-not-per-story.md) and the sibling pm-tool-adapter-contract.test.ts put artifact guards. Both pnpm test and pnpm skills:conformance run inside pnpm quality-gate, so the enforcement AC-7 asks for holds.
  • Guard is data-driven, no name table. Tool display name is read from each adapter's pinned H1 (# <Name> - Complete Implementation Guide); no adapter count is asserted — a 5th adapter reddens the guard by design, not by an updated count.
  • pair update side effects reverted. The transform also rewrote skill references in four unrelated .pair/adoption/** files (pre-existing drift, e.g. /loop/pair-loop); reverted to keep this diff scoped. Real, separate drift — worth its own issue.
  • No ADR/ADL recorded. Every choice here was already decided by the refined story (AC-2's by-reference token table, AC-7's data-driven guard) or an existing ADL. No new dependency, package, script, gate step or abstraction introduced.

Dependencies & Related Work

Related PRs

…adapter-enrollment conformance test

- setup-pm `## Notes` no longer contradicts its own Step 2 table: the supported-tools
  line names every adapter shipped (GitHub Projects, Filesystem, Azure DevOps, Linear)
- `$tool` accepts the canonical tool tokens BY REFERENCE — the alias table in
  way-of-working-pm-resolution.md gains `linear` + `filesystem` rows and is named the
  single source, so no token list is duplicated into the skill
- Step 3's Azure walkthrough routes through the adapter's Detection and HALT Behavior
  (`az` -> extension -> session) BEFORE any configuration write; the skill still never
  installs or authenticates on its own
- Step 4 writes what the adapter reads back (team + area path, the `--area` every create
  passes), a `## State Mapping` built from real work item states — never a board column —
  and REPORTS an unresolvable field instead of silently omitting it
- Azure adapter `### Adoption Configuration` and its website twin both declare Team +
  Area path
- New guard `packages/knowledge-hub/src/conformance/setup-pm.test.ts`: data-driven over
  the adapters discovered on disk in BOTH corpora, display name read from each adapter's
  pinned H1, no adapter count asserted — a 5th adapter reddens the gate until enrolled.
  Every assertion injection-tested; the sweep caught two vacuous ones (AC-4/AC-6 matched
  prose elsewhere in Step 4) and they are now scoped per act-step.
- Mirrors re-synced with the real `pair update` transform in this commit

Refs: #321
@rucka rucka added documentation Improvements or additions to documentation user story Work item representing a user story risk:green Classification: low risk tier labels Sep 9, 2026
@rucka rucka self-assigned this Sep 9, 2026
@rucka rucka added the pr-state:to-be-reviewed PR state: awaiting review / gate label Sep 9, 2026
@rucka

rucka commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator Author

Verdict

risk:green · cost:n/aCHANGES-REQUESTED — the AC-1 Notes guard passes with the pre-#321 defect restored, $tool now resolves to a token superset with no adapter, and Step 4's renumbering left a dangling "step 3" cross-reference.

Open findings: 3. (3 Major; 1 Question is informational and never blocking.)

PR: [#481] · Reviewer: independent review agent · Date: 2026-09-09 · Story: [US-321] · Type: feature

Classification matrix — per dimension
Dimension Tier Source Note
Service/domain criticality green story classification (risk:green) Unchanged — KB/docs + one conformance guard, no runtime code path.
Change/diff risk green diff footprint 8 files, +290/−16; one SKILL.md (+mirror), one adapter section, one website line, one new test.
Business impact green subdomain class Setup-flow documentation; no deployable behavior.
Security relevance not assessed green tier ⇒ security pass not in the declared review set
Coupling balance not assessed green tier ⇒ coupling pass not in the declared review set

Tier = max(assessed). Review value is a floor (D17): confirmed at risk:green, not lowered.

Declared review set (risk:green): general pass — /pair-process-review phases 1–4 (validation, technical review, adoption compliance, completeness). Security, boundary, architecture and adversarial passes are not in the green tier's set and are reported as not assessed below rather than dropped.

Assessments

Security — Input validation / Output handling / Authentication / Authorization / Introduced vulnerabilities

Verdict: not assessed — risk:green tier does not compose /pair-capability-assess-security. No credential, endpoint or executable code path is touched by the diff.

Cost

Verdict: cost:n/a/pair-capability-assess-cost not composed at this tier; no infrastructure, dependency or paid-service surface in the diff.

Architecture (Coupling)

Verdict: not assessed — /pair-capability-assess-coupling is not in the green tier's set.

Bug fix — Red test before fix

Verdict: not applicable — not a bug fix. The PR is a feature/docs wiring story; the RED-first conformance guard is nonetheless present and was verified to redden (see Positive feedback).

Details

Findings by severity

Critical (must fix before merge)

None.

Major (must fix before merge)

  • packages/knowledge-hub/src/conformance/setup-pm.test.ts:129-140 — The AC-1 Notes assertion is positional-blind: it asserts the tool name appears anywhere on the supported-tools line, so a line that names the tool in its negative clause satisfies it. Concrete failure: restore the exact pre-Wire setup-pm to Azure DevOps (KB guidelines shipped in #235, config flow not wired) #321 defect —

    - Supported tools with implementation guides: **GitHub Projects**, **Filesystem**. Anything else (Jira, GitLab, Azure DevOps, Linear) takes the Step 2.4 HALT.

    — a line asserting Azure DevOps and Linear are not supported, and the guard is 34/34 green. Evidence: perl -i -pe replaced line 163 of both SKILL.md copies with the text above, then pnpm exec vitest run src/conformance/setup-pm.test.tsTests 34 passed (34). This is the drift the story exists to close (T-2: "the single hard defect"), and AC-7 promises "this exact drift cannot recur silently". It is not contrived: the PR's own Notes wording already ends Anything else (Jira, GitLab) takes the Step 2.4 HALT, so the guard's own documented next-adapter scenario is already defeated — I injected a real gitlab-implementation.md into both corpora and only 4 of the 6 GitLab cases reddened; both Notes cases passed while Step 2 and Step 3 correctly failed. Fix: assert against the affirmative list only — slice the claim at the first sentence-ending ./ that closes the supported enumeration (or match the bolded **<Tool>** run), and assert the tool appears there, not in the remainder of the line. VERIFY: Notes line "...: **GitHub Projects**, **Filesystem**. Anything else (Jira, GitLab, Azure DevOps, Linear) takes the Step 2.4 HALT." -> the Azure DevOps and Linear cases FAIL; ORACLE: pnpm exec vitest run src/conformance/setup-pm.test.ts in packages/knowledge-hub; ASSERT: expect(affirmativeList).toContain(tool.toLowerCase()) where affirmativeList excludes the "Anything else (...)" clause.

  • packages/knowledge-hub/dataset/.skills/capability/setup-pm/SKILL.md:16 (+ generated mirror .claude/skills/pair-capability-setup-pm/SKILL.md:16, and way-of-working-pm-resolution.md:53) — $tool's accepted values are now resolved by reference to the code-host alias table, but that table is a strict superset of setup-pm's adapters: it carries gitlab, gitlab-issues, github-enterprise and azure-repos. Concrete failure: /pair-capability-setup-pm $tool: gitlab → Step 2.1 "Is $tool provided and valid?" resolves gitlab against the canonical table ⇒ valid ⇒ Step 2.2 "If valid $tool, proceed to Step 3 with that tool" ⇒ Step 3.1 lists guides for four tools only and none is GitLab. Step 2.4's HALT is unreachable on this path because it is worded "If developer selects a tool without an implementation guide" — the interactive branch only. The new Edge Case does not catch it either: gitlab is inside the canonical table, so it never "falls through to the Step 2 interactive selection table". Net effect: the non-interactive path configures a tracker with no adapter, contradicting AC-8 and the Notes line's own "Anything else (Jira, GitLab) takes the Step 2.4 HALT". AC-2 named exactly github/github-projects, filesystem, azure-devops/azure-boards, linear. Fix: constrain the reference to the rows that ship an adapter — e.g. "the canonical tokens of the tools listed in the Step 2 table" — and make Step 2.4's HALT explicitly cover a $tool token with no implementation guide, not just an interactive selection. VERIFY: $tool: gitlab -> Step 2.4 HALT (no implementation guide), not Step 3; ORACLE: .pair/knowledge/guidelines/.../way-of-working-pm-resolution.md#code-host-resolution alias table rows vs. the four *-implementation.md files on disk; ASSERT: a conformance case that every token the $tool row admits has a matching adapter file, and that Step 2.4's HALT wording covers the $tool path.

  • packages/knowledge-hub/dataset/.skills/capability/setup-pm/SKILL.md:107 (+ generated mirror, same line) — Step 4's two inserted act-steps renumbered the section, but the ## Assignment step's internal cross-reference was not updated: it still reads "When step 3 just declared a separate code-host, also ask whether that host knows the same person by a different identifier and write code-host-assignee". After this diff, step 3 is ## State Mapping — which never declares a code-host — and the Git Workflow step that does is now step 5. Concrete failure: configuring Linear (or filesystem, or Jira) — a hosts-no-code tracker where step 5 does declare a separate code-host — an agent evaluating the condition literally finds step 3 declared no code-host, so the code-host-assignee question is never asked and the key is never written. The cascade then falls back to default-assignee on the code host, which is exactly the wrong-identifier case that key exists to prevent. This is a regression introduced by this PR: on origin/main the same sentence pointed at the correct step. Fix: change "step 3" to "step 5" (or make it name the step: "the ## Git Workflow step"). VERIFY: Step 4 numbering -> the Assignment step's back-reference resolves to the step that writes code-host; ORACLE: sed -n '97,108p' packages/knowledge-hub/dataset/.skills/capability/setup-pm/SKILL.md; ASSERT: a conformance case that the Assignment step's referenced step index is the one whose title contains "## Git Workflow".

Minor (must fix before merge — same bar as Major, just lower impact)

None.

Questions (informational, never blocking)

  • PR description § Reviewer Notes — The note says pair update also rewrote skill references in four unrelated .pair/adoption/** files and that this was reverted to keep the diff scoped. Confirmed the drift is genuinely pre-existing on main (grep -rln '/loop\b' .pair/adoption/2026-08-23-pair-loop-resumes-from-its-own-audit-file-not-a-checkpoint.md, product/subdomain/collaborative-workflow.context.md, tech/automation.md), so reverting was the right scoping call and this PR introduces nothing. Flagging only that the transform is therefore not idempotent against those files: the next pair update on any story re-introduces the same churn. No action requested on this PR.
Positive feedback
  • The guard is genuinely data-driven: adapters are discovered by disk glob in both corpora, display names read from each adapter's pinned H1, and no count is asserted — exactly what AC-7 asked for. Verified by injecting a stub gitlab-implementation.md into both corpora: the Step 2 and Step 3 cases reddened (4 failures) with no test edit.
  • sectionBetween fails closed (throws on a missing boundary), and the AC-4 assertions are scoped per act-step rather than over the whole of Step 4 — I deleted the "Every field the selected tool's adapter reads back" bullet and both AC-4/AC-6 cases reddened, so that scoping fix from the injection sweep is real, not claimed.
  • The Azure adapter snippet and its hand-maintained website twin are asserted byte-identical rather than eyeballed, and the --area rationale in the adapter matches what Step 4 tells the skill to write — the read-back contract actually closes.
  • The skill stayed tool-agnostic: state literals and token lists are linked, not copied, honoring the story's single-source-of-truth rule.
  • pnpm skills:conformance → PASS (44 skills); the new guard's 34 cases pass on the branch as claimed.
Functionality & requirements (AC coverage)
AC Status Note
AC-1 partial Notes line names all 4 and matches the mirror; the guard protecting it is vacuous against a reworded line (Major 1).
AC-2 partial Resolved by reference with no duplication as asked, but the referenced set over-accepts gitlab/azure-repos/github-enterprise (Major 2).
AC-3 met Step 3 routes through the adapter's Detection and HALT Behavior before any configuration write; guarded and injection-verified.
AC-4 met Team + area path in the field list, ## State Mapping from real work item states, board-column exclusion stated; guarded per act-step.
AC-5 met Adapter ### Adoption Configuration and azure-devops.mdx § Configuration declare Team + Area path, asserted byte-identical.
AC-6 met Unresolvable field is reported as a follow-up with the board-view consequence, never omitted.
AC-7 partial Guard reddens for an unenrolled 5th adapter via Step 2/Step 3, data-driven, no count — but its Notes case does not (Major 1).
AC-8 partial Step 2.4 HALT and the Notes line are correct for the interactive path; the $tool path can bypass it (Major 2).
  • Integrates with existing systems — mirror transform, ADL-conformant test placement, skills:conformance green.
  • Acceptance criteria met — 3 of 8 partial, see above.

rucka added a commit that referenced this pull request Sep 9, 2026
…typed refusals are answers, handoffs in the main checkout

Canary run 1 (story #321, PR #481, runId canary-479): fases 0/A/B/C and the planner worked; the
RED author refused group r1-g1 as `split-required` — a guard-strength finding (positional-blind
assertion) with production already correct has no RED against unfixed production — and the engine
re-dispatched the identical prompt once before ending the card as failed-red-contract (8 agents).

- Third group mode `test`: allowedPaths [], RED proven against an injected regression, no GREEN,
  P3 verifies the sealed head; red-snapshot.mjs treats any production change under a test scope
  as `test-mode-production-change` (script test added; both copies synced).
- A typed refusal (`stale`, `split-required`; planner `stale`) is routed by status and never
  retried with the same prompt; the result carries `redRefusal` and `splitReason`.
- Handoff directory rule: `.pair/working/runs/<run>/<story>/` in the MAIN checkout the coordinator
  was started in, never a story/review worktree (the review's detached worktree took its handoff
  with it) — stated in every skill's $run row and in the dispatch text.

ADR-024 amended (§8–10); batch-engine.mdx and the remediation-plan description updated.

Tests: 382 workflow (+2 coordinator, +1 script), 4905 knowledge-hub, 177 website.

Refs: #479

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation pr-state:to-be-reviewed PR state: awaiting review / gate risk:green Classification: low risk tier user story Work item representing a user story

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant