Risk, security, and validation profiles - #15
Conversation
Spec revised across eight Codex Gate-A passes (21/10/13/10/8/5/3/1 findings; last four Blocker-free). Story AC 9 amended in the same commit: the design splits the P2+P6 todos row rather than closing it whole, since P6's standalone-section half was deliberately rejected.
Plan revised across three Gate-A passes (12/7/0 findings, 5 Blockers). Spec §5 read-case 3 updated in the same commit: only the latest mode override after the latest axis change can explain a mode/axes mismatch, since an axis change voids prior overrides.
Two human-confirmed axes in the story header — risk (trivial|standard|high, via named domain and effect triggers) and security relevance (none|standard|high) — plus a validation mode derived as max(risk, security). The axes append review lens sets at the gates; the mode sets what evidence the author owes before Gate B. Different questions, not more passes: the 3-pass floor, the Blocker/Major filter and the file-first findings protocol are unchanged, and no new way to skip a gate is added. The Gate-B triviality skip narrows for a profiled story to effective level 0 and still requires the change itself to be trivial. Unprofiled stories behave exactly as before. Story: docs/superpowers/stories/2026-07-26-risk-security-validation-profiles-story.md Verification: full battery green on the tree this commit records — shellcheck (6 files), hook tests, check-invariants + suite (123 assertions), check-version-bump + suite (36 assertions), claude plugin validate --strict; exit 0. The story is unprofiled, so no mode-derived evidence entry is owed. Docs audit: getting-started.md and coding-workflow.md updated (profile, its two halves, and the narrowed skip); README.md unchanged — its intake row is a capability summary, not a field list, and the flow shape is unaffected; architecture.md unchanged — it describes layout, not story fields; MANIFEST.md unchanged — an inventory of the frozen source-files/ seed. Scope-guard waiver (granted during Gate B pass 11, reasoning in the plan's decision record): one string in hooks/codex-gate.sh. Its below-floor reminder asserted a skip rule the narrowing made false, at the moment an author decides whether to skip. The edit removes a rule statement and defers to the policy file, so the hook reads no profile and gains no machinery. Gates: Gate A spec clean at pass 8 (21/10/13/10/8/5/3/1), Gate A plan clean at pass 3 (12/7/0), Gate B clean at pass 16 on both branches.
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Warning Review limit reached
Next review available in: 53 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe workflow now adds human-confirmed risk and security profiles with derived validation modes. Intake records profiles and logs, Gate B applies profile-specific lenses and evidence rules, trivial skips are narrowed, and plugin documentation, versioning, backlog, and verification plans are updated. ChangesRisk, Security, and Validation Profiles
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Author
participant Intake
participant Story
participant GateB
participant Commit
Author->>Intake: confirm risk and security axes
Intake->>Story: record profile and derived validation mode
GateB->>Story: reread cited profile each pass
GateB->>Author: apply lenses and request mode-specific evidence
Author->>Commit: write validated evidence in closing amend message
Commit->>GateB: supply durable evidence for revalidation
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR introduces story-level risk, security, and validation profiles across intake and review policy.
Confidence Score: 3/5The PR should not merge until process-pr-review reliably discovers all associated story paths before deciding that a review fix is unprofiled and skip-eligible. The new skip restriction can be bypassed because its command does not connect the durable commit-body story citation to the cited-story set used for eligibility. Files Needing Attention: plugins/dev-workflow/commands/process-pr-review.md Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Intake derives risk and security] --> B[Human confirms profile]
B --> C[Story header is writable source]
C --> D[Gate reads story path]
D --> E[Axes select review lenses]
D --> F[Validation mode selects evidence]
D --> G{Trivial fix and every story level 0?}
G -->|Yes| H[Battery and documented skip]
G -->|No| I[Run Gate B]
E --> I
F --> I
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
plugins/dev-workflow/commands/process-pr-review.md:147-150
**Storyless fallback bypasses profiles**
When a profiled story path exists only in the final commit body, this command provides no procedure for discovering it before entering the “PR citing no story” branch, causing an ineligible review fix to be treated as unprofiled and skip Gate B.
Reviews (1): Last reviewed commit: "feat(intake,§5): risk, security, and val..." | Re-trigger Greptile |
| - **every cited story is eligible.** Resolve the cited story path(s) first: a profiled | ||
| story is eligible only at effective level 0 (risk `trivial` *and* security `none`); an | ||
| unprofiled story is eligible on the old judgement call; a PR citing **no** story — the | ||
| common shape for a review-fix PR — is §5's unprofiled case. With several cited |
There was a problem hiding this comment.
Storyless fallback bypasses profiles
When a profiled story path exists only in the final commit body, this command provides no procedure for discovering it before entering the “PR citing no story” branch, causing an ineligible review fix to be treated as unprofiled and skip Gate B.
Prompt To Fix With AI
This is a comment left during a code review.
Path: plugins/dev-workflow/commands/process-pr-review.md
Line: 147-150
Comment:
**Storyless fallback bypasses profiles**
When a profiled story path exists only in the final commit body, this command provides no procedure for discovering it before entering the “PR citing no story” branch, causing an ineligible review fix to be treated as unprofiled and skip Gate B.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
Accepted and fixed in 5201d9b — the sharpest finding on this PR.
You are right, and the gap was created by two correct decisions that never met: the workflow deliberately records the story path in the commit body, while this command only knew to look at the PR. A profiled story cited there would have fallen into the "no story cited" branch and taken the unprofiled judgement call.
The step now resolves citations from the PR body and the commit bodies in the range, and only when neither carries one is it §5's no-story case. A cited profile that is present but unresolvable now stops the run instead of degrading to unprofiled.
Validated directly rather than by a finding-triage subagent: this PR edits instruction-bearing paths (CLAUDE.md, plugins/, commands/, skills/, hooks/), and the command's step-0 precheck routes those to manual validation so a subagent is not rewriting the rules it reviews under.
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/coding-workflow.md`:
- Around line 126-130: Update the skip summaries in
docs/coding-workflow.md:126-130 and docs/getting-started.md:99-103 to explicitly
require a commit-body evidence entry for every cited profiled story, while
preserving the existing battery and skip-reason requirements.
In `@docs/getting-started.md`:
- Around line 14-16: Update the profile guidance in the getting-started
documentation to state that users may confirm or correct only the risk and
security axes; Validation must remain derived from those axes. Explain that any
intentional mismatch requires the human-confirmed, logged override flow,
preventing users from editing Validation directly.
In `@docs/superpowers/plans/2026-07-26-risk-security-validation-profiles.md`:
- Around line 594-603: Update the Step 3 snapshot commands to stage only the
explicitly known Task 1–2 plugin paths instead of using git add -A, then verify
the staged file list with git diff --cached --name-only before committing.
Preserve the existing snapshot commit and log inspection flow, and ensure the
instructions explicitly confirm no unrelated or sensitive files are staged.
- Around line 12-16: Align the documented hook scope across all four sites: in
docs/superpowers/plans/2026-07-26-risk-security-validation-profiles.md at lines
12-16, clarify that hook machinery remains unchanged while permitting the waived
reminder-only hook edit; update the non-goal at lines 414-416 accordingly. Apply
the same non-goal correction in
docs/superpowers/specs/2026-07-26-risk-security-validation-profiles-design.md at
lines 414-416, and amend the scope criterion in
docs/superpowers/stories/2026-07-26-risk-security-validation-profiles-story.md
at lines 59-60 to include the exact reminder-only exception.
- Around line 237-249: The rollout documentation is missing consistent coverage
for profile-driven skip behavior in process-pr-review. Update
docs/superpowers/plans/2026-07-26-risk-security-validation-profiles.md at lines
237-249 to include plugins/dev-workflow/commands/process-pr-review.md and its
verification steps; update
docs/superpowers/specs/2026-07-26-risk-security-validation-profiles-design.md at
lines 394-407 to list process-pr-review as an affected surface and define its
contract; retain the existing claim in plugins/dev-workflow/CHANGELOG.md at
lines 48-49 only after the plan and specification coverage is present.
In
`@docs/superpowers/stories/2026-07-26-risk-security-validation-profiles-story.md`:
- Around line 35-37: Update the risk acceptance criteria to include effect-based
high-risk triggers alongside the existing named domains: irreversibility, data
loss or corruption, and outage exposure. Preserve the requirement that high risk
must be reachable through explicit triggers rather than unaided judgement.
- Around line 45-46: Update the §5 gate prompt requirement to state that the
trivial skip is available only when both profile axes are Risk: trivial and
Security: none. Preserve evidence obligations for stories with any security
relevance, and require the skip reason to remain recorded.
In `@plugins/dev-workflow/commands/process-pr-review.md`:
- Around line 147-151: Update the cited-story resolution and eligibility
instructions so any malformed or semantically inconsistent profile that cannot
be resolved causes the command to stop and surface the validation cause. Keep
normal Gate-B processing only for successfully resolved stories, while
preserving the requirement that every cited story must independently be
eligible.
- Around line 141-146: Update the trivial-fix criterion in the process-pr review
instructions so one-line behavior, logic, security, or path changes are
explicitly excluded from being trivial. Preserve the existing Gate B requirement
for all substantial fixes, and limit trivial classification to non-behavioral
changes such as comments or naming.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b26caba0-527b-433a-b7d8-fc19bbd988ce
📒 Files selected for processing (13)
CLAUDE.mddocs/coding-workflow.mddocs/getting-started.mddocs/superpowers/plans/2026-07-26-risk-security-validation-profiles.mddocs/superpowers/specs/2026-07-26-risk-security-validation-profiles-design.mddocs/superpowers/stories/2026-07-26-risk-security-validation-profiles-story.mdplugins/dev-workflow/.claude-plugin/plugin.jsonplugins/dev-workflow/CHANGELOG.mdplugins/dev-workflow/commands/process-pr-review.mdplugins/dev-workflow/commands/workflow-init.mdplugins/dev-workflow/hooks/codex-gate.shplugins/dev-workflow/skills/intake/SKILL.mdtodos.md
… consistency Ten bot findings from PR #15 (CodeRabbit 9, Greptile 1), all validated directly rather than by triage subagent — the PR edits instruction-bearing paths, which the command's step-0 precheck routes to manual validation. Greptile's P1 was the substantive one: `process-pr-review` resolved "cited story paths" without saying where citations live, so a PR whose story path sits in a commit body — where this workflow puts it — fell into the "no story cited" branch and took the unprofiled judgement call. It now reads the PR body and the commit bodies in the range. Also from the bots: triviality judged by behavioural effect rather than line count; a stop for an unresolvable cited profile; the per-profiled-story evidence entry named in both skip summaries; getting-started no longer implying the derived mode is directly editable; the hook-scope waiver propagated into the spec non-goal and story AC 10; `process-pr-review` added to the surfaces lists; `git add -A` removed from the plan's snapshot step; and story AC 2 and AC 6 amended to the shipped rules, reasons inline. Gate B on these fixes ran 9 passes (4/1/5/3/2/2/4/1/0), clean on both branches at 9. Those passes found that the two-condition skip rule and the profiled/unprofiled evidence contract were each stated incompletely at several sites; both are now explicit at their source in §5 and consistent across the spec, plan, story, command and docs. One finding dismissed with reason: the version-bump Blocker measured the range against an intermediate commit. Invariant 12 is per-pull-request and CI compares against origin/$BASE_REF; against main the check is green, because this PR bumps 0.6.0 -> 0.7.0. Both branches later confirmed the dismissal. Verification: full battery green on the tree this commit records — shellcheck (6 files), hook tests, check-invariants + suite (123 assertions), check-version-bump + suite (36 assertions), claude plugin validate --strict; exit 0. The story is unprofiled, so no mode-derived evidence entry is owed.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/superpowers/plans/2026-07-26-risk-security-validation-profiles.md`:
- Around line 607-615: Update the initial snapshot procedure around the git
status, git add, and WIP commit commands to record each target path’s
pre-existing dirty state and stage only changes introduced during this cycle,
following the guard pattern described later around lines 700-704. Before
committing, verify both the exact staged path list and staged content so
unrelated pre-existing edits cannot enter the snapshot.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c79ba410-0a8b-4d1e-a2bc-178cb31e7844
📒 Files selected for processing (9)
CLAUDE.mddocs/coding-workflow.mddocs/getting-started.mddocs/superpowers/plans/2026-07-26-risk-security-validation-profiles.mddocs/superpowers/specs/2026-07-26-risk-security-validation-profiles-design.mddocs/superpowers/stories/2026-07-26-risk-security-validation-profiles-story.mdplugins/dev-workflow/CHANGELOG.mdplugins/dev-workflow/commands/process-pr-review.mdplugins/dev-workflow/commands/workflow-init.md
🚧 Files skipped from review as they are similar to previous changes (8)
- docs/getting-started.md
- plugins/dev-workflow/CHANGELOG.md
- plugins/dev-workflow/commands/workflow-init.md
- CLAUDE.md
- docs/coding-workflow.md
- plugins/dev-workflow/commands/process-pr-review.md
- docs/superpowers/stories/2026-07-26-risk-security-validation-profiles-story.md
- docs/superpowers/specs/2026-07-26-risk-security-validation-profiles-design.md
Follow-up finding from CodeRabbit on the fix head: Task 5 Step 3 staged its explicit path list wholesale, folding any pre-existing edits in those files into the WIP snapshot. Step 6's fix loop gained that guard at Gate-B pass 7; the initial snapshot never did. Step 3 now states the same rule — if a listed path was already dirty before the cycle, stage only this cycle's hunks or stop and ask — and verifies with git diff --cached --name-only and git diff --cached before committing. Gate B: clean on both branches at pass 1, the documented early exit for a zero-finding pass. check-invariants green.
…ot a hazard Fourth occurrence (#12, #13, #15, #16 — the last observed while writing this row): the status check passes while the comment reads "Review rate limited" and the live head has no review record. The row now says never merge on the check alone; the review count is the arbiter, and the verification command runs on every merge rather than when something looks off. Docs-only (docs/**.md), so Gate B is N/A per CLAUDE.md §5's prose exemption.
* harden: three classes from the profiles cycle, at the rungs that fit Runs dev-workflow:harden-finding on the three classes that recurred through the risk/security/validation profiles work. All three land as text; none reaches a mechanical rung, and each ledger row says so rather than implying otherwise. unverified-enforcement-claim, 4th occurrence, rung P — prompt-standards item 11 gains two rules: where the reader can reach the authoritative source, cite it instead of restating it (with invariant 8's self-contained-template exception named), and delete a mechanism claim that has needed a fourth correction rather than refining it again. From a paragraph describing the hook's path matcher that took four consecutive Gate-B corrections, each a subtler version of the last, and closed only when the enumeration was deleted. rewrite-drops-prior-condition, NEW class, rung 1 — a new AGENTS.md Don't: never replace a decision procedure without listing its old conditions and marking each kept, moved, or deliberately dropped. Ten instances in one cycle, one of which briefly made an eligible profile sufficient for a Gate-B skip: a gate-off path invented by the change that exists to close one. docs-drift, 4th occurrence with a new mechanism, rung P — a standing lens on every Gate-B call in both §5 copies: "which existing statements does this diff falsify?" It found a shipped command that would have let a one-line fix skip Gate B, plus two docs teaching a rule the same change had narrowed. Two escalations deliberately refused, with the reasoning in the rows: both lineages' latest entries are rung-2 checks guarding one spelling each (Target-model lines; prose count claims), and both new defects fall outside those spellings — the over-escalation those rows warn about by name. Gate B: 7 findings at pass 1, then 1, 1, and clean on both branches at pass 4. Three of those findings were this change committing the classes it hardens — the item-11 fix reaching only the repo copy, the new rule contradicting invariant 8, and the lens claiming no check could reach the class. Verification: full battery green — shellcheck (6 files), hook tests, check-invariants + suite (123 assertions), check-version-bump + suite (36 assertions), claude plugin validate --strict; exit 0. * docs(bots): the CodeRabbit rate-limit pattern is settled behaviour, not a hazard Fourth occurrence (#12, #13, #15, #16 — the last observed while writing this row): the status check passes while the comment reads "Review rate limited" and the live head has no review record. The row now says never merge on the check alone; the review count is the arbiter, and the verification command runs on every merge rather than when something looks off. Docs-only (docs/**.md), so Gate B is N/A per CLAUDE.md §5's prose exemption. * docs(taxonomy): clarify the rewrite-drops-prior-condition definition PR #16 review finding: "one the old prose carried" was ambiguous; it now reads "one of the conditions the old prose carried". Docs-only (docs/**.md), so Gate B is N/A per CLAUDE.md §5's prose exemption. Validated directly rather than by a finding-triage subagent — the PR edits instruction-bearing paths, which the command's step-0 precheck routes to manual validation.
Docs-only, two files, no plugin path — no version bump. CodeRabbit leaves Wait for after five consecutive unreviewed heads (#12, #13, #15, #16, #17 — the last with zero review records on the PR) followed by a genuine review on #18. Real findings source, unpredictable delivery, and a completion signal that fires regardless of whether a review happened: the opportunistic category by this file's own definition. Wait for is now empty. Row additions: the status check goes green whether or not a review happened, and `@coderabbitai review` is a no-op while automatic reviews are active (CodeRabbit's own message on #17), which retroactively explains #14's "re-trigger produced nothing". Plan corrected to Free; "Pro Plus" was observed on PR #1 only. The count rule's two facts separated: the per-head count remains the arbiter of whether a head was reviewed, for any bot; the recorded-human-decision requirement binds only bots under Wait for, and is dormant while that list is empty. Every head reaching a PR has already passed Gate B, so a quiet supplementary reviewer needs no exception. MANIFEST.md: the bare `CLAUDE.md` row resolved to the repo root and produced a false Major on #18. Qualified to source-files/CLAUDE.md, with the three files distinguished, and root §6 (context canary) recorded as deliberately outside the §1–§5 template range and never to be synced into the scaffolded template. Pre-merge diagnostic: per-head count 1 on head 2634bf2 — reviewed. One Minor finding (drop or explicitly optionalize the re-trigger step) collected, not actioned, per §5's Minor/Nit rule. Gate B: N/A — every path is explanatory documentation per §5's prose rule. Battery green at each commit.
Closes the P2 half of the P2+P6 backlog row; P6's standalone-section half is recorded as deliberately rejected rather than shipped.
What ships
**Profile log:**### Profilessubsection — lens sets per axis, three answers for reading a profile, the author's evidence obligations, the change procedure — plus one Mechanics clause on the cycle-closing amendTwo axes, one derived mode. Risk
trivial|standard|highvia named domain triggers (auth, permissions, payments, migrations, data deletion, public APIs, personal data, supply chain) and named effect triggers (irreversibility, data loss or corruption, outage exposure). Security relevancenone|standard|high. Validation mode =max(risk, security)→battery/battery+check/battery+check+verification, plus a mandatory+abuse-pathat securityhigh.The axes steer the reviewer's questions; the mode steers the author's evidence. Orthogonal on purpose — had the mode also steered the gates it would be a second name for the axes.
What it deliberately does not do
trivialdoes not relax Gate A at any level. A spec misjudged as trivial would skip design review entirely and everything downstream inherits it.Single writable copy
The story header is the only place profile values live. Specs, plans, commit bodies and the gate prompts carry the story path and re-read the header at each pass. The profile log records changes — event kind, axis, per-axis direction, reason or trigger — and never restates values.
Scope-guard waiver
The story excluded
hooks/**. One string changed anyway, waived by the maintainer during Gate B pass 11, reasoning in the plan's decision record: the below-floor reminder asserted a skip rule the narrowing made false, at exactly the moment an author decides whether to skip. The edit removes a rule statement and defers to the policy file — the hook reads no profile and gains no machinery, ending up more profile-agnostic than before. It points at the policy without paraphrasing the rule, so no new sync surface is created.Gates
Findings files and per-pass dispositions are in
.context/codex-reviews/.Three defect classes recurred, and only one is mechanically catchable:
process-pr-review) that would have let a one-line fix on a security-relevant story skip Gate B entirely.Two findings deserve naming. The gate-overclaim class regenerated four times in one paragraph, each correction a subtler version of the last — exactly as AGENTS.md predicts — and was closed only by deleting the claim rather than refining it a fifth time. And a rewrite of
process-pr-reviewbriefly made story eligibility sufficient for a skip without requiring the fix to be trivial: a new gate-off path invented by the very change that exists to narrow one.Verification
Full battery green on the committed tree — shellcheck (6 files), hook tests,
check-invariants+ suite (123 assertions),check-version-bump+ suite (36 assertions),claude plugin validate --strict; exit 0. Re-run after every fix round and immediately before the closing commit.Summary by CodeRabbit
New Features
Improvements
Chores