Skip to content

[CI] - Stop pre-checking Sentry auto-fix PR reviewers via collaborators endpoint - #4313

Open
finnar-bin wants to merge 5 commits into
devfrom
fix/4298-resolve-reviewer-false-flag-checks
Open

[CI] - Stop pre-checking Sentry auto-fix PR reviewers via collaborators endpoint#4313
finnar-bin wants to merge 5 commits into
devfrom
fix/4298-resolve-reviewer-false-flag-checks

Conversation

@finnar-bin

@finnar-bin finnar-bin commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Fixes #4298

Summary

  • The "Open auto-fix PR" step in .github/workflows/claude-sentry-handler.yml used to pre-check each candidate reviewer (agalin920, geodem127, finnar-bin) via gh api repos/{owner}/{repo}/collaborators/{username} before calling gh pr create --reviewer.

  • That endpoint can't see access granted via org team membership under the default GITHUB_TOKEN, so it 404'd for all three despite them having real write access through team membership — the pre-check treated them as having no access and silently skipped every reviewer request on every auto-fix PR.

  • Fix: create the PR first without --reviewer, then request each candidate individually by POSTing directly to repos/{owner}/{repo}/pulls/{n}/requested_reviewers.

    • Not gh pr edit --add-reviewer: confirmed live against this PR that its GraphQL query pulls the deprecated Projects (classic) projectCards field, which errors out on every invocation in this repo regardless of the reviewer's validity — using it would have warned on every single reviewer, every time, masking valid ones just like the original bug.
    • The REST endpoint has its own quirk: it returns HTTP 200 and silently drops an unknown/nonexistent login instead of erroring, so exit code alone can't detect a bad login.
    • The warning now surfaces the actual reason instead of a generic message: GitHub's .message for a real access error (e.g. "not a collaborator"), or an explicit note when the login was silently dropped from a 200 response with no reviewer added.
  • Verified against three real response shapes by testing live against this PR (cleaning up each test review request immediately after):

    • valid login, has access → added, appears in requested_reviewers, no warning
    • nonexistent login → HTTP 200, silently absent from requested_reviewers → warning: "the login may not exist"
    • real user, not a collaborator → HTTP 422 with a message → warning includes that message verbatim

    All three are handled correctly, and set -euo pipefail does not abort the step on either failure case (the failing command sits inside resp=$(...) || true, so it isn't part of any &&/pipeline chain that -e would catch).

  • Single file changed: .github/workflows/claude-sentry-handler.yml.

Test plan

  • Validated YAML syntax locally: python3 -c "import yaml; yaml.safe_load(open('.github/workflows/claude-sentry-handler.yml'))"
  • Live-tested the REST reviewer-request call against this PR itself ([CI] - Stop pre-checking Sentry auto-fix PR reviewers via collaborators endpoint #4313) for all three outcome shapes (real access, nonexistent login, real user without access), then removed the test review requests immediately after
  • Confirmed gh pr edit --add-reviewer (and gh pr edit in general) fails unconditionally in this repo due to an unrelated Projects (classic) deprecation, which is why the fix avoids it entirely
  • Full end-to-end verification still requires a real sentry-rca workflow run that produces a simple verdict (per the issue's acceptance criteria), since org team-membership resolution under the Actions GITHUB_TOKEN can only be exercised from within Actions

… endpoint

GET repos/{owner}/{repo}/collaborators/{username} can't see access granted
via org team membership under GITHUB_TOKEN, so it 404s for team-granted
reviewers and silently drops all of them. Create the PR first, then request
each candidate reviewer individually so a genuinely stale login just warns
instead of masking valid ones.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@finnar-bin finnar-bin added bug Something isn't working severity: low Minor impact — cosmetic, rare edge case, or easy workaround labels Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

QA Review — ⚠️ INCONCLUSIVE

Validates #4298: CI - Sentry auto-fix PR reviewer check false-flags team-access reviewers as having no access

  1. ✅ Step no longer gates on GET repos/{owner}/{repo}/collaborators/{username} — that pre-check is removed; reviewers are now requested via a direct POST to .../pulls/{pr}/requested_reviewers.
  2. ⚠️ All valid candidates get requested on a real simple-verdict run — the request logic looks correct (POST per login, response checked for the login in requested_reviewers), but this can only be confirmed by an actual workflow run against the GitHub API.
  3. ✅ A single stale/invalid login degrades gracefully — the PR is created with no --reviewer args first, then each candidate is requested individually with || true guarding the API call, so one failure can't block the others or the PR itself.
  4. ✅ An un-addable reviewer still emits a visible ::warning:: instead of a silent skip.
  5. set -euo pipefail is untouched and the per-reviewer POST is wrapped with || true, so a failed add can't abort the step.
  6. ✅ No permissions: block changes are needed or present — the diff doesn't touch it.
Suggested Cypress coverage

This change is confined to .github/workflows/claude-sentry-handler.yml and doesn't touch any src/ app code, so there's no applicable spec under cypress/e2e/ — Cypress here exercises the manager-ui SPA, not GitHub Actions workflows, and no existing area (e.g. content/, settings/) has any bearing on this reviewer-request logic. Confidence in this fix instead comes from a live or manually triggered sentry-rca run (per the issue's own acceptance criteria) to confirm the three candidates are actually requested and that a deliberately-broken login still logs a warning without blocking PR creation.

Comment thread .github/workflows/claude-sentry-handler.yml Outdated
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers · 🟡 1 warning(s) — see inline comments

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Negative QA

No reproducible edge-case failures found on the surfaces this PR changes.

Also checked and working correctly
  • Confirmed the diff is scoped entirely to .github/workflows/claude-sentry-handler.yml (CI/automation config), with no changes to src/ — there is no app route, component, or UI surface for this PR to affect, so no browser testing was performed.

@finnar-bin
finnar-bin requested review from agalin920 and removed request for agalin920 September 1, 2026 01:25
finnar-bin and others added 2 commits September 1, 2026 09:27
…body

gh pr edit --add-reviewer fails on every call in this repo: its GraphQL
query pulls the deprecated Projects (classic) projectCards field, which
errors out regardless of the reviewer. Confirmed by testing directly
against PR #4313 with both a real accessible user and a nonexistent one —
both failed identically on the projectCards error, meaning every reviewer
request would have warned even for valid logins.

Switch to POSTing repos/{owner}/{repo}/pulls/{n}/requested_reviewers
directly. That endpoint also has its own quirk: it returns 200 and
silently drops an unknown login instead of erroring, so success can't be
read from the exit code alone. Verified against three real response
shapes (valid login added, unknown login silently dropped, real user
without access returns 422) — the fix now checks the response body for
the login rather than trusting exit status.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…dded

The prior warning just said "no longer has access" for every failure
case, without saying why. Extract the real reason from the response: the
GitHub-provided .message for an actual access error (e.g. not a
collaborator), or an explicit note that the login was silently dropped
when GitHub returns 200 without adding it (the observed behavior for an
unknown/misspelled login).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@finnar-bin

Copy link
Copy Markdown
Contributor Author

Thanks for confirming — matches expectations, since this PR only touches .github/workflows/claude-sentry-handler.yml (the Sentry auto-fix PR's reviewer-request step) and has no src/ or UI surface for the negative-QA browser pass to exercise.

For anyone following along: this change was also verified live against this PR itself while developing it — the reviewer-request REST call was exercised against three real response shapes (valid reviewer added, unknown login silently dropped by GitHub, and a real non-collaborator returning 422), with each test review request removed immediately after. Full end-to-end coverage of the actual reviewer-assignment behavior still needs a real sentry-rca run with a simple verdict, since it depends on org team-membership resolution under the Actions GITHUB_TOKEN that can only be exercised from within Actions.

Comment thread .github/workflows/claude-sentry-handler.yml
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers · 🟡 1 warning(s) — see inline comments

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Overall Coverage

Lines Statements Branches Functions
61.31% 60.88% 56.22% 55.68%

Changed Files Coverage

File Lines Statements Branches Functions
No changed files found - - - -

geodem127
geodem127 previously approved these changes Sep 1, 2026
@finnar-bin finnar-bin self-assigned this Sep 1, 2026
agalin920
agalin920 previously approved these changes Sep 2, 2026
Comment thread .github/workflows/claude-sentry-handler.yml
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers · 🟡 1 warning(s) — see inline comments

# Conflicts:
#	.github/workflows/claude-sentry-handler.yml
@finnar-bin
finnar-bin dismissed stale reviews from agalin920 and geodem127 via c241916 September 4, 2026 00:13
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Code Review — ✅ No blockers

@finnar-bin
finnar-bin enabled auto-merge (squash) September 4, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working severity: low Minor impact — cosmetic, rare edge case, or easy workaround

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI - Sentry auto-fix PR reviewer check false-flags team-access reviewers as having no access

3 participants