Skip to content

fix(error-reporting): Silence ValidationError#1232

Open
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/silence-validation-error-Msb3JA
Open

fix(error-reporting): Silence ValidationError#1232
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/silence-validation-error-Msb3JA

Conversation

@sentry

@sentry sentry Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Currently, ValidationErrors, which typically indicate malformed user input (e.g., invalid JSON for --condition flags), are not explicitly silenced in the CLI's error reporting mechanism. This leads to these user-facing errors being captured and sent to Sentry as if they were internal bugs.

This PR addresses the issue by:

  • Adding "validation_error" to the SilenceReason type in src/lib/error-reporting.ts.
  • Modifying classifySilenced() to recognize ValidationError and return "validation_error", thereby preventing these errors from being reported to Sentry.

This ensures that Sentry reports are focused on actual CLI defects, while user input errors are handled gracefully without generating unnecessary Sentry events.

Fixes CLI-1ZD

Comment @sentry <feedback> on this PR to have Autofix iterate on the changes.

@github-actions github-actions Bot added the risk: high PR risk score: high label Jul 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://cli.sentry.dev/_preview/pr-1232/

Built to branch gh-pages at 2026-07-13 22:40 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Comment on lines +106 to +108
if (error instanceof ValidationError) {
return "validation_error";
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The change silences all ValidationError instances, but some are used for internal logic errors (like exhaustiveness checks), not just user input. This will hide CLI bugs from Sentry.
Severity: MEDIUM

Suggested Fix

Instead of silencing the entire ValidationError class, the silencing logic should be more specific. Consider silencing only ValidationError instances that are known to be from user input, perhaps by checking for a specific field or message pattern, or by only silencing them when thrown from specific validation functions. This would allow internal errors to still be captured.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/lib/error-reporting.ts#L106-L108

Potential issue: The change in `classifySilenced` now silences all `ValidationError`
instances. However, the codebase uses `ValidationError` not only for user input errors
but also for internal CLI defects. For example, exhaustiveness checks in
`src/lib/trace-target.ts` throw `ValidationError` to catch unhandled cases in
discriminated unions. Silencing these errors means that genuine bugs in the CLI's logic
will no longer be reported to Sentry, hindering debugging and maintenance. An existing
test explicitly asserts that `ValidationError` should not be silenced, which will fail
with this change.

Also affects:

  • src/lib/trace-target.ts
  • src/commands/trial/start.ts

Did we get this right? 👍 / 👎 to inform future reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: high PR risk score: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants