feat: add verbose security scan diagnostics - #63
Conversation
…ty-scan-verbose # Conflicts: # README.md # sdk/typescript/README.md # sdk/typescript/src/cli.ts # sdk/typescript/tests-ts/cli.test.ts
|
@codex review |
ianw-oai
left a comment
ianw-oai
left a comment
There was a problem hiding this comment.
Looks good. I left one non-blocking suggestion to keep the quick start from running multiple paid scans.
There was a problem hiding this comment.
Pull request overview
Adds an opt-in verbose diagnostics mode for the codex-security scan CLI that emits redacted lifecycle telemetry to stderr (and supports env-based enablement) while preserving clean JSON stdout for CI and tooling.
Changes:
- Introduces
--verbose(andCODEX_SECURITY_LOG_LEVEL=debug/LOG_LEVEL=debug) to enable redacted scan lifecycle diagnostics on stderr. - Implements structured diagnostic events across scan configuration, authentication, worker status, reconnects, warnings, cost updates, completion, and cleanup.
- Expands test coverage and updates READMEs + package smoke test to validate help text and verbosity behaviors.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| sdk/typescript/src/cli.ts | Adds verbose flag parsing and emits redacted diagnostic events to stderr throughout runScan. |
| sdk/typescript/tests-ts/cli.test.ts | Adds extensive tests ensuring verbose diagnostics don’t contaminate JSON stdout, and validates env precedence + redaction. |
| sdk/typescript/scripts/smoke-package.mjs | Extends installed-CLI smoke checks to assert scan --help documents --verbose and CODEX_SECURITY_LOG_LEVEL=debug. |
| sdk/typescript/README.md | Documents --verbose and env-var enablement, including precedence rules and what diagnostics include. |
| README.md | Adds quick-start examples for --verbose and env-driven diagnostics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 45fd824abd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
ianw-oai
left a comment
ianw-oai
left a comment
There was a problem hiding this comment.
Approved. The updated quick start now shows one scan, with optional diagnostics explained separately.
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
sdk/typescript/src/cli.ts:893
- The scan --help text documents CODEX_SECURITY_LOG_LEVEL=debug but the implementation also supports a LOG_LEVEL=debug fallback (and CODEX_SECURITY_LOG_LEVEL precedence). This should be reflected in the option description so users discover the supported env var behavior from CLI help.
verbose: z
.boolean()
.default(false)
.describe(
"Print redacted scan lifecycle diagnostics to stderr; CODEX_SECURITY_LOG_LEVEL=debug also enables this.",
),
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3ec9ed8e67
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
sdk/typescript/README.md:221
- The README says "API keys, tokens, and raw provider messages are not printed", but verbose diagnostics explicitly include token counts (e.g., input/cached/output/cache-write tokens). This wording is misleading—consider clarifying that secret credentials are redacted while token counts/costs may be printed.
output. Diagnostics include CLI and runtime versions, the selected credential
source, requested reasoning effort, prepared scan output, worker phases and
capacity, classified connection retries, redacted scan warnings, cost updates,
completion, and runtime cleanup. API keys, tokens, and raw provider messages are
not printed. Use `--verbose --json` or
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…ty-scan-verbose # Conflicts: # sdk/typescript/src/cli.ts
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ac8692ba32
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ianw-oai
left a comment
ianw-oai
left a comment
There was a problem hiding this comment.
Approved; one non-blocking documentation fix noted inline.
|
@codex review |
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Summary
codex-security scan --verbosediagnostics entirely in the CLI; leave the bundled plugin and public SDK API unchanged.CODEX_SECURITY_LOG_LEVEL=debugand Promptfoo-compatibleLOG_LEVEL=debug; document flag precedence and identify both environment variables as CLI-only.main, preserving managed Codex credential keyrings, transient environment API keys, hardened Windows sandbox compatibility, expanded Node 22/24/26 CI, portable npm package-smoke timeouts, native multi-agent worker limits, and0.1.3release metadata.Test plan
bun test --timeout 30000 ./tests-ts— 493 passed, 6 existing integration/platform skips, 0 failures; 499 tests across 30 files.pnpm --dir sdk/typescript run types.pnpm --dir sdk/typescript run format.pnpm --dir sdk/typescript run build.node sdk/typescript/scripts/check-package.mjs <packed-tarball>— public SDK import, installed CLI, 95 bundled plugin files and all 179 package entries.--verbose, both log-level variables and precedence, explicit and legacy overrides, all three selected-profile combinations, consistent diagnostics and JSON stdout, and no paid scan.Maintainer tracking
Related to #70: verbose diagnostics may improve scan observability, but this PR does not by itself establish scan-progress guarantees.