docs: improve vulnerability writeup quality - #262
Conversation
|
@codex review Please review the exact current head |
github-actions
Bot
added
the
documentation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9232085f31
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - Do not manufacture PoC commands, logs, screenshots or sample output. Include observed output only when it was actually produced or when a supplied, identifiable trace was inspected. Label a prediction as expected output and explain that it was not observed. | ||
| - Write in the language and locale the user requests; when they do not specify one, use their normal default. Be warm, direct and exact; guide substantive reasoning with a natural `we`, and use `I` only for work actually performed. Describe what the software should do, what it actually does and why that matters in plain language. | ||
| - Give people clear, conventional names when they help explain the finding: Alice is the legitimate account or resource owner, Bob is another legitimate user or intended recipient, Mallory is the active attacker, and Eve is a passive observer. Use matching example usernames such as `alice`, `bob`, `mallory` and `eve` consistently in prose, commands and PoCs. | ||
| - Make every delivered report and PoC portable and self-contained. Use repository-relative source paths, report-relative commands and verified software versions. Never include a local absolute path in report prose, excerpts, citations, links, PoC code, build files, command examples or captured output. |
There was a problem hiding this comment.
When the vulnerability or PoC inherently targets an absolute path—such as /etc/passwd, a /proc or /dev node, or a Unix-domain socket—this blanket rule requires the writer to remove that path even from PoC source and command examples, making the artifact incorrect or unusable. Limit the prohibition to environment-specific author-machine paths while explicitly allowing absolute paths that are part of the verified target behavior.
Useful? React with 👍 / 👎.
| Do not hand off until validation passes or the remaining failures are explicitly explained to the user. | ||
| - a new reader can understand the component, the named actors and the relevant security boundary; | ||
| - the verified release, configuration, attacker prerequisites and affected-version history are accurately scoped; | ||
| - the source establishes the same trigger sequence and security failure described in the report; |
There was a problem hiding this comment.
When the exact source is unavailable and the user explicitly accepts the conditional report-only assessment permitted above, this unconditional acceptance check can never pass because there is no source available to establish the trigger sequence. The main agent is therefore instructed to reject indefinitely the exact fallback the skill allows; add an exception that accepts clearly conditional, source-unverified claims in this mode.
Useful? React with 👍 / 👎.
| - the report and every distributed PoC, script, build file and output contain no local absolute paths, internal provenance, placeholder text or fabricated detail. | ||
|
|
||
| A report with no natural first-person walkthrough, or no truthful singular account of the validation basis and limits, fails validation and must be rewritten. A mechanical pronoun match does not make an impersonal report pass. | ||
| Validate front matter and Markdown formatting with the repository's plugin authoring validator. Search every distributable report, PoC, build file, script and captured output for local macOS, Linux and Windows absolute paths, including user-home, temporary, checkout and `file://` paths; remove every match before delivery. Run relevant real PoC build or dry-run checks only when they exist, are safe and are supported by the target environment. When a disclosure package also contains an advisory, validate the one direct technical report through an isolated temporary validation directory or link rather than misclassifying the advisory or changing the final package layout. Re-run any supplied offline evidence verifier and ensure generated bytecode or local-path leakage does not enter the package. A word search can help identify accidental provenance or missing researcher voice, but neither a pronoun count nor required headings can establish factual accuracy. |
There was a problem hiding this comment.
At final acceptance this directs every run to a repository “plugin authoring validator,” but a repo-wide search finds no such validator or invocation, and plugin authoring validation would target a SKILL.md with YAML front matter rather than the generated disclosure report. The bundled scripts/validate_report_format.py is not a substitute because it requires the unrelated Scope/Threat Model/Findings security-review format, so the required validation cannot be completed as written; specify a real vulnerability-report validator and command, or remove this step.
Useful? React with 👍 / 👎.
|
|
||
| Discuss stronger exploitation routes only where the underlying source or authorised experiments support their premises. Explain meaningful constraints such as allocator behaviour, controllable bytes, protocol ordering, configuration, scheduling and cleanup. Label a possible chain or timing window as conditional when it has not been demonstrated. | ||
|
|
||
| Include positive and negative controls that rule out the strongest alternative explanations: show normal allowed access, fresh rejection at the crossed boundary, same-domain success where relevant, the real sink reached only by the attack and any one-setting mitigation that was actually tested. Explain what each control rules out. Do not turn ordinary reachability into remote code execution, an artificial interleaving into production reliability or a different identity into privilege escalation. |
There was a problem hiding this comment.
For a source-only assessment where no disposable target is available, this requires the report to show fresh rejection, allowed access, sink reachability, and tested mitigation even though the format explicitly permits an unexecuted trigger and expected-but-unobserved behavior. Those controls cannot actually rule out alternatives without runtime evidence, so the requirement either blocks an otherwise permitted report or pressures the writer to fabricate observations; require observed controls only when supplied traces or an authorized test environment exist, and otherwise record them as unperformed validation work.
Useful? React with 👍 / 👎.
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
@codex review Please review the exact updated head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4281cd3692
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| 1. Create the user-requested report directory, or use `reports/`. Inventory and deduplicate findings by root cause and source path rather than title. | ||
| 2. Read `references/report-format.md` completely. Require each drafting sub-agent to read it before writing. | ||
| 3. Pin and inspect each finding's vulnerable source. Independently check the decisive entry point, security check, state change, sink and available fix. Trace the introducing change and inspect the affected and fixed release tags before assigning the finding. |
There was a problem hiding this comment.
When the exact source is unavailable and the user accepts the report-only fallback allowed at line 21, this workflow still requires the main agent to pin and inspect that source before assigning the report, and the drafting prompt repeats the requirement at line 125. The updated head adds an exception only to final acceptance, so it does not provide a path to create the source-unverified draft in the first place; make the inspection and worker instructions conditional in report-only mode.
Useful? React with 👍 / 👎.
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
@codex review Please review the exact updated head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7e1aeeaf4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ## Vulnerability Details | ||
|
|
||
| 5. **Proof of Concept**: This should tie together all of the exploration and creative process performed in the Exploitability Analysis section into a maximally-effective and stable proof-of-concept that can be used with minimal configuration against the target. Don't just mention that a PoC is included: describe the approach it takes, any notes about reliability, and provide sample output from building and running it. Include exact relative commands from the report directory, expected output on vulnerable and fixed targets when possible, target requirements, cleanup instructions, and a clear warning when the PoC can crash, corrupt, delete data, or escalate on the test machine. Keep exploratory probes separate from the polished final PoC when both are included. | ||
| Follow the reported trigger in causal order. Begin with Mallory's actual controlled input, inspect each material check or state change and show the precise line where the code fails to enforce the expected behaviour. First establish that each ordinary policy or ownership check is configured correctly, then show the shared state, receiving component or dependency decision and real downstream operation that bypasses it. Carry the same request, object, field or state through that whole path. When a named attacker does not fit the actual mechanism, describe the real actor without forcing the example. |
There was a problem hiding this comment.
When the user accepts a report-only assessment because the source is unavailable, the worker is still required to read this format, which unconditionally tells it to inspect every material check and show the precise failing source line. The updated head makes the workflow and drafting prompt conditional, but this remaining requirement means the worker must either violate the required format or invent the unavailable source evidence; explicitly allow this section to present the supplied sequence as conditional when report-only mode is active.
Useful? React with 👍 / 👎.
| 3. When the vulnerable source is available, pin and inspect it. Independently check the decisive entry point, security check, state change, sink and available fix. Trace the introducing change and inspect the affected and fixed release tags before assigning the finding. When the user explicitly accepted a report-only assessment, record the unavailable source and require every source-dependent conclusion to remain conditional. | ||
| 4. Record the one-sentence attack and non-claims, verified release history, named actors, complete tested topology, defaults versus configured prerequisites, meaningful positive and negative controls, exact validation basis and testing boundary before assigning the finding. |
There was a problem hiding this comment.
When an exact source snapshot is available but its VCS history or release tags are not—for example, a vendor source archive or an offline shallow checkout—these steps still require tracing the introducing change, inspecting release tags, and recording verified release history before assigning the finding. That blocks an otherwise source-backed report even though the surrounding rules explicitly permit unavailable release evidence to be scoped as a limitation; condition these pre-assignment requirements on release history being available.
Useful? React with 👍 / 👎.
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
@codex review Please review the exact updated head |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
1 participant
Summary
Related: #204 addresses a narrower report-filename guidance issue; this PR syncs the broader upstream writeup rewrite.
Stack
main.Verification