Skip to content

feat(prompt): stop the model narrating change history in code - #281

Merged
andybons merged 2 commits into
mainfrom
prompt/no-narrating-comments
Sep 10, 2026
Merged

feat(prompt): stop the model narrating change history in code#281
andybons merged 2 commits into
mainfrom
prompt/no-narrating-comments

Conversation

@andybons

Copy link
Copy Markdown
Contributor

Problem

#280 made user-visible brevity actionable, but the base prompt still said nothing about the artifacts the model writes. meetneptune/boxes#837 is the concrete case:

production + test lines added on the branch 21,584
of which comment lines 5,354 (24.8%)
internal/api/notifications_types.go 294 comment lines in 480 additions

The comments narrate change history rather than explaining code:

That produces review churn, and it leaves prose beside the code that is false or stale the moment the code moves again — git blame and the PR already hold that history.

Change

Two paragraphs in baseBehaviorGuidance(). #280's response paragraph is tightened to pay for the new one:

Be concise, direct, and friendly. Keep responses under 10 lines unless correctness, security, review findings, or understanding require more. Lead with outcomes and next steps. Cite paths; don't repeat tool output.

Do not add comments unless explicitly requested or required by project instructions. Write docs only when needed; keep them concise and current. Put change history, incidents, and reviews in commits, PRs, issues, or history docs.

Codex basis. codex-rs/core/gpt_5_2_prompt.md carries both halves: "Do not add inline comments within code unless explicitly requested", and separately "Update documentation as necessary." This is the strong form of that rule, not an adaptive one.

Precedence is unchanged. Project instructions (AGENTS.md) override this guidance where they conflict. is still the first line of the block, so a repo whose AGENTS.md requires comments or docs still gets them. The paragraph also names that exception inline.

One deviation from the approved wording

The approved strings measured 301 words by the production test's own len(strings.Fields) — one over the 300 cap — because the response paragraph is 31 words, not the 30 the plan assumed. The existing TestBaseBehaviorGuidanceStaysUnderBudget caught it:

main_test.go:1244: baseBehaviorGuidance = 301 words, want at most 300

That is the "literal budget/count mistake" case, so I made the smallest possible fix: pull requestsPRs. It keeps every approved semantic element and full grammar. Flagging it explicitly because the prose was approved verbatim — say the word and I'll trade a different one instead.

The line count also came out 21, not the predicted 20: paragraphs join with \n\n, so an added paragraph adds 2 to strings.Count(block, "\n")+1.

Budget

lines words
before (#280) 19 298
after 21 300
cap 25 300

Measured with the production test's method, not by hand. Zero word headroom, deliberately — the next wording change must swap, not append.

Tests

TestBaseBehaviorGuidanceGovernsCommentsAndDocs asserts durable semantics rather than either paragraph byte-for-byte:

  1. comments prohibited by default;
  2. both exceptions present (explicitly requested, project instructions) so a repo can opt in;
  3. docs written only when needed, concise and current;
  4. change history / incidents / reviews routed to commits and issues;
  5. feat(harness): give the base prompt Codex's adaptive brevity default #280's adaptive response default survives (concise, direct, and friendly, 10 lines, unless).

Red on main: fails on the missing prohibition, on both exceptions, and on the docs and history-routing rules.

Two #280 assertions moved with the rewording: relaxunless, and file pathCite paths (in both #280's test and the TestSystemPromptCarriesBaseBehaviorGuidance contract list).

go test -race ./cmd/harness/, go test -race ./..., go build ./..., go vet ./..., changed-file gofmt, git diff --check — all clean.

Evidence status

#280's live A/B (separate box, real credential) measured the response paragraph's lineage: complex handoff −15% output tokens, −25% visible characters, −27% stream time; already-concise answer −4% tokens, with diagnosis, measurements, fix, verification and follow-ups all preserved.

No A/B is claimed for the comments/docs sentence. It targets code artifacts, not response text, so a prompt-response A/B cannot measure it. It will be evaluated on future PR data — comment-line share of additions, and review rounds per PR — against boxes#837's 24.8% as the baseline.

Prose audit

3 added // lines, all one named-failure comment on the new test. No new production comments. No measurements, issue numbers, or task history in code; they are in this body and the commit message.

#280 made user-visible brevity actionable but said nothing about the
artifacts the model writes. meetneptune/boxes#837 shows the gap: its branch
adds 21,584 production and test lines, 5,354 of them comment lines (24.8%),
with internal/api/notifications_types.go alone adding 294 comment lines in
480 additions. The comments narrate change history -- "pins the fix for...",
"the exact defect this change...", "before #279...", "exactly as before this
feature existed", "Red-verify: restore the previous floor" -- which causes
review churn and leaves prose that goes stale beside the code.

Add a second guidance paragraph prohibiting comments by default, tracking
Codex's own rule in codex-rs/core/gpt_5_2_prompt.md ("Do not add inline
comments within code unless explicitly requested", "Update documentation as
necessary"), and route change history to commits, PRs, issues, or history
docs. Tighten #280's response paragraph in the same edit to pay for it.

Project-instruction precedence is unchanged and still the first guidance
line, so a repo whose AGENTS.md requires comments or docs still gets them.

One deviation from the approved wording, forced by the existing budget test:
"pull requests" is "PRs". The approved strings measured 301 words by
len(strings.Fields), one over the 300 cap, because the response paragraph is
31 words rather than the 30 the plan assumed. This was the smallest change
that keeps every approved semantic element and full grammar. The block is now
21 lines / 300 words against the 25 / 300 caps -- zero word headroom, so the
next wording change has to swap rather than append.

Red-verified on main: TestBaseBehaviorGuidanceGovernsCommentsAndDocs failed
on the missing prohibition, both its exceptions, and the docs and
history-routing rules. Two #280 assertions moved with the rewording
("relax" to "unless", "file path" to "Cite paths").

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A now-inaccurate doc comment in cmd/harness/main.go contradicts the newly added comment/docs policy and should be updated to avoid misleading future maintainers.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates Harness’s compiled-in baseBehaviorGuidance() prompt floor to explicitly discourage change-history narration in code artifacts by default (comments/docs), while keeping the existing adaptive brevity guidance for user-visible responses and enforcing the existing budget caps via tests.

Changes:

  • Tighten the response-style paragraph (10-line default with explicit exceptions) and switch the “reference a file path” rule to “Cite paths”.
  • Add new prompt guidance that prohibits comments by default (with explicit opt-in exceptions) and routes change history/incidents/reviews to commits/PRs/issues/docs.
  • Update and extend tests to assert the new durable semantics and keep the prompt under the existing line/word budget.
File summaries
File Description
cmd/harness/main.go Updates baseBehaviorGuidance() prompt text to add comment/docs policy and tighten brevity wording.
cmd/harness/main_test.go Updates existing prompt-contract assertions and adds a new test covering comment/docs governance semantics.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cmd/harness/main.go
baseBehaviorGuidance's doc said comment policy was deliberately absent,
which the new paragraph makes false. Commit conventions are still absent
and that half stays.

Addresses the Copilot review on #281.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is low-risk (prompt text + tests), stays within the enforced budgets, and the updated/new tests concretely enforce the intended semantics.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@andybons
andybons merged commit 14d6030 into main Sep 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants