feat(prompt): stop the model narrating change history in code - #281
Conversation
#280 made user-visible brevity actionable but said nothing about the artifacts the model writes. meetneptune/boxes#837 shows the gap: its branch adds 21,584 production and test lines, 5,354 of them comment lines (24.8%), with internal/api/notifications_types.go alone adding 294 comment lines in 480 additions. The comments narrate change history -- "pins the fix for...", "the exact defect this change...", "before #279...", "exactly as before this feature existed", "Red-verify: restore the previous floor" -- which causes review churn and leaves prose that goes stale beside the code. Add a second guidance paragraph prohibiting comments by default, tracking Codex's own rule in codex-rs/core/gpt_5_2_prompt.md ("Do not add inline comments within code unless explicitly requested", "Update documentation as necessary"), and route change history to commits, PRs, issues, or history docs. Tighten #280's response paragraph in the same edit to pay for it. Project-instruction precedence is unchanged and still the first guidance line, so a repo whose AGENTS.md requires comments or docs still gets them. One deviation from the approved wording, forced by the existing budget test: "pull requests" is "PRs". The approved strings measured 301 words by len(strings.Fields), one over the 300 cap, because the response paragraph is 31 words rather than the 30 the plan assumed. This was the smallest change that keeps every approved semantic element and full grammar. The block is now 21 lines / 300 words against the 25 / 300 caps -- zero word headroom, so the next wording change has to swap rather than append. Red-verified on main: TestBaseBehaviorGuidanceGovernsCommentsAndDocs failed on the missing prohibition, both its exceptions, and the docs and history-routing rules. Two #280 assertions moved with the rewording ("relax" to "unless", "file path" to "Cite paths").
There was a problem hiding this comment.
🟡 Changes recommended
A now-inaccurate doc comment in cmd/harness/main.go contradicts the newly added comment/docs policy and should be updated to avoid misleading future maintainers.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates Harness’s compiled-in baseBehaviorGuidance() prompt floor to explicitly discourage change-history narration in code artifacts by default (comments/docs), while keeping the existing adaptive brevity guidance for user-visible responses and enforcing the existing budget caps via tests.
Changes:
- Tighten the response-style paragraph (10-line default with explicit exceptions) and switch the “reference a file path” rule to “Cite paths”.
- Add new prompt guidance that prohibits comments by default (with explicit opt-in exceptions) and routes change history/incidents/reviews to commits/PRs/issues/docs.
- Update and extend tests to assert the new durable semantics and keep the prompt under the existing line/word budget.
File summaries
| File | Description |
|---|---|
cmd/harness/main.go |
Updates baseBehaviorGuidance() prompt text to add comment/docs policy and tighten brevity wording. |
cmd/harness/main_test.go |
Updates existing prompt-contract assertions and adds a new test covering comment/docs governance semantics. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
baseBehaviorGuidance's doc said comment policy was deliberately absent, which the new paragraph makes false. Commit conventions are still absent and that half stays. Addresses the Copilot review on #281.
There was a problem hiding this comment.
🟢 Approval recommended
The change is low-risk (prompt text + tests), stays within the enforced budgets, and the updated/new tests concretely enforce the intended semantics.
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
- Review effort level: Lite
Problem
#280 made user-visible brevity actionable, but the base prompt still said nothing about the artifacts the model writes.
meetneptune/boxes#837is the concrete case:internal/api/notifications_types.goThe comments narrate change history rather than explaining code:
That produces review churn, and it leaves prose beside the code that is false or stale the moment the code moves again —
git blameand the PR already hold that history.Change
Two paragraphs in
baseBehaviorGuidance(). #280's response paragraph is tightened to pay for the new one:Codex basis.
codex-rs/core/gpt_5_2_prompt.mdcarries both halves: "Do not add inline comments within code unless explicitly requested", and separately "Update documentation as necessary." This is the strong form of that rule, not an adaptive one.Precedence is unchanged.
Project instructions (AGENTS.md) override this guidance where they conflict.is still the first line of the block, so a repo whose AGENTS.md requires comments or docs still gets them. The paragraph also names that exception inline.One deviation from the approved wording
The approved strings measured 301 words by the production test's own
len(strings.Fields)— one over the 300 cap — because the response paragraph is 31 words, not the 30 the plan assumed. The existingTestBaseBehaviorGuidanceStaysUnderBudgetcaught it:That is the "literal budget/count mistake" case, so I made the smallest possible fix:
pull requests→PRs. It keeps every approved semantic element and full grammar. Flagging it explicitly because the prose was approved verbatim — say the word and I'll trade a different one instead.The line count also came out 21, not the predicted 20: paragraphs join with
\n\n, so an added paragraph adds 2 tostrings.Count(block, "\n")+1.Budget
Measured with the production test's method, not by hand. Zero word headroom, deliberately — the next wording change must swap, not append.
Tests
TestBaseBehaviorGuidanceGovernsCommentsAndDocsasserts durable semantics rather than either paragraph byte-for-byte:explicitly requested,project instructions) so a repo can opt in;concise, direct, and friendly,10 lines,unless).Red on main: fails on the missing prohibition, on both exceptions, and on the docs and history-routing rules.
Two #280 assertions moved with the rewording:
relax→unless, andfile path→Cite paths(in both #280's test and theTestSystemPromptCarriesBaseBehaviorGuidancecontract list).go test -race ./cmd/harness/,go test -race ./...,go build ./...,go vet ./..., changed-filegofmt,git diff --check— all clean.Evidence status
#280's live A/B (separate box, real credential) measured the response paragraph's lineage: complex handoff −15% output tokens, −25% visible characters, −27% stream time; already-concise answer −4% tokens, with diagnosis, measurements, fix, verification and follow-ups all preserved.
No A/B is claimed for the comments/docs sentence. It targets code artifacts, not response text, so a prompt-response A/B cannot measure it. It will be evaluated on future PR data — comment-line share of additions, and review rounds per PR — against
boxes#837's 24.8% as the baseline.Prose audit
3 added
//lines, all one named-failure comment on the new test. No new production comments. No measurements, issue numbers, or task history in code; they are in this body and the commit message.