feat(harness): give the base prompt Codex's adaptive brevity default

[andybons]

Problem

baseBehaviorGuidance() said only:

Keep your final message short. Reference a file path instead of pasting a file you just wrote, and lead with the outcome.

"Short" is not actionable, so user-visible answer length fell back to model default. On one live Sol session:

	that session	fleet-wide
median output tokens	506	73
stream p50	17.4s	0.5s
TTFT	only ~2.6s slower	—

TTFT barely moved, so the cost is in visible response generation — not latency to first token, tools, or hidden reasoning. That is what this targets.

Change

One paragraph, replacing that one line:

Be concise, direct, and friendly, keeping the user informed without unnecessary detail. Brevity matters by default: no more than 10 lines, relaxed where detail is important for the user's understanding, or where correctness, security, or review findings require it. Lead with the outcome, assumptions, and next steps. Reference a file path instead of pasting a file you just wrote. Do not repeat tool output.

Codex comparison

The reference paths in the original brief are stale — codex-rs/core/prompt.md 404s, and the prompts are now versioned per model (gpt_5_2_prompt.md, gpt-5.2-codex_prompt.md, …). I read the current source. The prose above tracks two places in codex-rs/core/gpt_5_2_prompt.md:

• Personality (line 15): "Your default personality and tone is concise, direct, and friendly. You communicate efficiently, always keeping the user clearly informed about ongoing actions without unnecessary detail."
• Presenting your work (line 170): "Brevity is very important as a default. You should be very concise (i.e. no more than 10 lines), but can relax this requirement for tasks where additional detail and comprehensiveness is important for the user's understanding."

Deliberately not imported: Codex's whole final-answer style guide (headers, bullet counts, monospace rules, file-reference grammar). This is one paragraph adapted to harness's existing terse list, not a style guide.

Why not OpenCode's rule

OpenCode prefers fewer than four lines and one-word answers. Rejected: a public model-vendor report finds overly aggressive brevity can suppress planning and coding quality on reasoning models. Codex's adaptive ceiling is the target instead — hence the explicit relax clause, which also names correctness, security, and review findings so the paragraph cannot be read as licence to skip investigation, tests, blockers, or necessary context.

Scope

• User-visible prose only.
• The pre-tool progress sentence is untouched — already one sentence, matching Codex's 1–2 sentence preamble guidance, and it carries no brevity cap, so there is no duplicate instruction.
• The findings-first review line is untouched.
• Project/user instructions override this exactly as before (the AGENTS.md-precedence line is unchanged and first in the block).
• The file-path reference rule is carried into the new paragraph verbatim rather than dropped.

Budget

	lines	words
before	19	256
after	19	298
budget	25	300

Line count unchanged; +42 words, 2 under the cap. So any wording request here needs to be a swap, not an addition — or the budget test fails, which is what it is for. The prompt grows so that the answers do not.

Tests

TestBaseBehaviorGuidanceSetsAnAdaptiveBrevityDefault pins the durable behavior rather than the paragraph byte-for-byte:

1. the concise, direct, and friendly default;
2. the 10 lines ceiling that makes it actionable;
3. a relax exception, so the ceiling is adaptive rather than absolute;
4. absence of a second, tighter cap (4 lines, four lines, one-word, single word) — the OpenCode shape this rejects;
5. survival of the file path rule.

Red-verified on current main: fails on the missing concise, direct, and friendly and 10 lines strings and on the absent relax-exception. TestBaseBehaviorGuidanceStaysUnderBudget and TestSystemPromptCarriesBaseBehaviorGuidance still pass unchanged.

go test -race ./cmd/harness/ then go test -race ./... green; go build, go vet, gofmt clean.

Live A/B (run from a credentialed box)

I could not run this myself — only a dummy credential (CODEX_DUMMY_KEY) is present on my box. It was run separately on a box where gatekeeper injects the real ChatGPT credential: main and this head built as separate clean binaries, arms alternated to reduce ordering bias, same codex/gpt-5.6-sol, fresh no-save session per call, -no-instructions, identical prompt within each workload. Metrics from each binary's own turn_metrics; visible counts from stdout.

Workload	n/arm	Measure	main	candidate	change
Already-compact handoff	6	output tokens, median	99	95	-4%
		visible lines, median	8	7	-13%
		stream ms, median	3,449	3,390	-2%
Complex investigation handoff	4	output tokens, median	608	514	-15%
		visible characters, median	2,495	1,865	-25%
		visible lines, median	45	32	-29%
		stream ms, median	18,123	13,169	-27%

Two results matter more than the deltas:

• It barely touches an already-concise answer (-4% tokens), and cuts a complex handoff by ~25% visible size and ~27% stream time. That asymmetry is the adaptive ceiling behaving as designed rather than a flat cap.
• Nothing material was suppressed. Every candidate output kept the diagnosis, measurements, fix, verification, and all three incomplete follow-ups, and it exceeded 10 lines on the complex prompt because the relax exception applied. That is the direct answer to the OpenCode-brevity risk this PR cites.

TTFT was unchanged/noisy (complex median 6.61s → 6.14s), consistent with this targeting output generation rather than prefill.

Small samples, and not a quality benchmark. Fleet output_tokens/stream_ms after rollout remains the real evaluation — and since #271 merged, those rows carry service_tier and effort, so the comparison can hold tier constant.

Prose audit

3 added // lines total — the single named-failure comment on the new test. No new production comments, no incident chronology in code; the measurement lives in this body and the commit message.

[Copilot]

🟢 Approval recommended

The prompt change is narrowly scoped, internally consistent with the existing guidance block, and is backed by a focused regression test.

Pull request overview

This PR updates harness’s base system prompt behavior guidance to make “brevity” actionable (defaulting to a 10-line ceiling with explicit exceptions), aiming to reduce user-visible output length and streaming time while preserving correctness and necessary detail.

Changes:

• Replaced the prior “keep your final message short” instruction with a more specific “concise, direct, and friendly” paragraph, including a default 10-line limit and a relax/exception clause.
• Added a targeted test to pin the adaptive brevity default (including the absence of tighter OpenCode-style caps) and retention of the “reference a file path” rule.

File summaries

File	Description
cmd/harness/main.go	Updates baseBehaviorGuidance() to include an actionable adaptive brevity instruction and “do not repeat tool output.”
cmd/harness/main_test.go	Adds TestBaseBehaviorGuidanceSetsAnAdaptiveBrevityDefault to lock in the intended brevity defaults and guard against regressing to vague or overly strict caps.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[andybons]

Live Codex A/B completed from a credentialed box; the report’s “dummy credential means no probe” limitation does not apply here because gatekeeper injects the real ChatGPT credential.

Method: built current main and PR head e8b3881 as separate clean binaries, then alternated arms to reduce ordering/traffic bias. Same codex/gpt-5.6-sol, fresh no-save session per call, -no-instructions, identical prompt within each workload. Metrics came from each binary’s emitted turn_metrics; visible line and character counts came from stdout.

Workload	n/arm	Measure	main	candidate	change
Already-compact handoff	6	output tokens, median	99	95	-4%
		visible lines, median	8	7	-13%
		stream ms, median	3,449	3,390	-2%
Complex investigation handoff	4	output tokens, median	608	514	-15%
		visible characters, median	2,495	1,865	-25%
		visible lines, median	45	32	-29%
		stream ms, median	18,123	13,169	-27%

TTFT was essentially unchanged/noisy (complex median 6.61s → 6.14s), as expected: this targets output generation, not prefill. All candidate outputs preserved the diagnosis, measurements, fix, verification, and three incomplete follow-ups. They exceeded 10 lines on the complex prompt because the adaptive exception applied; the rule reduced prose without forcing material detail out.

Conclusion: directional evidence supports the change. It does almost nothing to an answer already concise, while reducing a complex handoff’s median visible size by ~25% and stream time by ~27%. Samples are small and not a quality benchmark, so fleet output_tokens/stream_ms after rollout remains the real evaluation.