fix(gateway): forward the caller's bytes, and measure them

[ojassug]

Closes audit M7 — the last open finding in max_audit.md.

M7 said Gateway savings are computed from summary.tokenEstimate, a property of the bundle render, while what leaves the process is JSON.stringify({...parsedPayload, messages}). It listed three consequences. All three were live when measured, and — as docs/audit-remediation-status.md §6 warned — they are not the same defect. The re-serialization half is the one that mattered.

What re-serialization was doing

Measured on a hand-written payload with one elision firing, which is the only shape the Gateway saves anything on:

client sent	provider received
"temperature": 1.0	"temperature":1
"top_p": 1e3	"top_p":1000
"seed": 12345678901234567890	"seed":12345678901234567000

The first two are cosmetic. The third is a different number. An integer past 2^53 does not survive JSON.parse → JSON.stringify, so a provider was being asked for a seed the caller never chose — by a proxy whose entire promise is faithfulness. Duplicate keys collapsed the same way, and pretty-printing was lost.

This is the mechanism the project already identified as the phantom -1.39% in the Python benchmark harness (Issue 5), reproduced in production code. It was found there by measuring, and found here the same way.

The fix is a splice, not a smaller re-serialization

Elided content is written into the caller's own bytes: each message's content is located by the canonical JSON encoding of the text the parser produced, searched forward from the previous message's end, and only that span is replaced. Every other byte is the caller's.

The forward cursor is the design, not an optimization. A first version searched globally and refused ambiguous matches — which declined every payload the Gateway can save on, because cleanup:session-dedup preserves the first copy of a block and elides the later ones, so the encoded text appears more than once by construction. Walking every message in order, replaced or not, keeps position and identity in agreement.

Where the caller's escaping differs from ours the text is equal after parsing but absent from the raw bytes. There the splice declines and the original body is forwarded: invariant 3's direction, since a lost saving costs tokens and a corrupted field costs correctness, and only one of those is recoverable by the caller. The same rule refuses any spliced body that is not smaller than the one that arrived — M7's third consequence, which nothing had ever asserted.

The metrics half was mild

Reported against measured on that payload: 48.5% claimed, 47.1% on the wire — directionally right, ~1.4pp optimistic, the gap being JSON structural overhead the render never sees and the provider always bills. Pointed at the forwarded body it now reads 46.3% against 46.5%.

Still counted in tokens, still through estimateBundleTokens. A saving denominated in bytes compared against a budget denominated in tokens is precisely the two-estimator defect DECISIONS §19 exists to prevent; what changed is the artefact measured, not the unit.

Reviewer note: the test file was green before it was right

test/integration/gateway-wire-metrics.test.ts passed on its first run — 4520 bytes in, 4520 out, tokensSaved: 0. session-dedup elides a block only once a previous turn has registered its hash, so a single-turn fixture elides nothing and every assertion holds vacuously. Each test now asserts the elision fired before asserting anything about it, and the file documents why.

That is the tenth instance of this project's oldest failure and the second in two sessions. Both recent ones were in new tests written to prove a fix — the moment the temptation to believe a pass is highest. Worth a look during review: the guard is expect(forwarded).not.toBe(rawBody) ahead of the substantive assertions.

Verification

• 662 tests, typecheck, lint and build green.
• Existing Gateway tests unchanged and still passing — including gateway-dedup-reality (the within-payload saving still works) and gateway-byte-fidelity.
• End-to-end through the built dist/: all three numeric literals preserved exactly, 4582 → 2451 bytes, reported 46.3% against 46.5% actual.

Scope

max_audit.md is now closed in full. docs/audit-remediation-status.md §1, §6 and §7 are updated, and §6 keeps the record of how the item went missing — gated behind "only if question B keeps the Gateway", B answered in §41, nothing carrying it across — because the way an item disappears matters more than the item.

Not released. This changes forwarded bytes and reported metrics, so it is a minor when it ships.

🤖 Generated with Claude Code