Close max_audit.md in full — M7, the LOW table, and the roadmap - #24
Merged
Conversation
Closes audit M7 -- the last open finding in max_audit.md. All three of M7's consequences were live when measured. They are not the same defect, and the re-serialization half is the one that mattered. When an elision fired, the proxy rebuilt the request with JSON.stringify, which rewrote fields it had never touched. Measured on a hand-written payload with one elision -- the only shape the Gateway saves on: "temperature": 1.0 -> "temperature":1 "top_p": 1e3 -> "top_p":1000 "seed": 12345678901234567890 -> "seed":12345678901234567000 The first two are cosmetic. The third is a different number: an integer past 2^53 does not survive JSON.parse -> JSON.stringify, so a provider was being asked for a seed the caller never chose, by a proxy whose whole promise is faithfulness. This is the mechanism the project already identified as the phantom -1.39% in the Python harness (Issue 5), reproduced in production code. Elided content is now spliced into the caller's own bytes: each message's content is located by the canonical JSON encoding of the parsed text, searched forward from the previous message's end, and only that span is replaced. The forward cursor is the design, not an optimization. A first version searched globally and refused ambiguous matches, which declined every payload the Gateway can save on -- session-dedup preserves the first copy of a block and elides the later ones, so the encoded text appears more than once by construction. Its tests passed, because with the splice declining nothing changed and every assertion held. Where the caller's escaping differs from ours, the text is equal after parsing but absent from the raw bytes; there the splice declines and the original body goes out. Invariant 3's direction: a lost saving costs tokens, a corrupted field costs correctness. The same rule refuses any spliced body that is not smaller than the one that arrived, which is M7's third consequence and had never been asserted. The metrics half was mild. Reported against measured: 48.5% claimed, 47.1% on the wire -- directionally right, about 1.4pp optimistic, the gap being JSON structural overhead the render never sees and the provider always bills. Pointed at the forwarded body it now reads 46.3% against 46.5%. Still counted in tokens through estimateBundleTokens: a saving in bytes compared against a budget in tokens is the two-estimator defect §19 exists to prevent. The new test file was green before it was right -- 4520 bytes in, 4520 out, tokensSaved 0, because session-dedup needs a previous turn to have registered the block hash and a single-turn fixture elides nothing. Each test now asserts the elision fired before asserting anything about it. Tenth instance of this project's oldest failure, second in two sessions, and both were in new tests written to prove a fix. DECISIONS §54. 662 tests, typecheck, lint and build green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
max_audit.md ends with a nine-row LOW table. The wave tables account for every
CRITICAL, HIGH and MEDIUM finding, and L1, L4, L5, L6, L7, L8 and L9 appear in
no wave, no decision and no row of the status doc. Verified against source: all
seven open. L2 and L3 were closed incidentally by the C2 Buffer work, which is
the tell -- the two that got fixed are the two that happened to sit inside
someone else's diff.
This is DECISIONS §54's failure mode three days later. §54 closed M7 and named
the rule -- an item that is in no table reads as done, exactly like a check that
never ran reads as a pass -- and naming it did not prevent the recurrence,
because the check being run was still "what did we work on?" rather than "what
does the document list?". Close a document against its own list of findings.
L7 was rated too low, and that is the interesting part. The audit calls it
"fails safe (skip) but silently loses the region", which reads like the cost is
one region. Measured end-to-end it is the whole file, because a one-region file
has nothing else to elide:
normal.py 434 bytes -> 96 bytes (77.9%)
blank_first.py 436 bytes -> 436 bytes (0%, fallback)
scanPythonDefBodies read the body indent off def+1 without asking whether that
line was blank. indentOf('') is 0, the region began at column 0, the marker
inherited column 0, PythonValidator reported AST_INDENTATION_ERROR, and
elideRegions skipped it as post_condition_rejected. The `last` scan in the same
function already skipped blanks; only the line reading the indent did not, so
the two disagreed about where a body starts.
The corpus cannot see this fix, and that is a fact about the corpus. Per-row
over the frozen 288-file corpus, 576 of 576 rows are byte-identical across all
fifteen compared fields -- because 0 of 45 Python corpus files have a blank line
in that position. pip internals and this repository's own Python are uniformly
PEP 8 there. Do not read 576/576 as "inert"; ask first whether the corpus
contains the shape. That is §52's corpus-bias caveat with the sign reversed.
L1 is §30 arriving by the other door. TOKENDAMPER_PLANNER_MODE=session_dedup was
silently discarded while --planner-mode session_dedup threw, and session_dedup
is a real member of OptimizationMode, so a user had every reason to think it
took effect. All four enum parsers now reject through one helper rather than one
being fixed and three keeping the trap. Accepted sets are unchanged; widening
defaultMode past pass_through is left as the separate question it is.
L4, L5 and L9 are recorded at their sites rather than changed. L4's premise does
not hold: it says the hash stops being a hash of item.content, implying it was
one, and on the route that reaches that stage it never was -- createContextBundle
hashes a provenance object and sets id to it. The narrower defect (the value is
chained) is real and unreachable, since the one consumer treating it as a content
identity is session-dedup, which runs only under a planner mode where
constraint-preservation is not planned. Changing it moves bundle.contentHash and
every pinned id in the suite while moving no output byte. L5's minimum is correct
for a safety gate and the doc comment was what was wrong. L9's 12-hex prefix is
ample for provenance and birthday-bound for identity, and is not widened because
TokenHasher.resolve already treats an ambiguous prefix as unresolvable.
Also removes three dead Gateway metric fields. GatewayOptimizationOutcome still
computed rawTokens/optimizedTokens/tokensSaved from summary.tokenEstimate after
wireTokenMetrics replaced them at both call sites (§54). The two disagree by
design -- 48.5% against 47.1% on the same payload -- and a field that looks
authoritative next to the one that replaced it is how a closed finding comes back.
Every fixed case was run against the unfixed engine first: 5 of 7 assertions fail
there, and the 2 that pass are the negative controls, which must pass both ways
or they are testing nothing.
max_audit.md is now closed in full. DECISIONS §55, status doc §8.
669 tests, typecheck, lint and build green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The document still ran on v1.4.0 as its baseline and carried M7 as open in three places -- the v1.1.x follow-on list, the v2.0.0 gate notice and the Version Summary row -- after §54 closed it. v1.5.0 had shipped with no section of its own, though v1.3.0 and v1.4.0 both have one. - v1.5.0 gets its section and becomes the baseline; v1.4.0 moves to prior release. - An unnumbered row records §54 and §55 as merged and unreleased. No number is reserved for it: §53's rule is that a number is a fact about what shipped, assigned at ship time, after four reservations in four releases turned out wrong. - The three M7-open notices are retired. The v2.0.0 sequencing constraint is split rather than deleted, because only half of it is satisfied: metrics now measure the bytes forwarded, so a /metrics endpoint would export a number that means what it says, but the premise still stands -- a Prometheus endpoint on a pass-through that saves nothing cross-turn instruments nothing. What is worth exporting today is within-payload dedup and the fallback rate, both real. - The gate notice records that "every finding is closed" was written there twice before it was true, and why both misses look identical from the inside. - "What to do instead" is down to one live item. Sub-region elision shipped (§50), per-item drift closed unbuilt (§51) after its precondition expired, and M7 is done (§54). Widening elision beyond TypeScript/JavaScript/Python is what remains and is the largest measured gain available -- every other corpus bucket is 0.00%. The entry now carries the warning that it is not one gate despite supportsRegionElision being one function: a new language needs a validator and extractSymbols coverage as well as a region scanner, and adding the scanner alone converts a 0% into a fallback via §33's measurement gate. - The 2026-08-07 measured-position table is marked historical. It read as "the numbers any of the above will be judged against" while every figure in it has since moved; the live baseline is the status doc §2, re-measured per release. No feature work started. The gate is open and the repository is ready for it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes
max_audit.md. Supersedes #23, which was closed as a side effect of renaming this branch — it covered only the first of these three commits.219b029d006cf7a80e32dThe audit was not closed, twice
max_audit.mdwas declared closed while M7 was open, then declared closed again while the entire nine-row LOW table was open. Both were missed the same way: they sat in no wave table, and a check that enumerates the work that was done cannot see a finding nobody scheduled.§54 named the rule after the first miss — an item that is in no table reads as done, exactly like a check that never ran reads as a pass — and it recurred three days later, which says the rule was not the fix. The fix is procedural and is now the first line of status doc §8: close a document against its own list of findings, not against the list of work that was done.
L2 and L3 were already closed, incidentally, by the C2
Bufferwork. That is the tell — the two that got fixed are the two that happened to sit inside someone else's diff.M7 (
219b029)When an elision fired, the proxy rebuilt the request with
JSON.stringify, rewriting fields it had never touched:"temperature": 1.0"temperature":1"top_p": 1e3"top_p":1000"seed": 12345678901234567890"seed":12345678901234567000The first two are cosmetic. The third is a different number — an integer past 2^53 does not survive
JSON.parse→JSON.stringify— so a provider was asked for a seed the caller never chose, by a proxy whose whole promise is faithfulness. Elided content is now spliced into the caller's own bytes; where the caller's escaping differs from ours the splice declines and the original body goes out, because a lost saving costs tokens and a corrupted field costs correctness. A forwarded body can no longer be larger than the one that arrived (M7's third consequence, never previously asserted).Savings now measure the bytes forwarded rather than the bundle render: 48.5% claimed against 47.1% actual, now 46.3% against 46.5%.
The LOW table (
d006cf7)Fixed: L1 (
TOKENDAMPER_*enum values rejected rather than dropped), L7, L8 (escaped newlines counted). Recorded at their sites as acceptable rather than correct: L4, L5, L9. Comment corrected: L6 named a branch-and-bound search the code does not perform.L7 was rated too low
The audit calls it "fails safe (skip) but silently loses the region". Measured end-to-end it loses the whole file, because a one-region file has nothing else to elide:
defdefindentOf('')is 0, so the region began at column 0, the marker inherited column 0, andPythonValidatorrejected it. Thelastscan in the same function already skipped blank lines; only the line reading the indent did not.L1 is §30 arriving by the other door
TOKENDAMPER_PLANNER_MODE=session_dedupwas silently discarded while--planner-mode session_dedupthrew — andsession_dedupis a real member ofOptimizationMode, so a user had every reason to think it applied. All four enum parsers now reject through one helper rather than one being fixed and three keeping the trap.Important
Technically breaking, for anyone with a typo'd
TOKENDAMPER_*variable set — it used to be ignored and now errors. Accepted sets are unchanged.Measurement
576 of 576 corpus rows byte-identical across all fifteen compared fields, per-row over one frozen 288-file corpus, both routes.
Warning
That does not mean L7 is inert. 0 of 45 Python corpus files have a blank line after a
def— pip internals and this repo's own Python are uniformly PEP 8 there. The instrument is blind to this shape. This is §52's corpus-bias caveat with the sign reversed, and reading 576/576 as "no effect" is the mistake it exists to prevent.Every fixed case was run against the unfixed engine first: 5 of 7 assertions fail there, and the 2 that pass are the negative controls, which must pass both ways or they are testing nothing.
669 tests, typecheck, lint and build green.What this unblocks
The feature gate is open and the roadmap is current. Widening elision beyond TypeScript/JavaScript/Python is the one live item — every other corpus bucket is 0.00%. It is not one gate despite
supportsRegionElisionbeing one function: a new language needs a validator andextractSymbolscoverage as well as a region scanner, and adding the scanner alone converts a 0% into a fallback via §33's measurement gate.No feature work is started here.
🤖 Generated with Claude Code