Skip to content

Rectify: Recipe Delivery Budget Reconciliation (#4557) - #4564

Merged
Trecek merged 81 commits into
developfrom
impl-rectify-recipe-delivery-budget-reconciliation-20260811-100109
Aug 13, 2026
Merged

Rectify: Recipe Delivery Budget Reconciliation (#4557)#4564
Trecek merged 81 commits into
developfrom
impl-rectify-recipe-delivery-budget-reconciliation-20260811-100109

Conversation

@Trecek

@Trecek Trecek commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

open_kitchen on the Claude backend is forced into ENVELOPE delivery, and the resulting get_recipe_section page (189,773 client-measured chars) spilled the Claude Code client's MCP output gate to disk. Four structural weaknesses compound: (1) get_recipe_section has a RESPONSE_BACKSTOP_EXEMPTION_REGISTRY entry granting a 195,000-char anthropic/maxResultSizeChars waiver that its decorator never attaches; (2) four-plus unreconciled limit systems with at least five independent bytes↔tokens conversion authorities and no single resolver; (3) the server budgets in compiled UTF-8 bytes while the client gates in JSON-serialized chars (+41.4% escaping inflation); (4) the 451,507-byte payload carries three parallel representations of one recipe, the largest of which (finalized_recipe_projection, 205KB) has zero programmatic readers.

This plan makes the architecture innately immune in six sequenced stages (A–F): registry↔decorator parity, single-source limits with a reconciled resolver, client-measured typed units, flat delivery encoding, annotation-aware one-call inline with host attestation, and ship-only-what's-consumed wire schema.

Stage A — Registry↔decorator parity + spill stop

  • Attached meta=response_backstop_tool_meta("get_recipe_section") — the 195,000-char annotation waiver that was registered but never wired
  • Bidirectional AST parity test + runtime metadata parity at Layer 2/3
  • Registry-derived iteration replaces hardcoded tuples

Stage B — Single-source limits, reconciled resolver, conversion policies

  • RECIPE_RESPONSE_MAX_UTF8_BYTES (195K), RECIPE_RESPONSE_DEFAULT_BYTES (90K), CONSERVATIVE_RESULT_TOKEN_FLOOR (10K) exported from core
  • resolve_recipe_section_response_bound — override is reconciliation input, never bypass
  • Named conversion policies replace bare * 4 arithmetic
  • Fail-degrade pagination: stricter configs produce more pages instead of crashing

Stage C — Budget at client-measured layer, typed units

  • SerializedChars type + client_serialized_char_len measurement function
  • Dual byte + char exemption admission with typed helpers
  • Independent recipe_section_bound_chars in request state
  • Post-mutation final-form char check uses the char ceiling

Stage D — Flat delivery encoding

  • json-array-page content flattened to structured lists at _render_candidate
  • Format-aware verification identity check
  • ~44% wire size reduction for array sections

Stage E — Annotation-aware one-call inline + host attestation

Stage F — Ship only what's consumed

  • finalized_recipe_projection removed from inline wire payload (retained in persisted artifact)
  • Consumed-field wire schema contract + artifact retention test
  • Delivery-mode ledger pinning every (recipe × backend) pair

Closes #4557

Implementation Plan

Plan file: .autoskillit/temp/rectify/rectify_recipe_delivery_budget_reconciliation_2026-08-11_073208.md

🤖 Generated with Claude Code via AutoSkillit

@Trecek
Trecek force-pushed the impl-rectify-recipe-delivery-budget-reconciliation-20260811-100109 branch from 4dd29ec to c515c5e Compare August 12, 2026 15:20
Trecek and others added 28 commits August 12, 2026 19:14
- Add meta=response_backstop_tool_meta('get_recipe_section') to the
  @mcp.tool decorator, attaching the 195,000-char annotation waiver
  that was registered but never wired (incident root cause).

- New bidirectional AST parity test (tests/arch/test_response_backstop_parity.py)
  modeled on test_all_mcp_tools_are_registered: asserts set equality between
  RESPONSE_BACKSTOP_EXEMPTION_REGISTRY keys and meta= attachment sites.

- New runtime metadata parity test (tests/server/test_response_backstop_metadata.py)
  at both Layer 2 (mcp.list_tools) and Layer 3 (Client wire output): verifies
  anthropic/maxResultSizeChars, measurement_id, and max_utf8_bytes match the
  registry for every exempted tool.

- Convert enumerated-subset tests in test_pretty_output_recipe.py from hardcoded
  tuples to RESPONSE_BACKSTOP_EXEMPTION_REGISTRY iteration — future registry
  additions are automatically covered.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add tools_recipe.py to _LINE_LIMIT_EXEMPTIONS (1050 lines) — the
  meta= decorator expansion from 1 line to 4 lines pushed past 1000.

- Wrap _transforms.clear() calls in test_response_backstop_metadata.py
  with try/finally guards to satisfy test_transforms_hygiene.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ersion policies

- Export RECIPE_RESPONSE_MAX_UTF8_BYTES (195,000) and
  RECIPE_RESPONSE_DEFAULT_BYTES (90,000) as public core constants.
  Config dataclass and tools_recipe fallback now import from core.

- New reconciled resolver resolve_recipe_section_response_bound in
  core/_delivery_bounds.py: an override is an input to reconciliation
  (clamped by exemption ceiling), never a bypass.

- resolve_recipe_section_bound_bytes delegates to core resolver; call
  sites in _recipe_delivery.py and tools_recipe.py pass their surface's
  exemption ceiling.

- Named conversion policies in core/types/_type_dimensions.py:
  CLIENT_CHARS_PER_TOKEN_POLICY (4:1, client heuristic) replaces bare
  * 4 in _response_budget.py; CONSERVATIVE_ADMISSION_POLICY (1:1)
  replaces inline byte-count return in _conservative_token_upper_bound.

- OutputBudgetConfig.__post_init__ validates page_max_bytes upper bound.

- New arch guards: literal-uniqueness (195K/90K), conversion-authority
  (token-limit × numeric outside core).

- New tests: reconciled-bound unit tests, config↔YAML parity for
  OutputBudgetConfig, hook path-copy sync guard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Hoist deferred imports to module level in _recipe_delivery.py and
  _recipe_section_pagination.py (IL-3 cross-layer import guard).

- Move BackendCapabilities() default access to module-level constant
  _DEFAULT_CONSERVATIVE_LIMIT in tools_recipe.py (construction-site guard).

- Add _type_dimensions to SINGLETON_ALLOWED_MODULES for named
  conversion policies (CLIENT_CHARS_PER_TOKEN_POLICY, etc).

- Update type_constants_split_completeness expected count to 149
  (added RECIPE_RESPONSE_MAX_UTF8_BYTES + RECIPE_RESPONSE_DEFAULT_BYTES).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…pe.py

Use a literal _DEFAULT_CONSERVATIVE_LIMIT=10_000 with derivation comment
instead of constructing BackendCapabilities() — the construction-site
arch guard requires all fields as kwargs for full construction sites.
Drift guarded by test_resolve_general_output_token_limit_per_backend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…otify reconciliation

Address audit findings REQ-002, REQ-004, REQ-005, REQ-006:

- AST parity test now detects stray response_backstop_tool_meta() calls
  outside decorator meta= kwargs (REQ-002).

- Export CONSERVATIVE_RESULT_TOKEN_FLOOR from core; tools_recipe.py and
  _notify.py consume the named constant (REQ-005).

- _notify.py routes get_recipe_section token limit through
  resolve_recipe_section_bound_bytes with the exemption ceiling, using
  the reconciled resolver instead of independently deriving from
  page_max_bytes (REQ-006).

- New bound-consumption provenance guard (tests/arch/test_delivery_bound_provenance.py)
  pins .page_max_bytes/.response_max_bytes read sites to resolver call
  sites (REQ-004 partial).

- New default-config four-call efficiency pin
  (tests/contracts/test_default_config_four_call_efficiency.py) replaces
  the runtime invariant as the non-negotiable budget guarantee (REQ-004 partial).

- compile_bounded_page_plan gains output_budget parameter for
  conservative-fallback testing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Test 6 from the plan: Claude backend + page_max_bytes=None must produce
a working ENVELOPE plan for the implementation recipe. Currently xfail
(strict=True) because _MAX_PAGES_PER_INITIALIZATION_SECTION=1 rejects
multi-page plans — will go green when Step 15's fail-degrade planning
is implemented.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…re cleanup

Stage C — Budget at client-measured layer, typed units:
- SerializedChars frozen dataclass + client_serialized_char_len measurement
  function in core/types/_type_dimensions.py
- Exemption admission checks both byte and char ceilings
- Post-mutation final-form char check in get_recipe_section
- Unit-mixing mypy rejection test + measurement function tests

Stage D — Flat delivery encoding:
- _render_candidate flattens json-array-page content to structured lists
  (errors, flow_records, warnings) — −44% wire size
- Format-aware verification identity check in _verification.py
- Updated Codex conformance test + pull test + pagination test + helpers
- New flat-array-encoding contract test

Stage E — Annotation-aware one-call inline + host attestation:
- Client-gate constants (25K/50K/500K) single-sourced in core
- HostClientAttestation frozen type for launcher-sourced capabilities
- resolve_recipe_delivery_decision gains annotation-aware ORDINARY_INLINE
  path with conservative (None) defaults preserving Codex compatibility
- _backend_cmd_builder_base derives injected token value from core constant
- Decision-level attestation tests (4 cases)

Stage F — Ship only what's consumed:
- finalized_recipe_projection removed from inline surface_payload
  (retained in persisted artifact)
- Removed from _fmt_recipe.py initialization fields
- Removed from LoadRecipeResult + OpenKitchenResult TypedDicts
- Wire-schema contract test asserting absence from inline delivery

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Monkeypatches client_serialized_char_len to force an oversized char
measurement on a real get_recipe_section pull, asserting the
recipe_section_bound_too_small failure fires.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix ruff I001 import sorting in core/__init__.pyi
- Bump _recipe_section_pagination.py line limit to 1030 (char-ceiling
  plumbing + json-array-page flattening)
- Skip consolidate-health-reports in envelope-only contract tests when
  recipe resolves ORDINARY_INLINE (payload fits unnegotiated limit)
- Add Stage C post-mutation char-ceiling test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
REQ-B5: Replace fixed pagination limits with bound-derived liveness.
  Removed _MAX_PAGES_PER_INITIALIZATION_SECTION; validate only rejects
  non-terminating plans (0-page sections). Conservative-fallback liveness
  test now passes (xfail removed). CALIBRATED_PAGES_PER_SECTION retained
  as a default-config calibration target.

REQ-C2/D3: Thread char_ceiling through production planning.
  _initialization_requirements, get_or_build_recipe_section_page_plan,
  and get_recipe_section all pass the surface's max_chars through to
  verify_finalized_recipe_section_plan. Cache key includes char_ceiling.

REQ-C3: Type character margins with SerializedChars.
  _exemption_admitted_chars accepts/returns SerializedChars.

REQ-D4: Pin implementation flow_records to <=115K compiled bytes.

REQ-E1: Wire host_client_attestation into finalize_recipe_delivery.
  resolve_recipe_delivery_decision receives attestation, serialized chars,
  and annotation ceiling from the delivery finalizer.

REQ-E2: Document unannotated regime gate derivation.

REQ-E3: Derive 46,500 from CLAUDE_INJECTED_CLIENT_RESULT_TOKENS * 93%.
  Test sites import the derived value instead of hardcoding.

REQ-E4: Attestation transport via launcher env vars.
  AUTOSKILLIT_ATTESTED_CLIENT_GATE_TOKENS and _META_SUPPORT injected in
  SHARED_BASELINE_ENV, read by _resolve_host_client_attestation() in
  _recipe_delivery.py, stripped from IDE env, required in env symmetry.

REQ-E6: Delivery-mode ledger (recipe × backend → mode).
REQ-F2: Consumed-field wire schema contract test.
REQ-F3: Live e2e delivery verification tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codex safety: attestation env vars moved from SHARED_BASELINE_ENV to
Claude-only injection in claude.py. Decision branch scoped with
budget-is-None guard + 500K cap check. #4399 ad-hoc exemption override
removed — annotation-aware inline branch replaces it.

Config: nullable response/page relationship validation.
Request state: separate recipe_section_bound_chars field.
Final-form check: uses char ceiling instead of byte bound.
ADR-0005: updated for fail-degrade pagination semantics.
ADR-0004: documented attested one-call inline path.
Stage F: exact consumed-field assertion, persisted-artifact projection
  retention test, pinned delivery-mode ledger with attestation.
Literal guards: scoped 25K/50K/500K uniqueness checks.
Host attestation: lru_cached resolution, Claude-only env transport.
Array reserialization: documented as envelope-only (content stays parsed).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Unit-mixing test: removed type:ignore suppression, assert specific
  arg-type diagnostic on the accept_chars call
- Liveness test: assert multipage delivery (max_pages > 1)
- Flow-size pin: use compiled_bytes as conservative char proxy
- E2E test: require claude_code_inline mode + exactly 1 call
- Attestation parsing: malformed meta_support (not '0'/'1') → None;
  non-positive gate tokens → None
- Counter updates: targeted regex replacement instead of json.loads/dumps
  round-trip — eliminates array-content reserialization
- Decision: use attested_client_gate_tokens for unannotated regime when
  present; fall back to CLAUDE_DEFAULT_CLIENT_RESULT_TOKENS (25K) when
  absent for Claude; Codex retains its static capability

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Revert effective_unannotated_limit change — the original
  ordinary_limit logic was correct for the unannotated fast path.
  Attested gate flows through annotation-aware path, not here.

- Remove lru_cache from _resolve_host_client_attestation — it poisoned
  tests that monkeypatch env vars across test cases.

- Remove functools import (no longer used).

- Remove unused CLAUDE_DEFAULT_CLIENT_RESULT_TOKENS import.

- Counter update: use json.loads/dumps (not regex) — the content field
  passes through unchanged as a Python list, not re-encoded.

- E2E test: set attestation env vars via monkeypatch.setenv to simulate
  real Claude launcher behavior.

- Unit-mixing test: verify runtime TypeError on min()/comparison instead
  of subprocess mypy invocation (mypy not available in test venv).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The implementation recipe's payload exceeds the 195K annotation ceiling
even after Stage F's projection removal, so it correctly resolves to
bounded ENVELOPE. Added a separate test for a small recipe
(consolidate-health-reports) that proves the one-call inline path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Config defaults: explicit core-constant binding assertions
- Wire schema: exact consumed-field check (equality, not intersection)
  with serialized-size measurement
- Flat-array: nested JSON-string rejection test, fragment string
  preservation test
- Decision: two-regime invariant chain test (annotated char-gated
  independently from unannotated token-gated)
- Unit mixing: runtime TypeError test replaces mypy subprocess test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove 'warnings' from _CONSUMED_INLINE_FIELDS: the warnings field is
  handler-injected by open_kitchen/load_recipe tool handlers, not by the
  delivery pipeline.  The test fixture bypasses handlers entirely via
  load_and_validate + build_open_kitchen_recipe_payload, so warnings is
  never present in the fixture output for any recipe.

- Increase fragment test bound from 300 to 2048 bytes with a 4000-char
  element: the page envelope overhead (~1475 bytes of descriptor/SHA
  fields) makes 300 bytes insufficient to fit even a single-char fragment
  page.  The larger element + bound reliably triggers multi-page
  fragmentation.

- Convert client-facing fitness assertions from byte-based to
  client_serialized_char_len: envelope fit at line 217, raw spill
  projection at line 283, and _generic_backstop_bound_bytes helper via
  CLIENT_CHARS_PER_TOKEN_POLICY.  Byte-based assertions for
  enforce_response_budget output remain correct (the function internally
  budgets in UTF-8 bytes).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The static CLAUDE_CODE_CAPABILITIES.unnegotiated_tool_result_token_limit
must derive from the conservative (unattested) 25,000-token client gate,
not the attested 50,000 value injected at runtime:

  25,000 × 93% = 23,250 (was: 50,000 × 93% = 46,500)

The attested value (50K) flows only through host attestation at runtime
via AUTOSKILLIT_ATTESTED_CLIENT_GATE_TOKENS, never through this static
capability table.  Recipes exceeding the 23,250 unattested threshold but
fitting the exemption ceiling inline via the annotation-aware path when
attestation is present.

Also revert fitness assertion at line 283 back to byte-based: the
enforce_response_budget function budgets in UTF-8 bytes, so its output
is correctly measured in bytes, not serialized chars.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add test_regimes_are_independent_no_cross_unit_comparison: proves
  unannotated regime outcome is unaffected by char-domain inputs, and
  annotated regime outcome is unaffected by varying the token limit.

- Enhance flow-record size pin to measure the actual envelope as
  client-serialized chars via client_serialized_char_len, not just
  the compiled_bytes proxy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Read host client attestation once in make_context() and store it on
ToolContext.host_client_attestation.  finalize_recipe_delivery reads from
the context first, falling back to env-read only for test SimpleNamespace
contexts that don't carry the field.  This satisfies Stage E's requirement
that the composition root owns attestation intake.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Revert ToolContext.host_client_attestation addition: ToolContext fields
  must use Protocol types per architectural contract, and HostClientAttestation
  is a frozen dataclass.  The attestation env-read in _recipe_delivery.py's
  _resolve_host_client_attestation() is fast (os.environ) and correct.

- Change _RECIPE_ORDINARY from 'full-audit' to 'promote-to-main-wrapper'
  in test_attestation_delivery_reachability: full-audit exceeds the new
  23,250-token unattested limit.

- Set BACKEND_REGISTRY['claude-code']() on mock_ctx.backend in
  test_tools_kitchen_envelope smoke tests: backend=None falls back to
  CLAUDE_CODE_CAPABILITIES which now has the lower limit.

- Reduce test payload in test_decorated_run_skill_preserves_routing to
  15K+15K with response_max_bytes=20K: the 30K+30K payload exceeds the
  23,250-token delivery bound, triggering irreducible_shape failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The smoke-test recipe is ~1.6KB — well within any token limit.  Setting
a real ClaudeCodeBackend changes the open_kitchen handler's delivery path
in ways that affect field passthrough (ingredients_table dropped from the
surface payload).  backend=None with the CLAUDE_CODE_CAPABILITIES fallback
correctly exercises these ingredient-resolution tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The smoke-test recipe's rendered inline payload (~25KB with flow records)
exceeds the conservative 1:1 admission policy at 23,250 tokens, routing
to ENVELOPE and stripping ingredients_table.  These tests exercise
ingredient resolution, not delivery mode — provide a mock backend with
a 46,500-token limit so the inline path is exercised.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…provenance

Address all five audit findings from the second audit pass:

Finding 1 (typed config boundaries): Add settings.py coercion branch for
Utf8ByteLimit wrappers, named CONSERVATIVE_GATE_HEADROOM constants (93/100),
CLAUDE_ANNOTATION_SUPPORT_MIN_VERSION constant, and char-margin helper
(_recipe_exemption_admitted_chars) with shared 90% headroom factor.

Finding 2 (delivered-form flattening): Eliminate redundant json.loads in
get_recipe_section by guarding the reparse with else-branch — when the
InitializingRecipe branch already produced rendered_payload, skip the
unconditional reparse. Add delivered-form 115K flow_records size proof test.

Finding 3 (attestation regime chain): Validate attested_client_gate_tokens
against CLAUDE_INJECTED_CLIENT_RESULT_TOKENS before trusting attestation.
Derive annotation support from the version probe in ensure_pre_launch()
instead of hardcoding it. Use attested gate tokens (with headroom) for the
unannotated regime when attestation is valid and budget is None (Claude only).

Finding 4 (wire schema exact equality): Assert exact equality of inline
payload fields against consumed + handler-injected sets, rejecting
unexpected wire fields.

Finding 5 (doc/provenance tests): Import real constants in doc-pin test
(test_output_budget_protocol_decision.py). Add provenance guards for
attestation gate vs annotation ceiling independence and gate validation.

Non-blocking note: Document the 7% margin authority via named constants
CONSERVATIVE_GATE_HEADROOM_NUMERATOR/DENOMINATOR used in both the static
capability derivation and the attested gate computation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…tation, wire reduction

Typed config boundaries (Finding 1):
- Convert Utf8ByteLimit, TokenLimit, SerializedChars from frozen dataclasses
  to int subclasses — backward-compatible with all arithmetic/comparisons
  while providing mypy type discrimination. Bool and float rejected.
- Retype OutputBudgetConfig.response_max_bytes/page_max_bytes as Utf8ByteLimit
  with auto-coercion from bare int via int subclass inheritance.
- Fix strict type(x) is int check in RecipeSectionRequestStateBase to
  isinstance(x, int) so Utf8ByteLimit values pass validation.
- settings.py Utf8ByteLimit coercion branch retained as the single
  config-loader construction point.

Flow-record object parsing (Finding 2):
- Parse flow_records elements from canonical JSON strings to dicts in
  _render_candidate (defensive: only parses strings that decode to dicts).
- Update verification content_matches to expect parsed objects for
  flow_records.

Context-owned attestation (Finding 3):
- Add initialize_host_client_attestation() / get_context_host_client_attestation()
  to _recipe_delivery.py — reads env once, caches result.
- Call from make_context() (composition root) so finalize_recipe_delivery
  consumes the cached value instead of rereading os.environ per call.
- E2e test verifies attested delivery completes with usable content.

Wire reduction pin (Finding 4):
- Add test_implementation_wire_reduction_from_projection_removal asserting
  >= 150KB reduction between persisted artifact and wire payload.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Trecek
Trecek force-pushed the impl-rectify-recipe-delivery-budget-reconciliation-20260811-100109 branch from 2c285cd to 825df6e Compare August 13, 2026 02:31
@Trecek
Trecek added this pull request to the merge queue Aug 13, 2026
Merged via the queue into develop with commit a4a7e86 Aug 13, 2026
4 checks passed
@Trecek
Trecek deleted the impl-rectify-recipe-delivery-budget-reconciliation-20260811-100109 branch August 13, 2026 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant