Skip to content

fix(claude): honor authoritative context windows in generated profiles - #854

Closed
park285 wants to merge 6 commits into
lidge-jun:devfrom
park285:agent/fix-claude-subagent-context-marking
Closed

fix(claude): honor authoritative context windows in generated profiles#854
park285 wants to merge 6 commits into
lidge-jun:devfrom
park285:agent/fix-claude-subagent-context-marking

Conversation

@park285

@park285 park285 commented Aug 1, 2026

Copy link
Copy Markdown

Summary

  • restrict generated Claude roster and self-profile [1m] markers to authoritative effective context windows of at least 1M
  • align Claude Sonnet 4.6, Opus 4.6, and Opus 4.7 registry and generated metadata with their 1M source values
  • preserve genuine routed [1m] model IDs while honoring provider caps and case-insensitive marker spelling
  • preserve an existing [1m] selector when context metadata is missing or non-positive, stripping it only when a positive authoritative window proves sub-1M

Problem

Generated agent profiles reused the main-session auto-context predicate, so a 372K native route could be written with [1m]. The Anthropic registry and generated jawcode metadata also disagreed about Claude 4.6/4.7 windows, allowing valid 1M selectors to lose their marker during catalog-to-profile generation.

The marker is also overloaded: it can be Claude's context-accounting suffix or part of a real routed selector such as Kimi k3[1m]. Unconditional stripping loses the real selector, while unconditional preservation can ignore an effective provider context cap.

Missing metadata is not evidence of a sub-1M context. Catalog-derived selectors that already carry [1m] must therefore keep it unless a positive effective window explicitly proves otherwise.

Fix

  • remove the Sonnet 4.6 generator-only 200K override and regenerate the owned metadata artifact
  • add the missing 1M Anthropic registry windows and require every configured Anthropic model to have a positive window
  • derive Claude 4.6/4.7 generated profiles through fetchProviderModels() and buildClaudeContextWindows() in regression coverage
  • make exact selector metadata authoritative before falling back to the stripped base selector
  • canonicalize only the trailing [1m] marker for exact lookup, preserving the rest of the model ID and honoring raw exact-key precedence
  • distinguish absent or non-positive metadata from a positive sub-1M window, preserving existing markers in the former case
  • cover incomplete catalog-derived Claude 4.6/4.7 metadata through both roster and self-clone generation paths, plus explicit 200K marker removal
  • keep main-session auto-context behavior unchanged

This absorbs the shared source-of-truth reconciliation requested on #839.

Verification

  • JAWCODE_MODELS_JSON=/tmp/opencodex-jawcode-854/opencodex-snapshot.json npx --yes bun@1.3.14 test --isolate tests/claude-agents-inject.test.ts tests/claude-context-windows.test.ts tests/codex-catalog.test.ts tests/provider-registry-parity.test.ts tests/jawcode-metadata-sync.test.ts — 178 passed, 0 failed
  • npx --yes bun@1.3.14 x tsc --noEmit — passed
  • JAWCODE_MODELS_JSON=/tmp/opencodex-jawcode-854/opencodex-snapshot.json npx --yes bun@1.3.14 scripts/test.ts — 6,963 passed, 9 skipped, 0 failed
  • git diff --check origin/dev — passed for the PR-relative diff

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed. (No user-facing configuration or documentation contract changed.)
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (No security boundary changed.)

Summary by CodeRabbit

  • Bug Fixes
    • Improved context-window labeling for Claude models.
    • Ensured only models with authoritative 1M-token support display the [1m] marker.
    • Prevented incorrect markers from appearing on native roster and self-referential models while preserving valid routing labels.
    • Normalized marker formatting and respected configured context limits, including exact-capacity models.
    • Improved handling of existing labels when model context information is unavailable.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Claude model metadata now uses registry context windows. Claude roster and self-clone definitions normalize [1m] markers based on context metadata and auto-context settings. Tests cover routed selectors, caps, normalization, and incomplete metadata.

Changes

Subagent context marker handling

Layer / File(s) Summary
Context-window metadata source
scripts/generate-jawcode-metadata.ts
Generated model rows now use each model’s contextWindow value directly.
Context marker resolution
src/claude/agents-inject.ts, tests/claude-agents-inject.test.ts
withSubagentContextMarker resolves marked, canonical, and bare selectors. Roster and self-clone models apply or remove [1m] according to context metadata. Tests cover catalog-derived markers, routed selectors, context caps, exact-cap precedence, case normalization, and incomplete metadata.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

Suggested reviewers: ingwannu, lidge-jun, wibias

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: honoring authoritative context windows in generated Claude profiles.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the bug Something isn't working label Aug 1, 2026
@park285
park285 marked this pull request as ready for review August 1, 2026 11:13
@lidge-jun

Copy link
Copy Markdown
Owner

Pre-merge review result: not merge-ready yet. The narrow GPT fix direction is correct (gpt-5.6-sol is authoritatively 372k while the synthetic Cursor route is 1M), but the Anthropic side of the metadata contradicts upstream truth:

  1. scripts/generate-jawcode-metadata.ts:30 forces both Sonnet 4.6 variants to 200_000, and tests/codex-catalog.test.ts:2216 locks that value — but Anthropic documents Sonnet 4.6 (and Opus 4.7) as 1M-context models. The source-of-truth values need to move together, not just the two literals in this PR. (This is the same reconciliation fix(providers): give Claude 4.6/4.7 their 1M context windows #839 needs; consider coordinating.)
  2. The static Anthropic registry (src/providers/registry.ts:216) lists Sonnet 4.6 and Opus 4.7 but omits their context windows, and the PR's [1m] strip-then-consult flow in src/claude/agents-inject.ts:62-64 can regenerate a valid saved 1M selector without its marker when the map is incomplete.
  3. The regression test pins the synthetic correction but never passes real catalog-derived Claude 4.6/4.7 metadata through buildClaudeContextWindows, so the inconsistency stays false-green. Please add an end-to-end generated-profile regression.
  4. Full cross-platform CI never ran on this head and there is no maintainer approval yet — both are required by merge policy.

Happy to re-review once the metadata source of truth is reconciled and CI runs.

park285 added 2 commits August 2, 2026 14:32
- align Anthropic registry and generated catalog windows with the 1M source values
- preserve exact routed selectors while honoring provider context caps
- normalize marker case and cover roster, self, catalog, and registry boundaries
@park285

park285 commented Aug 2, 2026

Copy link
Copy Markdown
Author

Addressed the pre-merge findings in 2f5a20be:

  1. Removed the Sonnet 4.6 200K generator override and regenerated both base and [1m] metadata at 1M.
  2. Added 1M registry windows for Sonnet 4.6, Opus 4.6, and Opus 4.7, plus a completeness guard for the configured Anthropic roster.
  3. Added catalog-derived generated-profile coverage through fetchProviderModels()buildClaudeContextWindows()buildClaudeAgentDefs().
  4. Added exact-selector, provider-cap, genuine routed [1m], and case-insensitive marker boundary coverage for roster and self profiles.

The branch now includes the latest dev, and the PR-relative diff remains limited to seven files. Local verification on this head: 177 focused tests passed, typecheck passed, generated metadata byte-sync passed, and the full suite completed with 6,913 passed / 9 skipped / 0 failed.

This PR now absorbs the overlapping source-of-truth work requested on #839. Please re-review this head; hosted cross-platform CI and maintainer approval remain the external merge gates.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/claude/agents-inject.ts`:
- Around line 62-71: Update withSubagentContextMarker to preserve an existing
[1m] marker when no positive matching window metadata exists; only strip the
marker when an authoritative positive window indicates a sub-1M context, while
retaining current auto-marking behavior for eligible windows. Add an end-to-end
regression using incomplete catalog-derived Claude 4.6 and 4.7 metadata that
covers both roster and self-clone generation paths.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 01e943b1-2719-4c8c-831f-db41e09ad3e0

📥 Commits

Reviewing files that changed from the base of the PR and between 04459a6 and 2f5a20b.

📒 Files selected for processing (2)
  • scripts/generate-jawcode-metadata.ts
  • src/claude/agents-inject.ts

Comment thread src/claude/agents-inject.ts Outdated
@park285

park285 commented Aug 2, 2026

Copy link
Copy Markdown
Author

Follow-up on the CodeRabbit finding:

  • withSubagentContextMarker() now treats only a positive context window as authoritative. An existing [1m] marker is preserved when exact/canonical/base metadata is absent or non-positive, and is stripped only when a positive window proves sub-1M.
  • The regression derives incomplete Claude 4.6/4.7 catalog rows through buildClaudeContextWindows() and verifies both roster and self-clone generation, plus explicit 200K marker removal.
  • The branch now includes dev at f9b9440c and remains limited to the same seven PR-relative files.

Verification on head 463749e6:

  • focused suite: 178 passed, 0 failed
  • typecheck: passed
  • generated metadata byte-sync: passed
  • full suite: 6,963 passed, 9 skipped, 0 failed
  • git diff --check origin/dev: passed

@lidge-jun

Copy link
Copy Markdown
Owner

Superseded by #889, which ports this PR's withSubagentContextMarker design and its full regression set (the modified 372k roster test plus the five additions: catalog-derived markers, routed [1m] preservation, 350K cap unmarking, marker-case precedence, incomplete-metadata preservation) together with #839's registry map, the generated metadata rows, and the codex-catalog/provider-registry-parity test updates. Consolidated as one fix crediting both PRs. Thanks — the port survived an adversarial audit unchanged.

@lidge-jun lidge-jun closed this Aug 2, 2026
@park285

park285 commented Aug 2, 2026

Copy link
Copy Markdown
Author

특붕이 잘쓰고있어요 유지보수 감사합니다.

@park285
park285 deleted the agent/fix-claude-subagent-context-marking branch August 2, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants