Part of #919. Phase 0 of the provider modernization roadmap. LP-004.
Prerequisite: #922 (LP-003) — the thinking budget must fit under whatever max_tokens becomes configurable here.
Summary
Add max_tokens and provider beta feature flags to the tier binding, and plumb them through the resolver and router so frontier-model capabilities are reachable from config.
Problem
Output is pinned at 4096 with no way to raise it. The Anthropic client defaults to 4096 when unset:
// internal/llm/provider.go:373
if config.MaxTokens <= 0 {
config.MaxTokens = 4096
}
and nothing ever sets it. LLMTierBinding (internal/config/types.go:141) and ResolvedLLMTier (internal/config/llm_resolve.go:22) have no max_tokens field, and helpers_llm_router.go:34 builds ProviderOptions without one — so MaxTokens is structurally always 0. Every tier is capped at 4096 output tokens regardless of model.
This also collides with LP-003: a thinking budget must be strictly below max_tokens, so any meaningful budget is unreachable while the cap is 4096.
Beta features are unreachable. anthropic-beta is one hardcoded constant:
// internal/llm/anthropic.go:16
const anthropicPromptCachingBeta = "prompt-caching-2024-07-31"
sent unconditionally at anthropic.go:67. Prompt caching went GA long ago, so this occupies the only slot with a header that no longer does anything — while 1M context, interleaved thinking, and extended cache TTL all need that slot.
It is also sent to third-party Anthropic-compatible gateways (config/default.yaml:21 points minimax at one), which may reject or mishandle an unknown beta.
Scope
Acceptance Criteria
Validation
Out of Scope
- Context-window budgeting — that is LP-006, which lands on the same tier binding and should be sequenced after this.
- Per-role output limits. Tier granularity is enough for now.
Part of #919. Phase 0 of the provider modernization roadmap. LP-004.
Prerequisite: #922 (LP-003) — the thinking budget must fit under whatever
max_tokensbecomes configurable here.Summary
Add
max_tokensand provider beta feature flags to the tier binding, and plumb them through the resolver and router so frontier-model capabilities are reachable from config.Problem
Output is pinned at 4096 with no way to raise it. The Anthropic client defaults to 4096 when unset:
and nothing ever sets it.
LLMTierBinding(internal/config/types.go:141) andResolvedLLMTier(internal/config/llm_resolve.go:22) have nomax_tokensfield, andhelpers_llm_router.go:34buildsProviderOptionswithout one — soMaxTokensis structurally always 0. Every tier is capped at 4096 output tokens regardless of model.This also collides with LP-003: a thinking budget must be strictly below
max_tokens, so any meaningful budget is unreachable while the cap is 4096.Beta features are unreachable.
anthropic-betais one hardcoded constant:sent unconditionally at
anthropic.go:67. Prompt caching went GA long ago, so this occupies the only slot with a header that no longer does anything — while 1M context, interleaved thinking, and extended cache TTL all need that slot.It is also sent to third-party Anthropic-compatible gateways (
config/default.yaml:21pointsminimaxat one), which may reject or mishandle an unknown beta.Scope
max_tokenstoLLMTierBindingandResolvedLLMTier; pass it throughhelpers_llm_router.go.max_tokens; keep 4096 as the last-resort fallback for unknown models.thinking_budget < max_tokensat resolve time with a loud error, not at request time.config/tars.config.example.yaml.Acceptance Criteria
max_tokensstill works, using a sane per-model default.thinking_budget >= max_tokensfails at config load with a message naming both values.kind: anthropicgateway tier can be configured to send no beta header at all.Validation
max_tokensthinking_budgetvsmax_tokensguardmake testmake lint-diffOut of Scope