From a1932b4ddfb43a507c1eaa14d102e45eed9a4350 Mon Sep 17 00:00:00 2001 From: devlikebear Date: Mon, 24 Aug 2026 11:38:52 +0900 Subject: [PATCH] fix(usage): let operator price overrides outrank built-in model rates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up review fixes for #934 (#924). Before per-model Anthropic pricing landed, `anthropic/*` was the only Anthropic key, so an operator on a reseller or flat-rate plan expressed their billing with that one override. The new per-model entries and the family prefix table were both consulted ahead of it, so the override stopped reaching any `claude-*` model: phantom spend accumulated and tripped the daily/weekly USD limits. The existing override test only used a model id that misses every prefix, so the regression was untested. - Resolve operator overrides ahead of every built-in entry (exact, then provider wildcard, then model wildcard within the override layer). Built-in rates are now the default when no override applies, not a shadow over one. - Price claude-fable-5 and claude-mythos-5 explicitly at $10/$50. They match none of the opus/sonnet/haiku prefixes, so they were landing on the Sonnet-rate wildcard — a 3.3x undercount on the priciest models. - Warn about a wildcard fallback only when the provider actually has per-model rates. gemini/* and gemini-native/* are deliberate catch-alls, so every normal Gemini call was logging a spurious "no per-model usage pricing". A `*/model` key is per-model pricing too, so it no longer warns either. - Emit the diagnostic after releasing the tracker mutex, which also guards Limits()/UpdateLimits. - Make the concurrency half of the dedupe test real: the goroutines are now the first callers for their model, so they exercise the map write rather than a pre-populated read. - Add the CHANGELOG entries #934 never wrote. Family-prefix matches still do not warn, per the explicit decision in #934. --- CHANGELOG.md | 3 + internal/usage/tracker.go | 6 ++ internal/usage/tracker_cost.go | 76 +++++++++++++++---- internal/usage/tracker_cost_test.go | 114 ++++++++++++++++++++++++++++ 4 files changed, 186 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 428cc9b1..9849cc4e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,9 @@ The format is based on Keep a Changelog and the project follows Semantic Version ### Fixed - **rolling cache breakpoint가 마커를 조용히 흘리던 문제 (#921)** — 예산은 마킹 가능 여부를 따지기 전에 최신 2개 턴으로 잘려 있어서, 가장 새 턴이 마커를 못 받는 형태(내용이 빈 assistant 메시지, 끝이 `tool_use`인 블록)면 그 슬롯을 더 오래된 턴으로 넘기지 않고 그냥 버렸다. 이제 최신 턴부터 역순으로 훑으며 **실제로 마커가 찍힌 경우에만** 예산을 소모하므로, 마킹 불가능한 턴은 건너뛰고 그 앞 턴이 fallback 자리를 채운다. 아울러 breakpoint 예산 계산에서 실행되지 않던 분기를 걷어내고(`hasSystemBlocks`/`hasTools` bool 두 개 → 예약 슬롯 수 `int` 하나), `anthropicMessageCacheBudget`으로 분리해 예약 수준별로 테스트한다. `cache_control` 리터럴 4곳은 `anthropicEphemeralCacheControl()` 한 곳으로 모았다. +- **`anthropic/*` 가격 오버라이드가 내장 per-model 요금에 가려지던 문제 (#924)** — per-model 요금표가 들어오기 전에는 `anthropic/*`가 유일한 Anthropic 키였고, 운영자가 이 하나로 리셀러·정액제 요금을 표현했다. per-model 항목과 family prefix 표가 추가되면서 둘 다 오버라이드보다 먼저 조회돼, `claude-`로 시작하는 모델은 오버라이드가 전혀 닿지 않고 유령 지출이 쌓여 일/주 USD 한도를 잘못 트립시켰다. 이제 오버라이드는 내장 항목보다 **항상** 먼저 해석되며(오버라이드 안에서는 exact → provider wildcard → model wildcard 순), 내장 표는 오버라이드가 없을 때의 기본값 역할만 한다. +- **Fable 5 / Mythos 5가 Sonnet 요금으로 기록되던 문제 (#924)** — 두 모델은 `claude-opus-`/`sonnet-`/`haiku-` prefix 어디에도 걸리지 않아 Sonnet 요율 wildcard로 떨어졌고, 라인업에서 가장 비싼 모델이 3.3배 저평가됐다. $10/$50(캐시 read $1.00, write $12.50) 명시 항목을 추가했다. +- **wildcard 요금 진단이 의도된 catch-all에서도 울리던 문제 (#924)** — `gemini/*`·`gemini-native/*`는 처음부터 per-model 요금을 두지 않은 provider인데 정상 트래픽마다 "no per-model usage pricing" 경고가 찍혔다. 이제 해당 provider에 per-model 항목이 실제로 존재할 때만(= wildcard 적중이 진짜 공백일 때만) 경고하고, `*/model` 키는 provider-agnostic한 per-model 요금이므로 경고 대상에서 뺐다. 진단 로그는 tracker 뮤텍스를 놓은 뒤 방출한다. - **프롬프트 캐시가 매 턴 무효화되던 문제 (#920)** — 시스템 프롬프트 첫 줄이 초 단위 wall-clock 타임스탬프라 어떤 프로바이더의 prefix 캐시에도 걸리지 않았고, 정적 본문 전체가 매 턴 write 요금으로 재과금됐다. 시각은 이제 프롬프트 맨 뒤 `## Current Time` 블록으로 내려가 분 단위로 truncate되고, `## Prior Context` 회상도 skills/style/goal/critic 등 세션 고정 섹션 **뒤**로 재배치된다. Anthropic 요청은 system을 블록 배열로 보내 `cache_control` breakpoint를 안정 블록에만 찍으므로 동적 tail이 캐시 prefix를 깨지 않는다. 정렬 규칙은 `prompt.BuildResultFor`에 불변식으로 문서화했다. - **메모리 prefetch가 세션 작업 디렉터리를 지우던 문제 (#920)** — 채팅 메모리 캐시가 조립된 프롬프트를 통째로 저장하는 바람에, work dir을 모르는 prefetch 고루틴의 결과가 캐시에 올라가면 그 다음 턴 프롬프트에서 `## Working Directories` 섹션이 통째로 사라졌다. 이제 캐시는 회상 payload만 보관하고 프롬프트는 항상 live 옵션으로 다시 조립한다 — 캐시 hit과 miss가 구조적으로 동일한 프롬프트를 낸다. diff --git a/internal/usage/tracker.go b/internal/usage/tracker.go index 29cf751a..594a8666 100644 --- a/internal/usage/tracker.go +++ b/internal/usage/tracker.go @@ -90,6 +90,10 @@ type Tracker struct { nowFn func() time.Time limits Limits priceByKey map[string]ModelPrice + // overrideByKey holds only operator-supplied rates. They are resolved + // ahead of every built-in entry, so a single `anthropic/*` override still + // covers models that ship with a per-model price. + overrideByKey map[string]ModelPrice // warnedFallbackModels guards the wildcard-pricing diagnostic so each // model warns once per tracker lifetime, not per call. Guarded by mu. warnedFallbackModels map[string]struct{} @@ -123,6 +127,7 @@ func NewTracker(workspaceDir string, opts TrackerOptions) (*Tracker, error) { nowFn: nowFn, limits: normalizeLimits(opts.InitialLimits), priceByKey: defaultPriceTable(), + overrideByKey: make(map[string]ModelPrice, len(opts.PriceOverrides)), warnedFallbackModels: make(map[string]struct{}), } for key, price := range opts.PriceOverrides { @@ -130,6 +135,7 @@ func NewTracker(workspaceDir string, opts TrackerOptions) (*Tracker, error) { if k == "" { continue } + t.overrideByKey[k] = sanitizePrice(price) t.priceByKey[k] = sanitizePrice(price) } _ = t.loadPersistedLimits() diff --git a/internal/usage/tracker_cost.go b/internal/usage/tracker_cost.go index edc8db96..a422a854 100644 --- a/internal/usage/tracker_cost.go +++ b/internal/usage/tracker_cost.go @@ -62,6 +62,11 @@ func defaultPriceTable() map[string]ModelPrice { "anthropic/claude-sonnet-4-6": {InputPer1MUSD: 3.00, OutputPer1MUSD: 15.00, CacheReadPer1MUSD: 0.30, CacheWritePer1MUSD: 3.75}, "anthropic/claude-haiku-4-5": {InputPer1MUSD: 1.00, OutputPer1MUSD: 5.00, CacheReadPer1MUSD: 0.10, CacheWritePer1MUSD: 1.25}, "anthropic/claude-haiku-4-5-20251001": {InputPer1MUSD: 1.00, OutputPer1MUSD: 5.00, CacheReadPer1MUSD: 0.10, CacheWritePer1MUSD: 1.25}, + // Fable/Mythos match none of the opus/sonnet/haiku family prefixes, so + // without explicit entries they fall to the Sonnet-rate wildcard — a + // 3.3x undercount on the priciest models in the lineup. + "anthropic/claude-fable-5": {InputPer1MUSD: 10.00, OutputPer1MUSD: 50.00, CacheReadPer1MUSD: 1.00, CacheWritePer1MUSD: 12.50}, + "anthropic/claude-mythos-5": {InputPer1MUSD: 10.00, OutputPer1MUSD: 50.00, CacheReadPer1MUSD: 1.00, CacheWritePer1MUSD: 12.50}, // Fallback for gateway-hosted or unrecognized Anthropic-kind models // (e.g. config/default.yaml routes MiniMax through kind: anthropic); // Sonnet-class mid rates so unknown traffic still gets an estimate. @@ -101,30 +106,69 @@ func (t *Tracker) EstimateCost(provider, model string, u llm.Usage) (float64, bo return cost, true } +// resolvePrice picks the rate for one (provider, model) pair. +// +// Operator overrides are consulted in full — exact, then provider wildcard, +// then model wildcard — before any built-in entry. Overrides are a deliberate +// statement about what this deployment is billed, so a reseller or flat-rate +// operator can still express it with a single `anthropic/*` entry even though +// the table now ships per-model Anthropic rates. Within each layer the more +// specific key wins. +// +// The fallback diagnostic is emitted after the lock is released, and only when +// falling back for a provider that does have per-model rates — for a provider +// priced by a single wildcard on purpose, a wildcard hit is the intended +// answer, not a gap. func (t *Tracker) resolvePrice(provider, model string) (ModelPrice, bool) { + price, ok, warnProvider, warnModel := t.lookupPrice(provider, model) + if warnProvider != "" { + warnPriceFallback(warnProvider, warnModel) + } + return price, ok +} + +func (t *Tracker) lookupPrice(provider, model string) (price ModelPrice, ok bool, warnProvider string, warnModel string) { t.mu.Lock() defer t.mu.Unlock() p := strings.TrimSpace(strings.ToLower(provider)) m := strings.TrimSpace(strings.ToLower(model)) if p == "" || m == "" { - return ModelPrice{}, false + return ModelPrice{}, false, "", "" + } + for _, key := range []string{p + "/" + m, p + "/*", "*/" + m} { + if price, ok := t.overrideByKey[key]; ok { + return price, true, "", "" + } } if price, ok := t.priceByKey[p+"/"+m]; ok { - return price, true + return price, true, "", "" } if price, ok := matchFamilyPrice(p, m); ok { - return price, true + return price, true, "", "" } if price, ok := t.priceByKey[p+"/*"]; ok { - t.noteWildcardFallback(p, m) - return price, true + wp, wm := t.noteWildcardFallback(p, m) + return price, true, wp, wm } if price, ok := t.priceByKey["*/"+m]; ok { - t.noteWildcardFallback(p, m) - return price, true + // A `*/model` key is per-model pricing, just provider-agnostic — + // nothing is missing, so this is not a fallback worth reporting. + return price, true, "", "" } - return ModelPrice{}, false + return ModelPrice{}, false, "", "" +} + +// providerHasPerModelPrices reports whether the table carries any per-model +// entry for this provider. Callers must hold t.mu. +func (t *Tracker) providerHasPerModelPrices(provider string) bool { + prefix := provider + "/" + for key := range t.priceByKey { + if strings.HasPrefix(key, prefix) && key != prefix+"*" { + return true + } + } + return false } func matchFamilyPrice(provider, model string) (ModelPrice, bool) { @@ -139,15 +183,21 @@ func matchFamilyPrice(provider, model string) (ModelPrice, bool) { return ModelPrice{}, false } -// noteWildcardFallback emits the fallback diagnostic at most once per model. -// Callers must hold t.mu. -func (t *Tracker) noteWildcardFallback(provider, model string) { +// noteWildcardFallback claims the once-per-model diagnostic slot and reports +// which (provider, model) the caller should warn about, or two empty strings +// when no warning is due. It never logs itself: callers hold t.mu, and the +// tracker mutex also guards Limits()/UpdateLimits, so the emission happens +// after the lock is released. Callers must hold t.mu. +func (t *Tracker) noteWildcardFallback(provider, model string) (string, string) { + if !t.providerHasPerModelPrices(provider) { + return "", "" + } key := provider + "/" + model if _, ok := t.warnedFallbackModels[key]; ok { - return + return "", "" } t.warnedFallbackModels[key] = struct{}{} - warnPriceFallback(provider, model) + return provider, model } func clampUsageTokens(u llm.Usage) (input int, output int, cached int, cacheRead int, cacheWrite int) { diff --git a/internal/usage/tracker_cost_test.go b/internal/usage/tracker_cost_test.go index c6f3990b..ce605d36 100644 --- a/internal/usage/tracker_cost_test.go +++ b/internal/usage/tracker_cost_test.go @@ -205,6 +205,120 @@ func TestEstimateCost_PriceOverridesTakePrecedenceOverBuiltIns(t *testing.T) { } } +// An operator on a flat-rate or reseller plan expresses it with a single +// `anthropic/*` override. Before per-model pricing existed that was the only +// Anthropic key, so it covered every model; built-in per-model rates must not +// silently take that away. +func TestEstimateCost_WildcardOverrideCoversModelsWithBuiltInPrices(t *testing.T) { + tracker, err := NewTracker(t.TempDir(), TrackerOptions{ + PriceOverrides: map[string]ModelPrice{ + "anthropic/*": {InputPer1MUSD: 0, OutputPer1MUSD: 0, CacheReadPer1MUSD: 0, CacheWritePer1MUSD: 0}, + }, + }) + if err != nil { + t.Fatalf("new tracker: %v", err) + } + + oneMillion := llm.Usage{InputTokens: 1_000_000, OutputTokens: 1_000_000} + cases := []struct { + name string + model string + }{ + {"built-in exact entry", "claude-opus-4-7"}, + {"family prefix match", "claude-opus-4-9"}, + {"built-in wildcard", "unknown-gateway-model"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got, ok := tracker.EstimateCost("anthropic", tc.model, oneMillion) + if !ok { + t.Fatalf("expected an estimate for %s", tc.model) + } + if got != 0 { + t.Fatalf("operator wildcard override must cover %s, got %v", tc.model, got) + } + }) + } +} + +// Claude Fable 5 and Mythos 5 match none of the opus/sonnet/haiku prefixes, so +// they land on the Sonnet-rate wildcard — a 3.3x undercount on the priciest +// models in the lineup. +func TestEstimateCost_FableAndMythosPricedAtTheirOwnRates(t *testing.T) { + tracker := newCostTestTracker(t) + for _, model := range []string{"claude-fable-5", "claude-mythos-5"} { + got, ok := tracker.EstimateCost("anthropic", model, llm.Usage{InputTokens: 1_000_000}) + if !ok { + t.Fatalf("expected pricing for %s", model) + } + if got != 10.00 { + t.Fatalf("%s input should price at $10.00/1M, got %v", model, got) + } + got, ok = tracker.EstimateCost("anthropic", model, llm.Usage{OutputTokens: 1_000_000}) + if !ok || got != 50.00 { + t.Fatalf("%s output should price at $50.00/1M, got %v ok=%v", model, got, ok) + } + } +} + +// A provider whose table entry is only a wildcard has no per-model pricing to +// be missing, so the diagnostic is noise there. +func TestEstimateCost_WildcardOnlyProviderDoesNotWarn(t *testing.T) { + var mu sync.Mutex + warned := map[string]int{} + original := warnPriceFallback + warnPriceFallback = func(provider, model string) { + mu.Lock() + defer mu.Unlock() + warned[provider+"/"+model]++ + } + defer func() { warnPriceFallback = original }() + + tracker := newCostTestTracker(t) + if _, ok := tracker.EstimateCost("gemini-native", "gemini-3-pro", llm.Usage{InputTokens: 1_000}); !ok { + t.Fatalf("expected gemini wildcard estimate") + } + + mu.Lock() + defer mu.Unlock() + if n := warned["gemini-native/gemini-3-pro"]; n != 0 { + t.Fatalf("wildcard-only provider must not warn, got %d warnings", n) + } +} + +// The dedupe map is written under the tracker mutex; the goroutines must be the +// first callers for this model or the write path is never exercised. +func TestEstimateCost_FallbackDiagnosticDedupesUnderConcurrency(t *testing.T) { + var mu sync.Mutex + warnCount := 0 + original := warnPriceFallback + warnPriceFallback = func(string, string) { + mu.Lock() + defer mu.Unlock() + warnCount++ + } + defer func() { warnPriceFallback = original }() + + tracker := newCostTestTracker(t) + usage := llm.Usage{InputTokens: 1_000} + + var wg sync.WaitGroup + for i := 0; i < 8; i++ { + wg.Add(1) + go func() { + defer wg.Done() + _, _ = tracker.EstimateCost("anthropic", "never-seen-gateway-model", usage) + }() + } + wg.Wait() + + mu.Lock() + defer mu.Unlock() + if warnCount != 1 { + t.Fatalf("expected exactly one warning across concurrent first calls, got %d", warnCount) + } +} + func TestEstimateCost_OpenAIEntriesReflectDocumentedPricing(t *testing.T) { tracker := newCostTestTracker(t)