Cost is always $0 when routing through the vercel gateway provider

[mostlydev]

Summary

Any model routed through the vercel provider reports cost_usd = 0. claw audit shows COST_USD 0.0000 for every agent and model, and reported_cost_usd is absent from response telemetry.

Confirmed on cllama v0.9.0.

Cause

internal/cost/pricing.go keys the rate table by provider:

return &Pricing{rates: map[string]map[string]Rate{
    "anthropic":  {...},
    "openai":     {...},
    "openrouter": {...},
    "google":     {...},

There is no vercel entry, and Lookup returns early on an unknown provider:

func (p *Pricing) Lookup(provider, model string) (Rate, bool) {
    models, ok := p.rates[provider]
    if !ok {
        return Rate{}, false
    }

vercel is a first-class provider in internal/provider/provider.go (https://ai-gateway.vercel.sh/v1, keyed by AI_GATEWAY_API_KEY), so it is reachable but unpriceable.

There is a second layer to this. Gateway model ids carry their own sub-provider prefix, so a configured model looks like vercel/openai/gpt-5.6-luna and the model string reaching Lookup is openai/gpt-5.6-luna. Even with a vercel table added, keys like gpt-5.6-luna would not match unless the sub-provider prefix is stripped or the table is keyed by the full gateway id.

The existing prefix-match fallback does not help, because it only runs after the provider lookup has already succeeded.

Impact

Telemetry is the visible part, but internal/proxy/budget.go enforces spend on the same value:

if normalized.LimitUSD != nil && summary.ReportedCostUSD >= *normalized.LimitUSD {

A limit_usd budget on a gateway-routed service can therefore never trip — it silently does not enforce. unknown_cost_requests is tracked in the decision log, so the information needed to detect this already exists.

Suggested fix

Resolve pricing against the effective downstream provider and model rather than the routing provider: for gateway providers, split the sub-provider prefix off the model id and look the rate up under that provider's table. That reuses the existing per-provider rates instead of duplicating a table per gateway.

Failing that, at minimum surface the condition — a startup warning when a configured model resolves to no rate, and/or refuse to accept a limit_usd budget for a service whose model has no known rate, so an unenforceable budget fails loudly instead of silently.

[mostlydev]

Investigated further and found a simpler, more accurate fix than the one I suggested above. Deferring the PR — filing the findings here so the work is not lost.

The gateway already reports cost

A non-streaming completion through the gateway returns cost in the usage block, in USD:

"usage": {
  "prompt_tokens": 7, "completion_tokens": 13, "total_tokens": 20,
  "cost": 1.7e-05, "market_cost": 1.7e-05, "gateway_cost": 1.7e-05,
  "is_byok": false
}

internal/cost/usage.go:79 already parses this into ReportedCostUSD via floatPtrFromAny(obj["cost"]). The only thing discarding it is providerReportedCostIsUSD in internal/proxy/handler.go, which allowlists openrouter and nothing else — so resolveCost falls through to the static table, misses on the unknown vercel provider, and returns 0.

So the primary fix is one case arm:

case "vercel":
    return true

That is strictly better than a static table: the gateway's figure already accounts for the resolved upstream provider, cache hits and tier boundaries, and it does not drift as prices change.

Static rates still worth adding as a fallback

For responses that arrive without usage.cost, the table should still price gateway traffic rather than report zero. Keyed by the full upstream id, same shape as the existing openrouter map, since that is the model string sent upstream:

• vercel → openai/gpt-5.6-{luna,sol,terra}, anthropic/claude-haiku-4.5, anthropic/claude-{opus,sonnet,fable}-5, google/gemini-3.6-flash, xai/grok-4.3 — all reusing the existing rate constants.
• xai is missing entirely as a direct provider, which is a separate pre-existing blind spot: direct xai/grok-4.3 traffic has always priced at $0. Rate is $1.25/$2.50 per Mtok, doubling above 200k tokens, which the existing LongContextThresholdTokens / multiplier fields model exactly.

Note the id-shape trap for anyone implementing this: the anthropic map uses claude-haiku-4-5 (dashes) while the gateway and openrouter maps use claude-haiku-4.5 (dot). Splitting the sub-provider prefix and reusing the direct table — my original suggestion — misses for that reason. Keying the gateway table by full id avoids it.

Both changes build clean locally. Happy to open the PR when it comes back up the queue.