Collapse repeated per-request snapshot loads and provider-key resolutions in the chat dispatch path

[membphis]

Context

A profiling audit of the zero-config hot path found the request lifecycle loads the config snapshot handle up to 6 times per chat request (auth extractor, dispatch, quota gate, request-metrics emit x2, usage-event emit; +1 on the streaming path) and resolves the same ProviderKey 4-5 times (dispatch-time resolutions plus provider_key_metric_name twice in request_metrics::record/record_usage and the telemetry-tags lookup in emit_usage_event).

The dominant costs found by the same audit (full exporter-table scans per request, empty-table shard walks) have been fixed at the ResourceTable chokepoint. What remains is the structural part:

• Thread one Arc<AisixSnapshot> (loaded once at dispatch entry) through the emit helpers (emit_usage_event, request_metrics::record/record_usage) instead of re-loading per helper. Note this freezes a request's config view to its start, which is arguably more correct but is a small observable change for exporters added mid-request.
• Resolve the attempt's ProviderKey entry once per completion and pass the resolved entry (or its display name + telemetry tags) into the metric and usage emitters instead of re-resolving by id in each.

Why deferred

Each load is one atomic RMW and each resolution one DashMap read + small clone (~0.5us/request combined, frame-inclusive). Capturing it means changing the signatures of emit helpers shared by the whole endpoint family (10+ handler files, streaming and non-streaming branches) — a wide, cross-family diff that was out of proportion for the remaining size in the zero-config package PR. Doing it later as a focused refactor keeps the family in lockstep in one review.

Size

~0.4-0.6us/request estimated recoverable; touches aisix-proxy handler family + request_metrics + usage_attr.

[coderabbitai]

🔗 Related PRs

#835 - feat(headers): request-context variables in default_headers + client-header forwarding allowlist [merged]
#851 - perf(proxy): make request state clones constant-cost [merged]
#852 - feat(obs): record client-abandoned requests as 499 [merged]
#891 - feat(proxy): serve from thread-per-core workers with per-worker upstream pools [merged]
#925 - perf(metrics): per-worker handle cache for every per-request emit path [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[membphis]

Landed in #953 (squash-merged as a04a66c).

What shipped. Both halves of this issue, across the whole client-facing
handler family in one PR:

• One Arc<AisixSnapshot> per request, threaded from handler entry through
  dispatch, the quota gate and every terminal emitter. A non-streaming chat
  request loads twice (the auth extractor, then the handler) instead of six
  times; the streaming path drops its request-start load as well.
• One ProviderKey resolution per emitted attempt, via a new
  usage_attr::ResolvedPk that serves both the provider_key_name metric
  label and the usage event's telemetry_tags from a single row read —
  replacing three reads per request.

request_metrics::Upstream now takes a PkLabels { id, name } pair rather
than a bare id, which makes the collapse compile-forced: a call site cannot
reintroduce a per-emit lookup, or pair a real id with a defaulted
"unknown" name, without a compile error. That is why every endpoint —
chat, messages (+count_tokens), responses, completions, embeddings,
rerank, images, audio, videos, realtime, MCP, A2A, passthrough and the
jobs surface, streaming and non-streaming — was converted in the same PR
rather than left as follow-ups.

Census (the primary evidence). Counting the distinct call sites
SnapshotHandle::load is sampled under, on same-rig c=128 flamegraphs:

call site	main (front)	main (back)	branch
auth::authenticate_token	0.0388%	0.0570%	0.0748%
chat::dispatch → chat::chat_completions	0.0323%	0.0618%	0.0326%
request_metrics::record	0.0257%	0.0572%	gone
chat::emit_usage_event	0.0194%	0.0333%	gone
quota::reserve_policy_layers	0.0130%	0.0238%	gone
request_metrics::record_usage	0.0064%	0.0143%	gone
distinct call sites	6	6	2

Those six are exactly the six this issue named, and both main legs
reproduce the set independently. The ProviderKey resolution collapses the
same way: two sampled call sites on main to one on the branch.

Worth recording as method, because the first pass got it wrong: comparing
frame shares between the two legs is not sound at this magnitude. The
binaries are LTO'd independently and disagree about what is inlined, the
folded captures carry different total sample counts (72.7G vs 99.9G), and
at 0.01-0.07% per site the sampling error dominates — the unchanged
auth load site alone swings +93% between the two main legs, and
ResourceTable::get_by_name, which this change never touches, moved
+135%. The call-site SET survives all of that, because an emit that no
longer loads cannot appear as a parent of the load frame in any sample.

Throughput: no regression, and the gain is not resolvable. Same-rig
anchored A/B, main → branch → main, each leg its own build, full grid,
25s x 4 windows, every window fail=0:

leg	c=16	c=32	c=128	768@10ms	c=128 CPU/req
main (front)	34,063	37,398	39,741	34,183	100.5 us
branch	33,752	37,781	39,743	34,332	100.5 us
main (back)	34,000	36,663	38,626	33,408	103.4 us

The front-to-back anchor drift at c=128 is −2.81%, larger than the effect
being measured. Against the front anchor the branch is +0.005%; against
the linearly interpolated anchor pool (the branch leg ran at the session's
temporal midpoint) it is +1.43%. The true value is somewhere in
[0%, +1.4%] and this session cannot narrow it further. The 0.4-0.6 us this
issue estimated sits inside that band — it was always likely to land below
single-leg resolution, which is why the census was the acceptance
criterion rather than the throughput delta.

Two corrections to this issue's framing, both carried in the PR:

• The "4-5 ProviderKey resolutions" figure counts the dispatch-side
  resolve_provider_key calls, which are routing's own work and out of
  scope here. The emit side did exactly three, and now does one.
• The change surface is 20 files rather than "10+" — the emit-helper
  signatures reach the quota gate, the ensemble sub-call path and the
  crate's own tests as well as the handlers.

Observable change, as this issue anticipated. A request's config view
is frozen at handler entry, so an exporter added mid-request no longer
reaches that request's usage event. Long-lived work is deliberately
excluded: the end-of-stream emits and the realtime session-end emit read a
fresh snapshot rather than pinning the request's, since either can outlive
several config generations.

Series names, label sets and label values are unchanged — verified by a
main-vs-branch /metrics comparison over 24 driven cases spanning the
whole family (including the single-attempt endpoints and the native
Anthropic-protocol path), 859 identical series with no drift.