Variant change unnecessarily flushes pending ops, causing cache busts on providers with implicit prefix caching

[null-axiom]

What happened?

Magic Context treats variant (reasoning effort) changes as a flush trigger, adding the session to pendingMaterializationSessions. This forces all queued pending drop operations to apply on the next request, modifying the message stream and busting the API cache prefix. However, on providers that use implicit prefix caching (OpenAI-compatible route), the cache key is based on input tokens only — request parameters like reasoning_effort are NOT part of the cache key. The variant change itself does not bust the cache; MC's flush of pending ops does.

Evidence

Evidence 1: Variant change with 0 pending ops → cache HIT

At 2026-07-27T18:36:24Z, a GLM 5.2 session changed variant from max to high:

18:36:24 — variant changed (max -> high), triggering flush
18:36:25 — transform completed in 145.9ms, reason=cache_hit (message stream NOT modified)
18:36:47 — API response: cache.read=224832 ← CACHE HIT

MC's transform ran but found nothing to apply (no pending ops queued). The message stream was unchanged. The API cache hit 224,832 tokens despite the variant change. This proves the variant change itself does not bust the cache on this provider.

Evidence 2: Variant change with 104 pending ops → cache BUST

At 2026-07-28T21:11:06Z, the same session changed variant from high to max:

21:11:06 — variant changed (high -> max), triggering flush
21:11:06 — heuristics WILL RUN — reason=explicit_flush, context=54.0%
21:11:06 — pending ops WILL APPLY — reason=explicit_flush, pendingOps=104
21:11:07 — applyPendingOperations elapsed=76.7ms
21:11:07 — neutralized 9 dropped + 0 system-injected messages
...
21:17:02 — API response: cache.read=0 ← FULL CACHE BUST (156,188 uncached tokens)

MC's flush applied 104 queued pending ops, dropping 9 messages. This modified the message stream, busting the cache prefix. The context was at 54% — well below the execute_threshold_percentage (65%) — so this flush was premature.

Evidence 3: The cache key does not include request parameters

OpenCode's cache-policy.ts (line 42) shows that the openai-compatible route relies on implicit prefix caching:

const RESPECTS_INLINE_HINTS = new Set(["anthropic-messages", "bedrock-converse"])

The openai-compatible route is NOT in this set. Implicit prefix caching (used by OpenAI-compatible providers) keys on input tokens (system + messages + tools) only. Request parameters like reasoning_effort, temperature, and top_p are not part of the cache key.

Root Cause

dist/index.js lines 31796-31802:

if (previousVariant !== undefined && input.variant !== undefined && previousVariant !== input.variant) {
    sessionLog(sessionId, `variant changed (${previousVariant} -> ${input.variant}), triggering flush`);
    args.historyRefreshSessions.add(sessionId);
    args.systemPromptRefreshSessions.add(sessionId);
    args.pendingMaterializationSessions.add(sessionId);
    args.lastHeuristicsTurnId.delete(sessionId);
}

When the variant changes, the session is added to three sets:

• historyRefreshSessions — triggers history re-injection
• systemPromptRefreshSessions — triggers system prompt re-hash
• pendingMaterializationSessions — forces pending ops to apply on next transform pass

The pendingMaterializationSessions.add(sessionId) is the direct cause of the cache bust. It forces queued pending drop operations to apply immediately, modifying the message stream. Without this, the pending ops would stay queued until the next natural bust event (threshold hit, system prompt change, TTL expiry).

Impact

• Per-agent variant persistence (e.g., Plan on max, Build on high) causes a full cache bust on every agent switch, even though the API wouldn't have busted on its own
• Premature flush at low context — pending ops are applied at 54% context when the threshold is 65%, wasting cache hits that would have served until the natural threshold trigger
• 156K uncached tokens consumed in a single turn from a variant change that the API would have served from cache

Dashboard mislabeling

The dashboard cannot identify variant-change-triggered flushes and falls back to misleading labels:

Actual cause	Dashboard label
Variant change + pending ops applied	"Manual flush"
Variant change + TTL expiry (no pending ops, no modification)	"Magic Context transform did not run (fail-open pass)"

Neither label mentions the variant change. Users cannot diagnose the real cause from the dashboard.

Suggested fix

Add a per-model config option to control flush behavior on variant changes:

// magic-context.jsonc
{
  "flush_on_variant_change": {
    "fireworks/glm-5.2": false,
    "fireworks/kimi-k3": true
  }
}

• Keys match model IDs (or provider/model patterns)
• false — skip all flush triggers on variant change for this model (pending ops stay queued until next natural bust)
• true — current behavior (flush immediately on variant change)
• Unlisted models — default to true (preserves current behavior for backwards compatibility)

When set to false for a model, the variant change handler skips pendingMaterializationSessions.add(sessionId), historyRefreshSessions.add(sessionId), and systemPromptRefreshSessions.add(sessionId) for that model's sessions. Pending ops stay queued until the next natural bust event (threshold hit, system prompt change, TTL expiry).

This lets users disable the flush for models on implicit-prefix-caching providers (where the cache key excludes request parameters) while keeping it enabled for models on providers whose cache key may include request parameters.

Alternative (automatic): Make the flush behavior route-aware. Detect whether the provider uses implicit prefix caching (openai-compatible route) vs explicit cache hints (anthropic-messages, bedrock-converse) and skip the flush for implicit-prefix routes. This requires no user config but assumes all implicit-prefix providers exclude request parameters from the cache key — which needs verification across providers.

Environment

• Magic Context: latest (@cortexkit/opencode-magic-context@latest)
• Model: GLM 5.2 (OpenAI-compatible route)
• OpenCode: v1.18.7
• execute_threshold_percentage: 65