Skip to content

fix: trim conversation history to fit context window (prevents 'No response came') - #167

Closed
Fahad090NP wants to merge 4 commits into
ltmoerdani:mainfrom
Fahad090NP:fix/history-trim-context-overflow
Closed

fix: trim conversation history to fit context window (prevents 'No response came')#167
Fahad090NP wants to merge 4 commits into
ltmoerdani:mainfrom
Fahad090NP:fix/history-trim-context-overflow

Conversation

@Fahad090NP

Copy link
Copy Markdown
Contributor

Summary

Fixes two related "No response came" failure modes with a single root-cause fix:

  1. Multiple VS Code windows — chatting in separate windows intermittently fails with no response.
  2. Repeated failures fixed by Compact Conversation — after many turns the request fails every time; running Compact Conversation and sending one message works again.

Root cause

provideLanguageModelChatResponse sent the entire conversation history as the request payload. The only history guard was image trimming (trimOldImagesFromHistoryInPlace) — there was no text-history truncation. So a long conversation grows past the model's context window and the upstream either:

  • rejects the oversized request with HTTP 400 (empty assistant message, finish_reason: null), or
  • returns an empty stream → VS Code surfaces "No response came".

Compact Conversation shrinks the history, which is exactly why it "fixes" the symptom. (This also explains the reported Muse Spark ~990 KB payload — it is the uncompacted history, not a model-specific quirk.)

Changes

  • src/provider/historyTrim.ts (new, pure) — trimOldMessagesToFitContext drops the oldest messages until the estimated prompt tokens fit contextWindow − maxOutputTokens − safetyMargin. It preserves the first (system/anchor) and last (current prompt) messages, and never splits a tool-call group (stops before the first assistant message carrying tool_calls), so no tool reference is orphaned.
  • src/provider/OpenCodeProvider.ts — calls it right after the image-history trim, before the prompt-token estimate, with a [history-trim] diagnostic log.
  • src/config.tsHISTORY_TRIM_SAFETY_MARGIN_TOKENS = 2048.
  • Testssrc/test/messages.test.ts (no-op when fitting, drops oldest keeping anchor + last, never splits a tool group, no-op when only anchor + last remain).
  • Docs — CHANGELOG Fixed entry + docs/issues/68-20260820-history-trim-context-overflow.md.

Verification

  • npm test — 337/337 pass
  • npm run lint — all 7 gates pass (Editorconfig, ESLint, Markdown, Prettier, Shell, TypeScript, Tests)

PEACE BE UPON YOU

…ontext window

Long multi-turn conversations (or many repeated turns without Compact Conversation) can grow past the model context limit, so the upstream rejects the oversized request (HTTP 400) or returns an empty stream — surfaced by VS Code as 'No response came'. Add a pure helper that drops the oldest messages until the payload fits the input budget, preserving the anchor and current prompt and never splitting a tool-call group. Add HISTORY_TRIM_SAFETY_MARGIN_TOKENS.
…hatResponse

Call trimOldMessagesToFitContext after the image-history trim, before the prompt-token estimate, so the request payload is bounded to the model's input context window. Logs a [history-trim] diagnostic when messages are dropped.
Add unit tests for no-op when fitting, dropping oldest while keeping anchor + last, never splitting a tool-call group, and no-op when only anchor + last remain.
Add a Fixed entry to CHANGELOG and a concise issue doc (68-20260820) describing the conversation-history trimming that prevents oversized-payload 'No response came' errors.
@Fahad090NP

Copy link
Copy Markdown
Contributor Author

Superseded by #169 — that PR is branched from this one and adds the hardening (70% context ratio, 512 KB byte cap, O(n) trim, tool-group dropping) that resolves the 503 / 10-min timeout / empty-response / session-slowness symptoms and closes #165.

@Fahad090NP Fahad090NP closed this Aug 20, 2026
@Fahad090NP
Fahad090NP deleted the fix/history-trim-context-overflow branch August 20, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant