fix(ai): abort degenerate generation instead of streaming it to the cap - #1099
fix(ai): abort degenerate generation instead of streaming it to the cap#1099r0h1tb wants to merge 1 commit into
Conversation
A thinking stream settled into emitting "The the the the the the the " back to
back 2,366 times, 79,222 characters, and stopped only when the user aborted by
hand. No layer owned this: provider streamers are faithful assemblers and never
inspect content, request construction attaches no repetition penalties, and the
agent loop treats a completed stream as progress. A runaway response could burn
the whole 32k output cap.
Adds a stream-level guard on the reasoning channel of openai-completions. Two
detectors:
- Periodic tail. The tail is one short unit repeated many times, found with the
KMP failure function, which gives the smallest period in O(n) with no scan
over candidate lengths.
- Novelty stall. Distinct word trigrams over total, for loops that drift enough
that verbatim periodicity misses them.
The novelty floor is calibrated rather than guessed. Measured over the corpus in
the tests:
loops verbatim 0.001 drifting 0.136
legit enumerated analysis 0.286 markdown table 0.485 JSON 0.513
source code 0.611 numbered list 0.671 prose 1.000
0.20 sits in that gap. Word-trigram Jaccard between paragraphs was tried first
and rejected: drifting loops scored 0.38-0.58 and structurally similar
legitimate paragraphs scored in the same band, so no threshold separated them.
False positives are the real risk, since a wrong abort destroys real work, so
the tests assert that code, markdown tables, numbered lists, JSON, prose and
enumerated analysis do not trip it. The text channel is deliberately left
unguarded: a long legitimate answer can contain generated tables or code.
On a hit the stream fails with a new degenerate_output failure kind rather than
committing the garbage. Kill switch: PRIME_AGENT_NO_REPETITION_GUARD=1.
Also wires recordStreamFailure into the openai-completions terminal catch.
Every other provider already called it; this one did not, so failures on the
most widely used path (OpenRouter, llama.cpp, any OpenAI-compatible endpoint)
reached the session with no classification at all. The new kind depends on it,
but this fixes classification for every existing kind on that provider too.
fixes PrimeIntellect-ai#1029
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Full Two files are excluded from that run: execSync("which ollama");
describe.skipIf(!ollamaInstalled)(...)so they skip on CI, which has no Ollama, but activate on a developer machine Together with the earlier runs: |
|
Thank you for the report and proposed work. This root cause is now covered by maintainer-owned stacked PR #1165, authored independently from We did not inspect or reuse this PR's diff, branch, commits, implementation code, or tests; its public description/comments were used only as a bug report. To keep one review surface, this PR is superseded by #1165 and is being closed. The complete review stack is #1158–#1165. It is being left unmerged for human review after CI and review-bot findings are cleared. |
2 participants
A thinking stream settled into emitting
"The the the the the the the "back toback 2,366 times — 79,222 characters — and stopped only when the user aborted by
hand. No layer owned this: provider streamers are faithful assemblers and never
inspect content, request construction attaches no repetition penalties, and the
agent loop treats a completed stream as progress. A runaway response could burn
the whole 32k output cap.
fixes #1029
Detection
A stream-level guard on the reasoning channel of
openai-completions, with twodetectors:
the KMP failure function, which yields the smallest period of a string in
O(n), so there is no scan over candidate period lengths.
enough that verbatim periodicity misses them.
The novelty floor is calibrated, not guessed
I first implemented near-duplicate paragraph clustering by word-trigram Jaccard,
following the prior art named in the issue. Measured, it does not work here:
drifting loops scored 0.38–0.58, and structurally similar legitimate paragraphs
scored in the same band. No threshold separates them, so I removed it.
Distinct-trigram novelty does separate. Measured over the corpus in the tests:
0.20sits in the gap with margin on both sides. The table is reproduced in themodule docstring so the number is not mistaken for a magic constant.
False positives are the real risk
A wrong abort destroys real work, which is worse than the loop it prevents. The
tests assert the guard does not fire on source code, markdown tables,
numbered lists, JSON, prose, enumerated analysis, whitespace padding, short
sub-threshold stutters, or anything under the minimum inspection length.
One of those tests caught a mistake of mine: my first "recovery text" repeated a
single sentence 40 times and the guard fired. It was right to — that shape is
degenerate. The test corpus was wrong, not the detector, and the comment now
says so.
The text channel is deliberately left unguarded. A long legitimate answer can
contain generated tables or code, and the cost asymmetry there is the wrong way
round. This is one of the two deviations the issue flagged as needing sign-off —
the other, default-on for all models, is how this ships. Kill switch:
PRIME_AGENT_NO_REPETITION_GUARD=1.A second gap found on the way
openai-completions.tswas the only major provider that never calledrecordStreamFailure— anthropic, google, google-vertex, amazon-bedrock,mistral, openai-responses and azure-openai-responses all do. So failures on the
most widely used path (OpenRouter, llama.cpp, any OpenAI-compatible endpoint)
reached the session with no classification at all.
The new
degenerate_outputkind depends on it, but wiring it in fixesclassification for every existing failure kind on that provider too. Flagging it
because it is scope beyond the issue and easy to miss in review — happy to split
it out.
Verification
25 tests across two files. Unit tests cover detection and the false-positive
corpus; the end-to-end test drives the real
streamOpenAICompletionsagainst alocal SSE server that replays the incident (2,366 reasoning deltas of the loop
unit), following the existing pattern in
openai-completions-thinking-as-text.test.ts.End to end the stream now terminates as an error, classifies as
degenerate_output, and stops after under 20,000 characters against the79,222 the incident produced. With the kill switch set, all 600 deltas stream
and the turn ends
done.All 9 existing
openai-completions-*test files still pass (81 tests).npm run checkexits 0; the pre-commit hook ran it again independently.Not done
No sampling penalties (
frequency_penalty/presence_penalty) and nocross-turn tool-call loop guard. Both are mentioned in the issue, both are
separate decisions, and neither is needed to stop this failure.
Note
Abort degenerate AI reasoning streams instead of forwarding repetitive output
RepetitionGuardinrepetition-guard.tsthat detects two degenerate patterns in streaming text: periodic tail repetition (via KMP failure function) and novelty stall (low trigram novelty ratio).reasoning_contentchannel and throw aStreamFailureErrorearly when degeneracy is detected.degenerate_outputwith arepetition:...providerErrorType and attaches structured diagnostics viarecordStreamFailure.REPETITION_GUARD_DISABLED_ENV) disables the guard and restores the original pass-through behavior.Macroscope summarized a17252d.