MiniCPM-o 4.5: greedy decoding degenerates (empty / repetition-loop outputs) on long multimodal prefixes (>12k tokens)

[LukeLIN-web]

Summary

MiniCPM-o 4.5's greedy decoding degenerates on long multimodal prefixes: once the audio+video prompt grows past roughly 12k tokens, an increasing fraction of generations come back as an empty string, a single-character repetition loop (CCCCCC… for 130 tokens), or stray junk (5, 44) instead of an answer. The same inputs fed to another omni model (Qwen3-Omni-30B-A3B) at even longer prefix lengths produce 100% well-formed outputs, so this looks like a model/training-distribution issue rather than a harness issue.

Environment

• Model: MiniCPM-o 4.5 (HF snapshot, config.json transformers_version: 4.51.0), trust_remote_code=True
• torch.bfloat16, attn_implementation="sdpa", init_tts=False, single 48G GPU (L40S)
• Decoding: greedy — do_sample=False, num_beams=1, repetition_penalty=1.02 (we call model.generate mirroring chat() preprocessing; also reproduced through the stock model.chat(..., do_sample=False, max_new_tokens=128, max_inp_length=32768) path, see below)
• Inputs: long-video QA (public benchmarks: TraceAV-Bench, MMOU, LVOmniBench, Video-Odyssey). Prompt = several 75s video segments (10 frames each) interleaved with their audio, plus an MCQ question. Prefix length varies with how many segments are packed.

Dose–response: failure rate climbs with prefix length

Rate of well-formed answers (non-empty, parseable option letter) vs. prompt length, greedy decoding, same prompt template throughout — only the number of packed segments varies (n in parentheses):

prefix tokens	TraceAV	MMOU	LVOmniBench	Video-Odyssey
< 6k	94.4% (54)	99.4% (2656)	—	—
6–10k	93.7% (646)	98.8% (881)	100% (65)	100% (12)
10–12k	80.2% (258)	90.7% (647)	89.2% (139)	89.1% (165)
12–13k	74.3% (175)	86.5% (111)	88.0% (183)	95.9% (97)
≥ 13k	73.2% (1067)	83.0% (705)	90.0% (602)	89.3% (788)

Failure signatures at ≥13k on TraceAV (286 failures): 192 off-answer prose / option echo, 71 short junk (5, 44), 23 completely empty generations, plus repetition loops like:

CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC…  (130 tokens of "C")

Also reproduces through the stock chat() whole-video path

Feeding whole long clips through model.chat() itself (official-style ingest, do_sample=False, max_inp_length=32768) on TraceAV-Bench: 24.9% of items return a fully empty string and 57.7% are unparseable overall. Bucketing by prefix length shows the same cliff: parse rate 99.1% at 8–12k tokens → 73.6% at 12–16k → worse beyond.

Evidence it's the LM decoding, not the media ingest

• Same-input A/B: with the identical packed prompts, attaching a LoRA fine-tuned on long-prefix QA data brings every bucket in the table above to 100% well-formed. The base weights' output distribution at long prefixes is the only thing that changes.
• Cross-model control: the identical segment-union prompts (same videos, same windows, longer prefixes up to ~18k tokens) on Qwen3-Omni-30B-A3B greedy decode: 100% well-formed (0 empty / 0 loops).

So the failure is recoverable by light SFT, which suggests the pretraining/SFT mix has little coverage of >12k-token interleaved audio-video prefixes, and greedy decoding falls off the manifold there (empty EOS-first or repetition loops).

Questions

1. Is a >12k-token multimodal prefix outside the intended operating range for MiniCPM-o 4.5? The docs advertise long-video understanding, and max_inp_length=32768 is accepted silently.
2. Is there a recommended decoding configuration (sampling, repetition penalty, min_new_tokens) that mitigates this?
3. If this is a known training-coverage gap, it would be worth documenting on the model card — the empty-string failure mode silently scores as wrong answers in any benchmark harness.

Happy to provide item-level IDs / exact prompts for any of the public benchmarks above if useful.

[tc-mb]

Thanks for the detailed report and controlled experiments.

MiniCPM-o 4.5 was trained and validated with a recommended maximum multimodal input length of 8K tokens, which is also why chat() defaults to max_inp_length=8192. Setting this parameter to 32768 only overrides the preprocessing truncation limit; it does not mean that generation quality is guaranteed at that length.

The continuous video and audio capabilities of MiniCPM-o 4.5 are designed primarily for its streaming interaction modes. They should not be interpreted as support for packing arbitrarily long audio-video segments into a single non-streaming prefix. Prefixes above 12K tokens are outside the recommended operating range of the current release, so the reported degradation, empty outputs, and repetition loops are consistent with a limitation of the current model rather than an inference implementation issue.

Sampling, min_new_tokens, or repetition penalties may mitigate some abnormal outputs, but they cannot address the underlying long-context limitation. For the current release, we recommend keeping multimodal inputs within 8K tokens.

We agree that this usage boundary should be stated more clearly. Supporting longer multimodal contexts requires improvements to both the model and training data, which we will work on in a future release. Thanks again for sharing these detailed results.