feat(pipeline): widen the reasoning window to see placements (gate item 6) - #13
Merged
Merged
Conversation
…em 6)
Placement detection reads *change*: the VLM contrasts widely-separated frames
and sees "holding" -> "put down, walked away". A tight window shows only
continuous motion toward the surface, so the transition is invisible. Spike 5b
measured this directly -- a +/-5 s window took placement detection from 2/2 to
0/2 on real glasses clips -- and the shipped `reason_window_seconds` was 6 s,
tighter still.
Local validation on LFM2.5-VL-3B (proxy for Cosmos), on spike 5b's own clips
(`clips/recordings/VID_20260819_1207{27,02}_{hi,win}.mp4`), sampling frames
endpoint-inclusive exactly as cosmos does:
* The narrow +/-5 s window answered NOT_PLACED at every frame count 4..8 for
both objects -- window WIDTH is the binding constraint, reproducing 0/2.
* The wide clip detected the placement, but only when sampled densely enough:
the large wallet from 5 frames, the small/thin keyring only at 8. At 4
frames even the wide clip missed both -- widening alone is not sufficient.
So two coupled changes:
- reason_window_seconds 6 -> 20 (kept below the 30 s ring; spike bracketed the
minimum between a ~10 s window that failed and a 28 s clip that passed).
- reason_max_frames 4 -> 8: over a 20 s window 4 frames sit ~7 s apart and fall
either side of the transition; 8 (~3 s apart) recovered detection for the hard
small-object case in the proxy. Stays within the 8192 context at some latency;
exact floor is model-specific, so it is a knob to revisit against real Cosmos.
- event_cooldown_seconds 20 -> 25: wide windows overlap, so one placement is seen
by several; the cooldown must cover a window span or it is rewritten once per
interval across it.
- Two model-validator invariants lock the coupling: window <= ring retention and
cooldown >= window. New test_config.py covers both.
Cosmos already endpoint-subsamples the window, so a wider window costs tokens
only through the frame-count bump, not the width. A wider window also yields a
richer evidence clip (hold -> place -> walk away). Full suite 178 passed, ruff
and pyright clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Placement detection reads change — the reasoner contrasts widely-separated frames and sees "holding" → "put down, walked away". A tight window shows only continuous motion toward the surface, so the transition is invisible. Spike 5b measured this: a ±5s window took detection 2/2 → 0/2 on real clips. The shipped
reason_window_secondswas 6s — tighter still.reason_window_secondsreason_max_framesevent_cooldown_secondsTwo
model_validatorinvariants lock the coupling (window ≤ ring retention, cooldown ≥ window), with newtest_config.py.Local validation on LFM2.5-VL-3B (proxy for Cosmos)
Ran on spike 5b's own clips (
clips/recordings/VID_20260819_1207{27,02}_{hi,win}.mp4;_hi= full 720p,_win= ±5s 720p), sampling frames endpoint-inclusive exactly as cosmos does:reason_max_frames=4, even the wide clip missed both — so widening alone is not sufficient, which is whyreason_max_framesis bumped too.Caveats: LFM is a proxy for Cosmos; the absolute frame floor (8) is model-specific, so
reason_max_framesis a knob to re-tune against real Cosmos (watching the 8192-token context and latency). The direction — narrow never works, density matters for small objects — is robust.Notes
Cosmos already endpoint-subsamples the window (
np.linspace), so widening the window costs tokens only through the frame-count bump, not the width. A wider window also yields a richer evidence clip (hold → place → walk away). Full suite 178 passed, ruff and pyright clean.🤖 Generated with Claude Code