Make the voice end-of-turn pause longer and configurable - #301
Conversation
The VAD closed an utterance after 0.9-1.3 s of silence, derived from the CEFR level and hardcoded in ConversationMode. A learner reaching for a word routinely pauses longer than that, so the turn was submitted while they were still speaking and the tutor answered half a sentence. Nothing in Settings could change it, and a level assigned too high made the window shorter still. The automatic values now leave more room — 1800 ms for A1/A2, 1500 ms for B1/B2, 1200 ms for C1/C2, 1500 ms without a level — and learners who need more can choose 1, 2 or 3 seconds in Settings under Conversation. The resolution rule lives in lib/conversation-vad.ts: the stored conversation_speech_pause wins when set, 0 means automatic. The value is persisted on the user, validated against the offered set, and read when the VAD is created, so a change applies to the next session. The setting is a comfort preference, not a quota, so a subscription downgrade leaves it untouched.
|
Thanks for the contribution. The implementation preserves the stable-turn behavior, so speaking while Lingu is responding still does not interrupt the tutor. Before merging, could you please review the speech-pause selector on narrow mobile screens? The four buttons use a single non-wrapping flex row, and longer German, Spanish, or Russian translations can overflow or become clipped at widths of 320–375 px. A two-column grid on mobile and four columns on larger screens would solve it. There is also a small correction needed in the PR description: I will handle the remaining documentation and versioning updates separately. |
Closes #296.
Problem
ConversationModeset the VAD redemption window from a hardcoded, level-derived table: 1300 ms for A1/A2, 1100 ms for B1/B2, 900 ms for C1/C2, 1000 ms without a level. A learner pausing to search for a word regularly exceeds that, so the utterance is closed and sent mid-sentence and the tutor answers a fragment. Nothing in Settings exposes the window, and a level assigned too high makes it shorter still.Change
Longer automatic values. 1800 ms for A1/A2, 1500 ms for B1/B2, 1200 ms for C1/C2, 1500 ms when the level is unknown. This raises the window for every user, which is the behaviour change in this PR — the scaling-by-level idea is kept, each tier just gets more room.
A user setting.
conversation_speech_pauseon the user stores the window in milliseconds:0means automatic, and1000/2000/3000override it. It is offered in Settings → Conversation next to the existing session settings, validated server-side against that set, and returned by/api/auth/me.One resolution rule.
frontend/src/lib/conversation-vad.tsholdsresolveVadRedemptionMs(speechPause, cefrLevel)— the stored value wins when set, otherwise the level decides. The localvadRedemptionMs()inConversationModeis gone.Two notes on behaviour:
useMicVADreadsredemptionMswhen the VAD is created, so a changed setting applies to the next conversation, not to a session already running.The adaptive option from the issue (keep the turn open after an obviously incomplete utterance) is not implemented; it needs transcript-level signals that are not available at the point the VAD closes a turn.
Migration
0051_conversation_speech_pauseadds the column withserver_default="0". It is hand-written, since autogenerate needs the Docker stack; it applies with the usualalembic upgrade head.Tests
backend/tests/test_conversation.py— PATCH stores a chosen pause,0restores automatic, an unoffered value returns 422, and/api/auth/meexposes the field with its default.frontend/tests/lib/conversation-vad.test.ts— automatic tiers, unknown/missing level, explicit override wins, stored values outside the offered set are ignored, and every tier is longer than the previous window.frontend/tests/components/ConversationSection.test.tsx— the options render, the chosen pause is sent, automatic is sent back as0, and saving another setting preserves the stored pause.Backend: 976 passed, coverage 85%. Frontend: 460 passed, lint and
tsc --noEmitclean.Docs
specs/phase-3-conversation.instructions.md,specs/database-models.instructions.md,specs/api-endpoints.instructions.md,specs/architecture-frontend.instructions.mdand the conversation-settings line inAGENTS.mdare updated.CHANGELOG.mdandspecs/version.mdare left untouched — releases and version bumps look like maintainer territory, and version edits would conflict across parallel PRs.Not covered here
The roadmap/
ENABLE_CONVERSATION_BARGE_INdiscrepancy noticed in the issue is untouched — it is a separate question about whether the roadmap or the flag is out of date.