Product: transcripts for imported audio — editing/subtitle source selectors, media panel gap

[EtienneLescot]

Context

#543 (issue #350) lands imported audio as first-class timeline regions: voiceover and music lanes, preview + export mixing, agent tools. Two gaps remain, both product-shaped:

1. A voiceover is speech with no transcript. The transcript tab — which drives trims and feeds the agent's word-level grounding — only knows the recording.
2. Imported audio is invisible in the media panel — the review's videoSources/audioSources split (an imported mp3 must never become a clip) left the panel listing footage only, so an imported file lives nowhere except as a pill.

Architecture (agreed — supersedes the selector phasing below this section's earlier drafts)

One tab. The caption tab disappears; caption settings become a popover launched from the transcript tab (same pattern as the background popover in Composition). The caption tab's "transcribe video" entry point dies with it — the transcript tab's no transcription yet empty state becomes the single gate for transcription.

Transcription is automatic for what can carry speech, and only that. No user-facing decision lives at the asset layer either way — the difference is that music is not transcribed at all, rather than transcribed and then excluded from the selector. STT on a music bed is noise we pay for.

"Can carry speech" is not a property of the asset. AxcutAsset.kind is only video | audio; the voiceover/music distinction lives on the region (audioRegionSchema.kind), which is a timeline fact. So the speech-bearing set is: footage (the recording and any imported video), plus an audio asset placed on a voiceover lane.

Three consequences of that, each of which would otherwise be discovered mid-implementation:

• The trigger moves from import to placement. An mp3 dropped on the music lane costs nothing. Transcription starts when an asset first lands on a voiceover lane.
• "First landing", not "currently there". The lane is user-changeable — the inspector's lane selector and the agent's setAudio both write kind. A rule keyed on the current lane would re-trigger on every music→voiceover flip, and would silently invalidate the aggregate the user is editing in.
• Voiceover → music keeps the transcript. It is already paid for, and discarding it makes the round-trip lossy.

Note for whoever implements this: addAssetInputSchema.autoTranscribe already exists (schema/index.ts:870) and nothing reads it. It is the natural place to carry this decision — or it should be deleted rather than left as a decoy that reads like a working switch.

The user choice happens at the aggregation layer: which lane feeds the aggregated transcript. The transcript tab never shows a single asset transcript — it shows aggregated-transcript.ts's join of the per-asset transcripts over the timeline's clips (source windows, synthetic silences, trim runs). The aggregation becomes parameterized by lane:

• Recording lane (default) — today's behaviour: aggregate over timeline.clips.
• Voiceover lane — aggregate over the voiceover pills. Their source windows come from placeAudioRegions, the same projection playback and export already use: the aggregation primitive is a timeline-ordered list of (asset, source window, output window), and clips vs voiceover placements are just two providers of it.

Because every downstream consumer — the tab, word edits (#540), trim actions, agent grounding, captions — consumes the aggregate, nothing below the aggregator changes when the lane changes. User-facing, the question is simply: transcrire depuis l'enregistrement, ou depuis la voix off ?

Invariants

• The aggregate stays derived, never stored. The SSOT remains the per-asset transcripts; re-aggregation happens on every timeline edit; word edits land on the asset transcript and pass through with their wordIds — no re-minting (feat(document): add immutable transcript word edits #540 compatibility).
• Trims stay time-spans on the shared timeline — a cut driven by voiceover words goes through the same trim machinery as one driven by recording words. No new trim type.
• The music lane stays out of the selector, and out of transcription entirely (see above).
• Captions render the current aggregate — follow-by-default becomes structural, not a preference to keep in sync. An export-time override renders the other lane's aggregate, explicitly and visibly.

Phasing

1. Lane-parameterized aggregation + the voiceover lane + automatic transcription of imported audio; caption settings popover on the transcript tab; caption tab removed.
2. Export-time lane override for captions (named, flagged when it differs from the editing lane).
3. Merged captions for mixed-sound recordings (transcribe the exported mix, or interleave lane aggregates) — refinement, not a prerequisite.

Media panel

Imported audio should be listable/re-addable in the media panel (its own section), keeping the footage-only invariant for anything that makes clips.

Related: #543 (imported audio), #547 (lane keyboard activation), #540 (transcript word edits).

[EtienneLescot]

Design dependency flagged during review of this proposal: transcripts are becoming editable — #540 adds the document-layer word edit (setWordText, immutable, wordId-keyed, segment rebuilt from segment.wordIds; UI in a follow-up PR).

What that changes here — for the better:

• The unit is the asset's transcript, edited once, consumed everywhere. A word correction persists on the asset's transcript, and both selectors (editing source, subtitle source) read the same edited view. There is no per-surface copy of the words, so the two selectors can stay independent because the document underneath is shared.
• The subtitle source renders the edited transcript of the chosen source — corrections are the caption truth, which is the stated goal of feat(document): add immutable transcript word edits #540 (correct native OpenScreen subtitles safely). Phase 2 should build on the edited view, not the raw STT output.
• Anchors survive edits by construction: feat(document): add immutable transcript word edits #540 keys edits by wordId and keeps word identity, so word→timeline anchoring — which trim-on-word and caption timing both depend on — is stable across corrections. A voiceover transcript gets the same guarantee for free as long as phase 1 stores it in the same per-asset shape.

Sequencing implication: phase 1 (STT on imported audio + the tab selector) should land after or on top of #540's model, so the voiceover transcript is editable from day one rather than retrofitted.

[EtienneLescot]

Design revision (from the same discussion): the original phase 2 above described two independent persisted selectors — that was wrong. Since the transcript tab is the only surface where words are corrected (#540's UI), an independently persisted subtitle source can silently diverge from the transcript being corrected, and the corrections would never reach the captions. The body now reads: one working selection (the tab: word edits, trim actions, agent grounding) and a subtitle source that follows it by default, overridable — explicitly, stickily, and visibly — in the export dialog, which names the active source and flags it when it differs from the editing source. Divergence is never invisible.

[EtienneLescot]

Architecture revision 2 (agreed with @EtienneLescot): the body now specifies the aggregation-layer design — one transcript tab (caption settings become its popover, the caption tab and its duplicate transcribe entry point are removed), automatic asset-level transcription for all media, and the user choice expressed as which lane feeds the aggregated transcript (recording clips vs voiceover pills, the latter sourced from placeAudioRegions windows). Downstream consumers see only the aggregate, so word edits (#540), trims and agent grounding are lane-agnostic by construction.