[V1 Phase 0 · P0] Support stereo and multi-channel inputs with one canonical mono stream

[douglasjarquin]

Status: Ready
Parent: #1
Audit baseline: 4258b8d2cfb98bb4809ad698028aeb905b0c732c
Priority: P0 — common microphones and audio interfaces are currently rejected before recording
Depends on: #76 for selected-device inventory and routing
Feeds: #99, #100, and #97

Problem

The production capture and transcription paths accept only an input configuration whose channel count is exactly one.

AudioRecorder.captureFormat() and AudioRecorder.start() reject formats where channelCount != 1, and TranscriptionService.start() repeats a mono-only guard. The recorder also serializes PCM to Speech through assumptions that are not a complete contract for interleaved, noninterleaved, and multi-buffer AudioBufferList layouts.

This rejects common valid hardware: stereo USB microphones, webcams, aggregate/virtual devices, and interfaces exposing two or many input channels. It is especially likely to affect the same users who need explicit microphone selection.

Required behavior

1. Oigo accepts ordinary mono, stereo, and multi-channel Core Audio input devices.
2. One explicit deterministic channel/downmix policy produces a canonical mono stream.
3. The same canonical stream is written to audio.caf and fed to Speech so live analysis and saved-audio retry consume equivalent audio.
4. No path assumes that the first AudioBuffer contains the complete frame set.
5. Unsupported or malformed layouts fail clearly before valuable audio is accepted.
6. Conversion remains bounded and real-time safe and hands canonical buffers to [V1 Phase 1 · P1] Move disk I/O and teardown off the audio tap callback #100's capture pipeline.
7. The selected device remains fixed for one recording operation.
8. [V1 Phase 1 · P1] Bound downstream Speech buffering and surface live-analysis degradation #97 converts canonical buffers to whatever public analyzer format Speech actually returns; it must not assume one hard-coded common format/interleaving layout without evidence.

Phased implementation plan

Phase A — Define the canonical capture contract

• Document the v1 durable/live format: mono PCM, sample format, sample-rate policy, frame ordering, and channel mapping.
• Define deterministic behavior for mono pass-through, stereo downmix, and devices exposing more than two channels.
• Decide whether v1 mixes all active channels or uses one explicit channel-selection rule; do not guess dynamically during a recording.
• Preserve duration, frame count, and timing across durable and Speech paths.

Phase B — Implement format adaptation once

• Introduce a capture-format adapter using public AVFAudio/Core Audio APIs selected by measurement.
• Convert each accepted hardware buffer into the canonical mono representation once.
• Correctly handle interleaved and noninterleaved layouts and every buffer in the AudioBufferList.
• Reuse bounded converter and buffer resources per recording; do not create unbounded objects on the tap callback.
• Hand canonical buffers to [V1 Phase 1 · P1] Move disk I/O and teardown off the audio tap callback #100's ordered producer pipeline so disk and Speech see the same accepted stream.

Phase C — Remove duplicated mono assumptions

• Replace raw-device channelCount == 1 guards in recorder and transcription startup with validation of the canonical post-conversion stream.
• Make AudioCaptureFormat describe what is actually forwarded to the downstream pipeline rather than merely the hardware's raw channel count.
• Ensure saved-audio retry reads the canonical file without a special alternate path.
• Surface actionable format/conversion failures through session metadata and History.
• Keep hardware-to-canonical conversion here and canonical-to-analyzer conversion in [V1 Phase 1 · P1] Bound downstream Speech buffering and surface live-analysis degradation #97; do not duplicate either conversion in the tap callback.

Phase D — Deterministic and native verification

• Add fixtures for mono, stereo interleaved, stereo noninterleaved, multi-buffer, and multi-channel PCM.
• Prove every intended source channel contributes according to the documented rule and no channel is silently truncated.
• Verify exact frame/order equivalence between durable audio and the Speech feed.
• Exercise analyzer formats returned by the supported Speech environment instead of assuming Int16/interleaved output.
• Native-test the built-in microphone plus at least one stereo or multi-input device/interface.
• Measure converter latency, callback/handoff pressure, memory, dropped buffers, CAF playability, and live/retry equivalence under [10/13 · Performance] Enforce resource budgets and lifecycle release gates #11/[11/13 · Quality] Complete reliability, compatibility, and dogfood validation #12.

Acceptance criteria

• A two-channel input no longer fails solely because it is stereo.
• A multi-input interface can complete a durable dictation using the documented channel policy.
• Mono behavior remains numerically equivalent within the documented conversion tolerance.
• Interleaved, noninterleaved, and multi-buffer fixtures prove all intended frames/channels are handled.
• The first AudioBuffer is never treated as the entire general input stream.
• audio.caf contains the canonical mono stream and remains playable after normal stop, interruption, and force quit.
• Live Speech and saved-audio retry consume equivalent ordered audio from the same recording.
• Transcription validates the canonical stream rather than rejecting the raw hardware channel count.
• Downstream analyzer conversion accepts the public format Speech returns and does not require an undocumented fixed common format/interleaving layout.
• Conversion resources are created once per recording and released on every terminal path.
• The tap callback performs no model work, network work, content logging, synchronous unbounded allocation, or newly added blocking I/O.
• Native performance remains within [10/13 · Performance] Enforce resource budgets and lifecycle release gates #11 budgets with no silent frame loss.

Out of scope

• User-configurable channel matrices
• Per-channel gain, phase, latency compensation, or isolated multitrack recording
• Noise suppression, automatic gain control, or denoising
• Automatic source switching during a recording
• Aggregate-device creation

Handoff

Implement after #76 establishes selected-device routing and before #99/#100/#97 consume the canonical buffer. Add stereo-interface, multi-channel, interleaved/noninterleaved, analyzer-format, retry-equivalence, interruption, and device-change rows to #12.