Vox

AI-powered audio and video transcription platform. Converts media to timestamped transcripts, editable subtitles, summaries, and exportable files using faster-whisper and FFmpeg.

Overview

Transforms audio and video into timestamped text, editable subtitle segments, lightweight summaries, and downloadable exports. Backend handles transcription with faster-whisper, subtitle generation, and video burn-in via FFmpeg. Frontend provides a dark-themed dashboard for upload, editing, and export.

Core Architecture

flowchart LR
    User -->|uploads| Frontend["Next.js Dashboard"]
    Frontend -->|API calls| Backend["FastAPI Backend"]
    Backend -->|validates| FileSvc["File Service"]
    Backend -->|extracts audio| AudioSvc["Audio Service / FFmpeg"]
    AudioSvc -->|transcribes| WhisperSvc["Whisper Service / faster-whisper"]
    WhisperSvc -->|segments| SubtitleSvc["Subtitle Service"]
    SubtitleSvc -->|summarizes| SummarySvc["Summary Service"]
    Backend -->|burn-in| VideoSvc["Video Service / FFmpeg"]
    FileSvc -->|reads| Storage["Local Storage (uploads/ outputs/ temp/)"]

Loading

System Components

Component	Responsibility
frontend/	Next.js dashboard — upload, editor, export UI
backend/app/	FastAPI application and route definitions
backend/services/	File, audio, whisper, subtitle, summary, video services
backend/models/	Pydantic schemas
scripts/	Development and deployment scripts

Repository Layout

Directory	Purpose
frontend/	Next.js dashboard application
backend/	FastAPI machine-learning backend
scripts/	Development automation scripts

Technology Stack

Layer	Technology	Purpose
Frontend	Next.js + TypeScript	Dashboard and editor UI
Styling	TailwindCSS	UI styling
Animation	Framer Motion	Page transitions
Backend	FastAPI + Python	REST API and ML orchestration
ML	faster-whisper	Speech-to-text transcription
Media	FFmpeg	Audio extraction and subtitle burn-in
Storage	Local filesystem	Upload, output, and temp directories

Requirements

• Node.js 18+
• Python 3.10+
• FFmpeg
• NVIDIA GPU with CUDA (recommended) or CPU fallback

Configuration

File	Purpose
backend/.env	Backend configuration (model size, device, paths)
frontend/.env.local	Frontend API URL

Key backend variables: BACKEND_HOST, BACKEND_PORT, WHISPER_MODEL_SIZE, WHISPER_DEVICE (cuda/cpu), WHISPER_COMPUTE_TYPE, MAX_UPLOAD_SIZE_MB, UPLOAD_DIR, OUTPUT_DIR, TEMP_DIR.

Getting Started

# Backend
python -m venv backend/venv
backend/venv/bin/pip install -r backend/requirements.txt

# Frontend
cd frontend
npm install

# Run both
npm run dev:backend   # FastAPI on :8010
npm run dev:frontend  # Next.js on :3000

Development

npm run dev:backend:cpu   # CPU-only backend
npm run dev:frontend      # Next.js dev server

Request / Data Flow

sequenceDiagram
    participant User
    participant Frontend
    participant Backend
    participant FFmpeg
    participant Whisper

    User->>Frontend: Upload audio/video file
    Frontend->>Backend: POST /api/upload
    Backend->>Backend: Validate type and size
    Backend-->>Frontend: Upload confirmed

    User->>Frontend: Start transcription
    Frontend->>Backend: POST /api/transcribe
    Backend->>FFmpeg: Extract audio (if video)
    FFmpeg-->>Backend: WAV audio
    Backend->>Whisper: Transcribe segments
    Whisper-->>Backend: Timestamped segments
    Backend->>Backend: Generate TXT, SRT, VTT, JSON, CSV
    Backend-->>Frontend: Transcript + segments

    User->>Frontend: Edit subtitle timings
    Frontend->>Backend: POST /api/segments/update
    Backend->>Backend: Regenerate exports
    Backend-->>Frontend: Updated files

Loading