- 🎙️ Audio & Video Upload - Drag and drop support for common formats (MP3, WAV, MP4, MOV)
- 🔌 Pluggable Providers - Provider-agnostic architecture (AssemblyAI implemented)
- 📝 Word-Level Timestamps - Precise timestamp tracking for every word
- 🎯 Interactive Transcript - Click any word to seek media playback
- 🔍 Raw Data Access - View original provider responses for debugging
- 🎨 Normalized Schema - Provider-agnostic transcript format for UI consistency
graph TB
Client[React Client]
Server[Express Server]
Registry[Provider Registry]
AI[AssemblyAI Provider]
Client -->|Upload File| Server
Client -->|Request Transcription| Server
Server -->|Route to Provider| Registry
Registry -->|Delegate| AI
AI -->|Normalize & Return| Server
Server -->|Transcript + Raw| Client
classDef frontend fill:#e1f5ff,stroke:#01579b
classDef backend fill:#fff3e0,stroke:#e65100
classDef provider fill:#f3e5f5,stroke:#4a148c
class Client frontend
class Server,Registry backend
class AI provider
pulseclip/
├── client/ # React frontend
│ ├── src/
│ │ ├── components/ # UI components
│ │ │ ├── FileUpload.tsx
│ │ │ ├── MediaPlayer.tsx
│ │ │ └── TranscriptViewer.tsx
│ │ ├── App.tsx # Main application
│ │ └── types.ts # TypeScript types
│ └── package.json
├── server/ # Express backend
│ ├── src/
│ │ ├── providers/ # Transcription providers
│ │ │ ├── assemblyai.ts
│ │ │ └── registry.ts
│ │ ├── types/ # TypeScript types
│ │ │ └── transcription.ts
│ │ ├── cache.ts # Transcription cache
│ │ ├── featured.ts # Featured pulses management
│ │ └── index.ts # Server entry point
│ ├── artipods/ # Artipod storage (gitignored)
│ │ └── {uuid}/ # Each artipod folder contains:
│ │ ├── media.ext # Original media file
│ │ ├── thumbnail.png # Thumbnail image
│ │ └── (future: transcript.json, beats.json, short.mp4)
│ ├── data/ # Persistent data
│ │ └── featured.json # Featured pulses list
│ └── package.json
└── package.json # Workspace root
- Node.js 18+ and npm
- AssemblyAI API key (get one at assemblyai.com)
- Clone the repository:
git clone <repository-url>
cd voicepoc-- Install dependencies:
npm install- Set up environment variables:
cd server
cp .env.example .env
# Edit .env and add your ASSEMBLYAI_API_KEY- Start the development servers:
# From the root directory
npm run devThis will start:
- Server on http://localhost:3001
- Client on http://localhost:3000
- Open http://localhost:3000 in your browser
- Drag and drop an audio or video file (or click to browse)
- Select "AssemblyAI" from the provider dropdown
- Click "Transcribe" and wait for processing
- Click any word in the transcript to seek to that timestamp
Returns list of available transcription providers.
Response:
{
"providers": [
{
"id": "assemblyai",
"displayName": "AssemblyAI"
}
]
}Upload a media file. Creates a new artipod with a UUID.
Request: multipart/form-data with file field
Response:
{
"success": true,
"artipodId": "61dd3471-dd98-4ff3-a5ae-27afee3fc8af",
"filename": "audio.mp3",
"url": "/artipods/61dd3471-dd98-4ff3-a5ae-27afee3fc8af/audio.mp3",
"size": 1234567,
"mimetype": "audio/mpeg"
}Get artipod info by UUID.
Response:
{
"success": true,
"artipodId": "61dd3471-dd98-4ff3-a5ae-27afee3fc8af",
"filename": "audio.mp3",
"url": "/artipods/61dd3471-dd98-4ff3-a5ae-27afee3fc8af/audio.mp3",
"size": 1234567,
"thumbnail": "/artipods/61dd3471-dd98-4ff3-a5ae-27afee3fc8af/thumbnail.png"
}Transcribe media file using selected provider.
Request:
{
"mediaUrl": "/artipods/61dd3471-dd98-4ff3-a5ae-27afee3fc8af/audio.mp3",
"providerId": "assemblyai",
"options": {
"speakerLabels": false
}
}Response:
{
"success": true,
"provider": {
"id": "assemblyai",
"displayName": "AssemblyAI"
},
"transcript": {
"durationMs": 120000,
"words": [
{
"text": "Hello",
"startMs": 100,
"endMs": 500,
"confidence": 0.95
}
]
},
"raw": { /* Original provider response */ }
}The system uses a provider-agnostic schema:
interface Transcript {
durationMs: number;
speakers?: Speaker[];
words: TranscriptWord[];
segments?: TranscriptSegment[];
}
interface TranscriptWord {
text: string;
startMs: number;
endMs: number;
speakerId?: string;
confidence?: number;
}To add a new transcription provider:
- Create a new provider class in
server/src/providers/:
import { TranscriptionProvider, ProviderResult } from '../types/transcription';
export class MyProvider implements TranscriptionProvider {
id = 'my-provider';
displayName = 'My Provider';
async transcribe(mediaUrl: string, options?: any): Promise<ProviderResult> {
// Call provider API
const response = await callProviderAPI(mediaUrl);
// Normalize to common schema
const normalized = this.normalize(response);
return {
normalized,
raw: response
};
}
private normalize(response: any): Transcript {
// Convert provider response to normalized schema
}
}- Register the provider in
server/src/providers/registry.ts:
export function initializeProviders(): ProviderRegistry {
const registry = new ProviderRegistry();
// Register new provider
const myProviderKey = process.env.MY_PROVIDER_API_KEY;
if (myProviderKey) {
registry.register(new MyProvider(myProviderKey));
}
return registry;
}- Google Medical STT provider integration
- Speaker diarization visualization
- Word-level editing capabilities
- Server-side timestamp alignment verification
- Export to EDL/subtitle formats
- HIPAA-grade deployment with BAA-covered providers
ISC