🎙️ OmniVoice AI — Voice-to-Table Restaurant Ordering & Live Kitchen Display System (KDS)

OmniVoice AI is an intelligent, multi-tenant restaurant automation platform that enables customers to place food orders naturally using voice. Powered by a Python Voice AI Agent, a Node.js/Express REST Backend, and a Real-Time Kitchen Display System (KDS) with Socket.IO, OmniVoice AI bridges spoken intent with kitchen fulfillment with zero friction.

---

🌟 Key Highlights

• 🗣️ Conversational Voice AI Agent: Natural language order taking in English/Hinglish with fuzzy menu entity extraction and portion recognition.
• ⚡ Real-Time Kitchen Display System (KDS): Instant bidirectional order telemetry using Socket.IO store rooms (kitchen:<storeId>).
• 🛡️ ReDoS-Safe Menu Matcher: Robust regex sanitization and alias resolution for diverse spoken phrases (e.g., "Butter Chicken", "Murgh Makhani", "CTM").
• 🏢 Multi-Tenant Architecture: Isolate menus, categories, orders, and live socket channels per store instance.
• 🍳 Live Kitchen Workflow: Interactive KDS cards supporting real-time status transitions (Draft ➔ Confirmed ➔ Preparing ➔ Ready ➔ Completed).
• 🔊 Audio Chimes & Visual Alerts: Audio-synthesized bell rings and pulse animations for incoming orders.

---

🏗️ System Architecture

flowchart TD
    Customer([👤 Customer / Voice Kiosk]) -->|Spoken Voice Input| VoiceAgent[🐍 Python Voice AI Agent]
    VoiceAgent -->|Fetch Menu Context| API_Context[GET /api/ai/menu-context/:storeId]
    API_Context --> NodeBackend[⚡ Node.js / Express Backend]
    
    VoiceAgent -->|Validate & Build Draft| API_Validate[POST /api/ai/validate-voice-order]
    API_Validate --> NodeBackend
    
    Customer -->|Confirm Order| API_Confirm[POST /api/orders/:orderId/confirm]
    API_Confirm --> NodeBackend
    
    NodeBackend -->|Persist Order Document| MongoDB[(🍃 MongoDB Database)]
    NodeBackend -->|Broadcast 'new_kitchen_order'| SocketServer[🔌 Socket.IO Server]
    
    SocketServer -->|Push to 'kitchen:storeId' Room| KDS[👨‍🍳 Live Kitchen Display System]
    KDS -->|Update State 'Preparing'/'Ready'| API_Status[PATCH /api/orders/:orderId/status]
    API_Status --> NodeBackend

Loading

---

📂 Project Structure

OmniVoice AI_SIH/
├── Backend/                        # Node.js & Express API Server
│   ├── public/                     # Static Web Assets
│   │   ├── socket.io.min.js        # Offline Socket.IO client library
│   │   └── test-kitchen.html       # Real-Time Kitchen Display System (KDS)
│   ├── src/
│   │   ├── config/                 # DB & Socket.IO initializers
│   │   ├── Controllers/            # Route business logic (AI, Orders, Menu, Stores)
│   │   ├── Middlewares/            # Centralized error handlers & validators
│   │   ├── Models/                 # Mongoose schemas (Store, Order, MenuItem, etc.)
│   │   ├── Routes/                 # Express API routes
│   │   ├── Utils/                  # API response & error wrappers
│   │   └── seedMenu.js             # Database seeder with sample restaurant menu
│   ├── server.js                   # Application entry point (Port 3000)
│   └── package.json
├── Backend_With_python/            # Python Voice AI Agent
│   └── ...                         # Voice recognition, entity parsing & AI pipeline
├── Frontend/                       # Client web application / Kiosk interface
├── .gitignore                      # Git exclusion rules
└── README.md                       # Documentation

---

🚀 Getting Started

1. Prerequisites

• Node.js (v18.x or higher)
• Python (v3.10 or higher)
• MongoDB (Local instance or MongoDB Atlas URI)

---

2. Backend Setup (Node.js & Socket.IO)

1. Navigate to the Backend directory:

   cd Backend

2. Install dependencies:

   npm install

3. Create a .env file in the Backend directory:

   PORT=3000
   MONGO_URI=mongodb://localhost:27017/omnivoice_db
   CORS_ORIGIN=*

4. Seed the database with sample menu and store data:

   npm run seed

   (Copy the output DEFAULT_STORE_ID for testing)

5. Start the backend server:

   npm run dev
   # or
   npm start

   Server runs on http://localhost:3000.

---

3. Kitchen Display System (KDS) Simulation

1. Ensure the backend server is running.
2. Open the KDS in your browser:

   http://localhost:3000/test-kitchen.html
   

3. Enter your Store ID (e.g. 65f1a2b3c4d5e6f7a8b9c0d1) and click Join Room.
4. The socket will connect and display incoming orders in real time.

---

4. Python Voice Agent Setup

1. Navigate to the Backend_With_python directory:

   cd Backend_With_python

2. Create and activate a virtual environment:

   # Windows
   python -m venv venv
   .\venv\Scripts\activate
   
   # Linux/macOS
   python3 -m venv venv
   source venv/bin/activate

3. Install required Python packages and launch the agent.

---

📡 API Reference

🤖 AI Voice Agent Endpoints

Method	Endpoint	Description
GET	/api/ai/menu-context/:storeId	Fetch formatted catalog context for LLM prompt injection
POST	/api/ai/validate-voice-order	Resolve spoken items against catalog, check stock & build draft order

📦 Order Endpoints

Method	Endpoint	Description
GET	/api/orders/:orderId	Get single populated order document
GET	/api/orders/store/:storeId	Get all orders for a specific store
POST	/api/orders/:orderId/confirm	Confirm order and broadcast to store kitchen room
PATCH	/api/orders/:orderId/status	Update order status (Preparing, Ready, Completed)

🏬 Store & Menu Endpoints

Method	Endpoint	Description
POST	/api/stores	Register new store instance
GET	/api/menu/:storeId	Retrieve complete categorized store menu
GET	/health	Service health & liveness probe

---

🔌 Socket.IO Events

Event Name	Direction	Payload / Description
join_kitchen_room	Client ➔ Server	storeId — Subscribes KDS client to kitchen:<storeId> room
joined_kitchen_ack	Server ➔ Client	Confirms room subscription
new_kitchen_order	Server ➔ Client	Emits confirmed order with line items, table number, total
order_status_updated	Server ➔ Client	Emits state change (Preparing, Ready, etc.)

---

👨‍💻 Contributing & License

Developed for the Smart India Hackathon (SIH).
Licensed under the ISC License.