Skip to content
View ArPaN-DS's full-sized avatar

Block or report ArPaN-DS

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
ArPaN-DS/README.md


       

  

Who I Am

Data Scientist and AI Systems Engineer with an M.Sc. in Data Science from the University of Kalyani, specializing in production-ready AI that bridges advanced academic research and enterprise engineering.

I drive technical innovation across dual capacities: as a Data Scientist at Insightrix Consultancy, and as a core engineer within the RKMV R&D Cell in industry collaboration with FaceOff Technology. Through these institutional roles and specialized consulting engagements, I architect end-to-end intelligence systems—spanning speech forensics, multimodal deepfake detection, agentic RAG, and real-time telephony analysis.

Published researcher at ACL 2025 and CLEF 2025. Competed globally in multilingual NLP shared tasks.

"Whether leading R&D initiatives or consulting for commercial clients, I build systems that listen, reason, and deliver actionable intelligence—at scale and in real time."


Tech Arsenal

Category Technologies
Languages &
Core Tools
Speech &
Audio AI
NLP, LLMs
& GenAI
Computer
Vision
Backend,
Cloud & MLOps

ML & Data

How My Systems Work

An end-to-end intelligence pipeline — from raw wiretap audio to court-admissible forensic reports

Industry R&D — What I Build

Architecture-level highlights from production systems built through an industry R&D incubation program.
Descriptions are intentionally written at an architectural level to respect confidentiality while demonstrating technical depth.


Voice Forensics & Speaker Intelligence

  • Offline-first target voice extraction from degraded wiretap-quality recordings using speaker embeddings with adaptive quality gating
  • Dual-pass heuristic scanning (600s sparse → 1.5s dense) for multi-hour inference with dynamic peak/sustained threshold enforcement
  • Automated speaker segmentation with diarization-based identity canonicalization and cosine drift prevention
  • Court-admissible evidence packaging — isolated WAV clips + JSON metadata + timestamped PDF forensic reports
  • Browser-based forensic audio editor with multi-region waveform editing, effects chain, and undo/redo state management



PyTorch SpeechBrain ECAPA-TDNN Silero VAD pyannote Librosa Django Flask Celery Redis



PyTorch WavLM Wav2Vec2 MediaPipe ViT OpenCV FastAPI Twilio Django Channels WebSockets Docker

Deepfake Detection & Real-Time Threat Analysis

  • Multi-task audio deepfake detector with learnable 12-layer aggregation, Transformer encoder, and jointly predicted real/fake + codec + attack-family classification heads
  • Unified training across 4 benchmark datasets with mislabeled sample correction (+43% real data recovery) and production-grade augmentation (Opus/AMR codec, WebRTC effects, MUSAN noise)
  • 6-class speech emotion recognition (9,587 utterances, F1=0.71) with layer-wise attention and telephony-aware augmentation
  • Real-time telephony fraud screening — live call analysis with deepfake risk scoring, emotion inference, and multilingual forensic summaries
  • Multimodal meeting security — concurrent audio + video analysis over WebSocket streams with temporal smoothing and risk-level alerts

Conversational AI & Multimodal Assessment

  • Session-aware AI assistant for audio/video case investigation with analysis-conditioned RAG prompting and persistent video-linked chat history
  • Real-time voice-to-voice AI assessment platform with async orchestration running STT, speculative RAG pre-fetching, and continuous forensics as concurrent tasks
  • Tri-modal reasoning engine fusing audio forensics (Wav2Vec2), text forensics (LLM linguistic analysis), and visual forensics (micro-expression/gaze/posture) into structured agentic decisions
  • Bidirectional PCM audio streaming via Gemini Live API with auto-reconnection, fire-and-forget filler phrases, and sub-200ms perceptual latency



FastAPI WebSockets Gemini 2.5 Gemini Live API Milvus Wav2Vec2 Whisper Django asyncio



Qwen-32B-AWQ DeepSeek-R1-7B llama.cpp Milvus BGE-M3 BGE-Reranker HNSW Django

Agentic RAG & Knowledge Intelligence

  • Dual-LLM agentic architecture — Qwen-32B-AWQ (GPU, Flash Attention 2) as reasoning LLM + DeepSeek-R1-7B (CPU via llama.cpp) as critic model with singleton pattern
  • 3-stage hybrid retrieval: bi-encoder semantic search (HNSW on Milvus) → cross-encoder re-ranking → smart context stitching at sentence boundaries
  • ReAct agentic loop with custom tool registry, multi-angle evidence injection, thought-based loop detection, and fallback summaries
  • Query Cognition layer — LLM-powered intent classification, domain-jargon translation, and adaptive retrieval strategy injection
  • Deployed over streaming NDJSON endpoints with real-time reasoning trace visualization and session-aware memory

Research & Publications

ACL 2025 — SemEval Workshop (Vienna)
Multilingual claim retrieval using MiniLM + FAISS cross-lingual ranking — published in ACL Anthology
CLEF 2025 — Working Notes (Madrid)
Dual-encoder scientific discourse detection (SciBERT + Twitter-RoBERTa), Macro F1 = 0.8262 — published in CEUR-WS
DRDO-Funded Research
Multilingual Indian-language news claim verification (7000+ samples) with XLM-RoBERTa, MuRIL, GPT-4o Mini — under review

Open Source Work

Audio feature engineering (MFCCs, spectral contrast, chroma) + ML/DL for mood prediction. ~83% accuracy with Random Forest.

Python Librosa scikit-learn

Large-scale customer classification using MLP in PySpark with automated feature transformations and distributed preprocessing.

PySpark MLP Big Data

End-to-end fraud detection workflow with feature engineering, Random Forest achieving 94% accuracy, and precision/recall tuning.

Python scikit-learn Pandas

LLM-powered Reddit bot with OpenAI GPT for intelligent comment generation and real-time automated response loops.

Python OpenAI PRAW

More: News Headlines Scraper · Breast Cancer Diagnostic · Zomato Analysis · Share Trading Analysis

GitHub Analytics

Contribution Snake

Snake animation

Let's Connect

     

Open to conversations around speech AI, multimodal intelligence, NLP systems, LLM engineering, and applied research.

Pinned Loading

  1. JobScout-AI JobScout-AI Public

    The ultimate open-source, local-first AI agent for job search. Find, score, tailor, and track career opportunities using a multi-agent self-correcting QualityCritic loop.

    Python 2

  2. vidyamandira-faceoff/Audio_Cutter_Pro vidyamandira-faceoff/Audio_Cutter_Pro Public

    🎵 Fetch, edit & export audio, all in your browser. AI-powered music fetcher + full editor (cut, trim, fade, normalize, MP3/WAV). Flask + WaveSurfer.js. Zero cloud. Self-hosted.

    JavaScript 2

  3. Share-Trading-Data-Analysis Share-Trading-Data-Analysis Public

    Past 10 years reliance share treading data collection and analysis

  4. Bank-Customer-Classification-MLP Bank-Customer-Classification-MLP Public

    Classify bank customers using a Multilayer Perceptron (MLP) model with PySpark. Includes preprocessing, training, and evaluation on financial data.

    Jupyter Notebook

  5. breast-cancer-diagnostic breast-cancer-diagnostic Public

    This project implements a Breast Cancer Diagnostic System using R Programming. It involves data cleaning, exploratory data analysis (EDA), feature selection, Principal Component Analysis (PCA), and…

    Jupyter Notebook

  6. music-mood-classification music-mood-classification Public

    Music mood classification using ML and DL models such as Random Forest & used Feature extraction achieving ~83% accuracy.

    Jupyter Notebook