🤖 JARVIS

A local assistant that learns from your own writing and conversations

A privacy-focused, locally running assistant that can learn from user-provided writing samples and conversations.

✨ Features

• 💬 Natural Chat Interface: Clean, modern UI for seamless conversations
• 🎤 Voice Input (STT): Speak to JARVIS using OpenAI Whisper - just like ChatGPT!
• 🔊 Voice Output (TTS): Hear JARVIS respond with natural-sounding voice using Microsoft Edge TTS
• 🧠 RAG-Powered Memory: Retrieves relevant context from past conversations using vector search
• 🎓 Personality Training: Learns your unique writing style from text samples
• 📊 Confidence Scoring: Shows how confident the AI is in each response
• 🔒 Local-first: LLM inference, memory, and embeddings run locally; voice output may use Microsoft Edge TTS
• 📝 Memory Management: View, search, and manage all stored conversations

🛠️ Tech Stack

• LLM: Ollama (llama3.2:3b) - Local inference
• Vector DB: ChromaDB - Semantic search and storage
• Embeddings: sentence-transformers (all-MiniLM-L6-v2)
• Voice Input: OpenAI Whisper - Speech-to-text (STT)
• Voice Output: Microsoft Edge TTS - Text-to-speech (TTS)
• UI: Gradio - Modern web interface
• Framework: LangChain - RAG orchestration

📦 Installation

Prerequisites

1. Install Ollama (https://ollama.com)

   # On Windows (PowerShell)
   winget install Ollama.Ollama
   
   # Or download from: https://ollama.com/download

2. Pull the LLM model

   ollama pull llama3.2:3b

Setup

1. Clone the project

   git clone <repository-url>
   cd jarvis

2. Install Python dependencies

   pip install -r requirements.txt

3. Create local configuration

   copy .env.example .env

   On macOS/Linux, use cp .env.example .env. Edit .env only if you need to change the defaults.

   Note: The first time you run the app, Whisper will automatically download the model (~150MB for base model). This only happens once.

4. Run the application

   python app.py

5. Open in browser

   • Navigate to: http://localhost:7860

Voice Chat Setup

Voice chat features are automatically enabled when dependencies are installed:

• Voice Input (STT): Uses OpenAI Whisper (automatically downloaded on first use)
• Voice Output (TTS): Uses Microsoft Edge TTS (works out of the box, no API key needed)

Voice Models: The app uses the "base" Whisper model by default (good balance of speed and accuracy). You can change this in app.py by modifying the VoiceHandler initialization.

🚀 Quick Start Guide

Step 1: Train Your Clone

1. Go to the 🎓 Training tab
2. Upload text files containing your writing:
   • Emails you've written
   • Chat message exports (WhatsApp, Discord, Slack)
   • Notes, blog posts, tweets
   • Any text in .txt, .md, or .json format
3. Click "Analyze & Train"
4. Review the personality analysis

Step 2: Start Chatting

1. Switch to the 💬 Chat tab
2. Text Chat: Type your message and press Enter or click Send
3. Voice Chat (Like ChatGPT!):
   • 🎤 Click "Voice Input" to speak your message (voice will be transcribed to text)
   • 🔊 Enable "Voice Output" checkbox to hear AI responses
   • Speak naturally - the AI will understand and respond!
4. Watch the confidence score - higher is better!
5. The AI will reference past conversations when relevant

Step 3: Manage Memory

1. Go to 🧠 Memory tab
2. View all stored conversations
3. Export your data
4. Clear memories if needed

📊 How It Works

┌─────────────────────────────────────────┐
│          User Input                     │
└──────────────────┬──────────────────────┘
                   │
┌──────────────────▼──────────────────────┐
│    Conversation Manager                 │
│  - Retrieves context from vector DB     │
│  - Builds enhanced prompt               │
│  - Generates response via Ollama        │
│  - Calculates confidence score          │
└──────────────────┬──────────────────────┘
                   │
        ┌──────────┴──────────┐
        │                     │
┌───────▼────────┐   ┌────────▼─────────┐
│   ChromaDB     │   │  Ollama LLM      │
│  Vector Store  │   │  llama3.2:3b     │
└───────┬────────┘   └────────┬─────────┘
        │                     │
        └──────────┬──────────┘
                   │
┌──────────────────▼──────────────────────┐
│          Response + Confidence          │
└─────────────────────────────────────────┘

🎯 Key Components

1. RAG Engine (core/rag_engine.py)

• Stores conversations in ChromaDB
• Retrieves relevant context using semantic search
• Manages training data

2. LLM Handler (core/llm_handler.py)

• Interfaces with Ollama
• Generates responses
• Handles streaming

3. Personality Analyzer (core/personality_analyzer.py)

• Extracts writing style patterns
• Analyzes tone, vocabulary, sentence structure
• Generates personality-aware system prompts

4. Confidence Scorer (core/confidence_scorer.py)

• Evaluates response quality
• Checks context relevance
• Detects uncertainty in responses

5. Conversation Manager (core/conversation_manager.py)

• Orchestrates all components
• Builds prompts with context
• Stores conversations

📁 Project Structure

my-jarvis/
├── app.py                      # Main Gradio application
├── requirements.txt            # Python dependencies
├── README.md                   # This file
├── .env.example               # Safe configuration template
│
├── core/
│   ├── conversation_manager.py  # Main orchestration
│   ├── rag_engine.py           # Vector DB & retrieval
│   ├── llm_handler.py          # Ollama interface
│   ├── personality_analyzer.py  # Style extraction
│   └── confidence_scorer.py     # Response scoring
│
├── utils/
│   ├── file_processor.py       # Process training files
│   └── config.py              # Configuration management
│
├── data/                      # Local runtime data (ignored by Git)
│
└── prompts/
    ├── system_prompt.txt      # Base personality prompt
    └── rag_prompt_template.txt # RAG query template

🔐 Privacy & Security

• Local-first: Core inference and storage run on your machine
• Local inference: Ollama, ChromaDB, and sentence-transformers run on your machine
• Your Data Stays Yours: Conversations and training data are stored locally
• Open Source: Full transparency - audit the code
• No Tracking: No analytics, no telemetry

Do not commit .env, personal writing samples, conversation exports, audio, vector databases, or correction history. The repository includes .env.example for safe setup. Whisper downloads a model on first use, and Edge TTS may send text to Microsoft’s speech service when voice output is enabled. See SECURITY.md for reporting guidance.

🤝 Contributing

See CONTRIBUTING.md for development and pull request guidance.

📄 License

This project is licensed under the MIT License.

🎓 Training Tips

Best Practices

1. Quantity Matters: Upload at least 500-1000 messages for good results
2. Quality Matters: Use text that represents your actual communication style
3. Variety Helps: Include different contexts (casual, formal, technical)
4. Recent Content: Your current writing style is most relevant

What to Upload

✅ Good Training Data:

• Personal emails you've written
• Chat message exports
• Blog posts or articles
• Social media posts
• Text messages
• Work communications

❌ Not Helpful:

• Text written by others
• Generic content
• Auto-generated text
• Very old writing (unless still relevant)

🐛 Troubleshooting

Ollama Connection Failed

Problem: "Ollama connection test failed"

Solution:

# Make sure Ollama is running
ollama serve

# Pull the model
ollama pull llama3.2:3b

# Test it works
ollama run llama3.2:3b "Hello"

Low Confidence Scores

Problem: AI always shows low confidence

Solution:

• Upload more training data
• Have more conversations to build memory
• Check that training files are in supported formats

Slow Responses

Problem: Responses take a long time

Solution:

• Use a smaller model (already using 3b)
• Close other applications
• Check CPU/RAM usage
• Consider GPU acceleration if available

🚀 Advanced Usage

Custom Model

Edit .env file:

OLLAMA_MODEL=llama3.2:1b  # Faster, less accurate
# or
OLLAMA_MODEL=llama3.2:8b  # Slower, more accurate

Adjust Confidence Threshold

In .env:

CONFIDENCE_THRESHOLD=0.6  # Lower = more lenient

Chunk Size for RAG

RAG chunking currently uses the built-in defaults of 500 words with 50 words of overlap. Change CHUNK_SIZE and CHUNK_OVERLAP in utils/config.py if you need different behavior.

📚 Use Cases

• Email Assistant: Draft emails in your style
• Meeting Stand-in: Answer questions as you would
• Content Creation: Generate text matching your voice
• Learning Tool: See what patterns define your writing
• Time Saver: Quick responses to common questions
• Backup Brain: Never forget past conversations

🎯 Demo Script

Perfect for presentations:

1. Show Privacy: "Everything runs locally - watch my data folder"
2. Upload Training: "Here are 1,000 messages I've written"
3. Ask Questions: "JARVIS, how would I explain RAG to a beginner?"
4. Show Confidence: "See? 87% confidence - it knows me!"
5. Show Memory: "It remembers past conversations automatically"

🛣️ Roadmap

• Voice clone integration
• WhatsApp export auto-parser
• Multi-language support
• Response A/B testing
• Mobile app
• Browser extension

🙏 Acknowledgments

• Ollama - Amazing local LLM runtime
• ChromaDB - Simple yet powerful vector database
• Gradio - Beautiful web UI framework
• LangChain - RAG implementation made easy

📞 Support

Having issues?

1. Check the Troubleshooting section
2. Make sure Ollama is running
3. Verify all dependencies installed
4. Check that you have enough disk space

---

Built with ❤️ for privacy and personalization

Your AI clone, your rules, your machine.