Open-source JARVIS-class AI that runs entirely on your hardware.
Speak · Gesture · Type — it sees your screen, controls your computer, runs automations,
visualizes data in 3D, and talks back with personality. No cloud lock-in. No subscriptions.
📖 Table of Contents
Friday is an open-source JARVIS-class AI that lives on your desktop. Speak to it, gesture at it, or type — it sees your screen, controls your computer, runs automations, visualizes data in 3D, and talks back with personality. Everything runs locally. Your API key, your LLM, your rules.
The 3D orb reacts to your voice. The Intelligence panel streams 10 live data sources. The Holodeck renders your metrics as animated 3D bars. Zen mode turns everything into a monochrome orb + chat. And it all starts with one command.
npm run friday # → boots API server + frontend together (Windows)
# or
docker compose up -d # → Frontend: http://localhost:5173 · Backend: http://localhost:8080|
🧘 Zen Mode A radical minimal UI — one monochrome orb + chat by default. ⌘B toggles to the full dashboard. Ambient floating widgets orbit the orb; hover/drag to expand. |
🖥️ Computer Control Open apps, focus windows, type, click, and summarize your desktop. "Organize my desktop" is one goal for the autopilot — every action asks for your confirmation first. |
|
🧩 Plugin Marketplace Install & remove community plugins from Settings. Ships with built-ins for screen, email, calendar, web, and system — plus a community plugin registry. |
🛠 Custom Tool Builder Describe a tool in natural language and Friday generates, registers, and persists it — no code required. |
|
📚 Local RAG Pipeline Ingest documents and search them with sentence-aligned chunking + lexical reranking on top of your three parallel memory engines. |
🧠 Knowledge Graph Entities & relations extracted from your chats. Every new session seeds context from the graph + diary — "Last time you were working on…" |
|
🚫 Blackout Mode One toggle for total privacy: network tools blocked, local Ollama forced, PRIVATE seal on the orb. |
⚡ Single-Command Start
|
Cloud assistants are convenient — and that's the problem. They live behind a website, own your conversation history, upload your screen on request to a vendor you didn't choose, and charge a subscription for features you could run yourself.
Friday is the alternative that puts you back in control:
- 🖥️ Desktop-first — it runs where you work. No tab required, no "sorry, I can only do that in the cloud" moments.
- 🔑 Bring your own LLM — plug in OpenRouter, OpenAI, Ollama, or any OpenAI-compatible endpoint. Your key, your provider, your billing, your rules. No Friday servers exist — there is nothing to charge you for.
- 🔒 Local by design — memory, automation definitions, and Google tokens live in a
memory_store/folder on your machine, not in someone else's database. See Privacy & Security. - 🎭 A personality, not a chatbot — three voice personas, ambient conversation, and a 3D orb that reacts to you. It feels like a companion, because that's the whole point.
- 🧩 Extensible — plugins, custom tools, and a planner that breaks big goals into executed steps. If you can script it, Friday can run it.
The pitch in one sentence: Friday is a JARVIS-class AI assistant you actually own — free, open source (MIT), and running entirely on your hardware.
🎥 A short GIF/video walkthrough of the orb, voice, and Holodeck is coming soon. In the meantime, the dashboard above shows the full interface — and the best demo is running it yourself (30 seconds, below).
git clone https://github.com/alimaandev/Friday.git
cd Friday
cp config/providers.toml.example config/providers.toml
# Edit config/providers.toml — paste your OpenRouter (or other) API key
npm run friday # boots API server + frontend, opens your browsergit clone https://github.com/alimaandev/Friday.git
cd Friday
cp config/providers.toml.example config/providers.toml
# Edit config/providers.toml — paste your API key
docker compose up -dgit clone https://github.com/alimaandev/Friday.git
cd Friday
pip install -r requirements.txt
cp config/providers.toml.example config/providers.toml
# Edit config/providers.toml — paste your API key
cd desktop && python api_server.py &
cd .. && npm install && npm run dev|
Procedural Three.js orb with 10 state-driven animation profiles. Plus a full 3D data visualization canvas — animated metric bars, ambient particle fields, orbital rings. Camera follows hand gestures in real time. |
10 real-time data modules: News, Weather, Stocks, Crypto, GitHub, Earthquakes, Space, World Clocks, CVE, Screen. All pushed via a single SSE connection — replaced 18 polling loops. |
|
Token-by-token responses with plan visualization and tool-call tracking. Three voice personas (JARVIS, FRIDAY, Cortana) with unique TTS rate/pitch and custom system prompts. Switch anytime via settings or ⌘K. |
Voice input/output with "Hey Friday" wake word (offline, in-browser). Ambient conversation mode — natural back-and-forth with auto-send on pause. Webcam hand-gesture control — open palm to speak, fist to send. Multi-language (English, Hindi, Urdu). |
|
Schedule recurring actions with cron expressions ("every weekday at 9am"). Natural language creation — say "create automation for daily briefing at 8am". Background engine checks every 30s and fires via SSE. Toggle, trigger manually, or delete from the Intelligence panel. |
Analyze your screen or webcam feed via LLM vision. Ask "what's on my screen" for instant description. Background SSE push auto-describes screen changes. Optional OCR via pytesseract. Capture camera frames from the Intelligence panel buttons. |
| Integration | Type | Details |
|---|---|---|
| Google Calendar | OAuth 2.0 | View upcoming events inline |
| Gmail | OAuth 2.0 | Unread count + inbox preview |
| Memory | TF-IDF + Jaccard + Vector | Cross-session semantic search |
| Knowledge Graph | Entity/relation extraction | Session continuity + proactive connections |
| Local RAG | Chunking + reranking | Document ingest + semantic retrieval |
| Computer Control | pyautogui + pywin32 | Open apps, type, click, windows, desktop summary |
| Plugin Marketplace | Manifest registry | Install/remove community plugins |
| Custom Tools | Natural-language builder | Persisted, code-free tool definitions |
| Proactive Alerts | SSE | System anomalies, reminders, notifications |
| LLM Providers | Pluggable | OpenRouter, OpenAI, Ollama, Anthropic, custom |
| Morning Pulse Briefing | Template | Daily weather, news, crypto, calendar, email summary |
| Automations Engine | Cron + SSE | Conditional triggers with background scheduler |
| Vision | LLM + PIL | Screen capture, camera frames, optional OCR |
Friday is designed so that your data stays yours. Here is exactly what happens with it — no fine print.
| Data | Location | Notes |
|---|---|---|
| API keys | config/providers.toml |
Only read by Friday to call the provider you configured. Never uploaded anywhere. |
| Conversation memory | memory_store/long_term.json |
Plain JSON on your disk — easy to inspect, back up, or delete. |
| Vector/embedding index | memory_store/vector_store.pkl, embeddings.pkl |
Local search indexes for semantic recall. |
| Google OAuth tokens | memory_store/ |
Stored locally after you authorize Calendar/Gmail. |
| Custom tools | memory_store/custom_tools.json |
Tool definitions you build stay on your machine. |
| Blackout state | memory_store/blackout.json |
Privacy toggle state is persisted locally. |
- Only calls to your configured LLM provider (OpenRouter, OpenAI, Ollama, etc.) — and only the content you ask Friday to process. If you run Ollama locally, nothing related to AI leaves your machine at all.
- Live data modules (news, weather, stocks, crypto, CVE, etc.) fetch public APIs — standard HTTP requests, same as any dashboard.
- Google Calendar/Gmail are only contacted when you use those features, under your own OAuth consent.
- Screen & camera analysis — your screen is captured only when you ask for it, and only the frames you request are sent to your provider for vision analysis. The background screen-change monitor runs entirely locally (it only compares image hashes — it never uploads pixels).
- Voice & wake word — "Hey Friday" detection and speech-to-text run offline in your browser. No audio is uploaded.
- Computer control — click, type, and window actions run locally. Control tools always require your confirmation first.
Flip one toggle in Settings and Friday goes fully local: all network tool calls are blocked, the provider is forced to Ollama, and a PRIVATE seal appears on the orb. Zero outbound AI traffic. Learn more →
- 🗑️ Wipe it all: delete the
memory_store/directory and yourconfig/providers.toml. - 🔄 Go fully local: switch the provider to Ollama — AI processing stops leaving your machine entirely.
- 👀 Audit it: memory is plain JSON. Open it and see exactly what Friday remembers about you.
- 🚫 No telemetry — Friday has no analytics, no crash reporters, and no phone-home code.
Friday/
├── core/ # Business logic
│ ├── executor.py # Tool-call execution & streaming
│ ├── computer.py # Desktop control (apps, windows, input)
│ ├── rag.py # Local document chunking + reranking
│ ├── knowledge.py # Entity/relation knowledge graph
│ ├── plugin_store.py # Plugin marketplace manifest registry
│ ├── custom_tools.py # Natural-language tool builder
│ ├── blackout.py # Local-only privacy mode
│ ├── hotkey.py # System-level global hotkey
│ ├── autopilot.py # Goal decomposition → executed steps
│ ├── memory/ # Three parallel memory engines (TF-IDF, vector, embeddings)
│ ├── proactive.py # Background monitors → SSE alerts
│ ├── automations.py # Cron-triggered automations engine
│ ├── vision.py # Screen/camera capture + analysis
│ └── auth/ # Google OAuth (Calendar, Gmail)
├── providers/ # LLM provider abstraction (pluggable)
│ ├── registry.py # Auto-registration
│ ├── openai_compat.py # OpenRouter/OpenAI/any OpenAI-compatible API
│ └── ollama.py # Local models
├── plugins/ # Tool plugins (auto-discovered at startup)
│ ├── builtins/ # Screen, email, calendar, web, system, computer…
│ ├── community/ # Community plugin registry (marketplace)
│ └── manifest.json # Plugin manifests
├── agent/ # Agent core (goals, planning, desktop context)
├── voice/ # Voice I/O & wake word (offline, in-browser)
├── browser/ # Headless browser automation
├── desktop/ # Frontend (React 19 + Three.js + Vite + Tailwind)
│ ├── src/ # Components, stores, hooks
│ ├── public/ # Assets & feature images
│ └── api_server.py # Quart backend (REST + SSE, port 8080)
├── config/ # providers.toml — your API keys & providers
├── memory_store/ # Local memory (JSON + pickle) — created at runtime
├── docs/ # API reference & v4 plan
└── tests/ # 270+ pytest tests · desktop/src/test/ (107 vitest)
All configuration lives in a single file: config/providers.toml (copy from config/providers.toml.example).
| Setting | Default | Description |
|---|---|---|
[default] provider |
"openrouter" |
Active provider — openrouter, openai, ollama, or any OpenAI-compatible API |
[openrouter] api_key |
— | Your OpenRouter key |
[openrouter] model |
"openrouter/free" |
Model to use (free tier by default) |
[openrouter] fallback_model |
"meta-llama/llama-3.2-3b-instruct:free" |
Used when the primary model is unavailable |
[openai] base_url |
"https://api.openai.com/v1" |
Any OpenAI-compatible endpoint |
[ollama] base_url |
"http://localhost:11434" |
Local Ollama server |
temperature |
0.7 |
Response randomness (per provider) |
max_tokens |
4096 |
Max response length (per provider) |
[embeddings] engine |
"sentence" |
Memory embedding engine |
Switch providers in three steps:
- Paste the API key in the provider's section
- Change
[default] providerto that provider's name - Restart the backend — done
A single-page React frontend communicates with a Python Quart backend via SSE and REST. The backend pools connections to any OpenAI-compatible LLM provider. Three parallel memory engines (TF-IDF, Jaccard, Vector) enable cross-session semantic recall, now augmented with a local RAG pipeline and a knowledge graph.
Everything is tested and linted — CI runs both suites on every pull request.
pip install -r requirements.txt
python -m pytest tests/ -v # 270+ tests
python -m pytest tests/ --cov # with coverage report (CI enforces a 50% gate)
ruff check . # lint
ruff format --check . # format checkcd desktop
npm install
npm run test # 107 vitest tests
npm run lint # oxlint
npx tsc --noEmit # type check
npm run build- API server —
desktop/api_server.py(Quart, port 8080); module-level code runs at import time, so tests patchcore.registry.discover_pluginsanddesktop.api_server._proactivebefore import - Adding a tool — create a plugin class in
plugins/builtins/extendingToolPlugin; it's auto-discovered at startup - Adding a plugin to the marketplace — add a manifest to
plugins/manifest.jsonand a package underplugins/community/ - Adding an endpoint — define the route in
desktop/api_server.py, use@require_authfor authenticated endpoints - Memory — three engines run in parallel: keyword (TF-IDF), vector (cosine), embeddings (sentence-transformers)
- Conventions — PEP 8, line length 120,
async/awaitfor I/O,asyncio.to_threadfor blocking calls; conventional commits (feat:,fix:,test:…)
Does Friday work offline? Partially. The frontend, voice, wake word, memory, automations, and computer control run locally. LLM responses need an LLM provider — use Ollama and everything runs 100% offline.
Which LLMs can I use?
Any OpenAI-compatible API: OpenRouter (default), OpenAI, Anthropic, Ollama, or a custom base_url. Bring your own key.
Is my screen data sent to an AI company? Only when you ask Friday to analyze the screen, and only to the provider you configured. The background screen-change monitor compares hashes locally and never uploads pixels. Computer control actions always ask for your confirmation first.
What is Blackout mode? One toggle that blocks all network tools, forces the Ollama provider, and shows a PRIVATE seal on the orb — total local-only privacy. Perfect for sensitive work.
How do I make Friday fully local?
Set [default] provider = "ollama" in config/providers.toml, or just enable Blackout mode from Settings.
Can I customize Friday's personality? Yes — three voice personas (JARVIS, FRIDAY, Cortana) ship built-in, each with its own TTS voice and system prompt. Switch anytime via settings or ⌘K.
Can I build my own tools? Yes — the Custom Tool Builder turns plain-English descriptions into working, persisted tools. And the Plugin Marketplace lets you install community plugins in one click.
What are automations? Cron-scheduled actions you create in natural language ("create automation for daily briefing at 8am"). The background engine checks every 30s and fires results via SSE.
Is Friday free? Yes — MIT licensed and free forever. You only pay your LLM provider if you use a paid one (Ollama is free).
How do I uninstall / wipe my data?
Delete memory_store/ and config/providers.toml, then stop the containers (docker compose down).
| Release | Status | Highlights |
|---|---|---|
| v1 | ✅ Shipped | Core chat, 3D orb, intelligence dashboard |
| v2 | ✅ Shipped | Async backend, SSE push, voice/gesture, Google integration, performance overhaul |
| v3 | ✅ Shipped | Voice personality, cron automations, screen/camera vision, Holodeck 3D viz, ambient conversation |
| v4 | ✅ Shipped | Zen mode, computer control, plugin marketplace, custom tools, local RAG, knowledge graph, blackout mode |
| v5 | 📋 Backlog | Desktop app (Tauri), offline-first, multi-user mode |
Track progress on the project board and open issues.
We welcome contributions of all sizes — from typo fixes to new features.
- Read the guide → Your First Contribution walks you through the full flow
- Pick a task → Browse the Good First Issues project or the
good first issuelabel - Claim it → Comment on the issue so others know you're on it
- Run locally →
python main.py --ui(backend) +npm run dev(frontend) - Open a PR → We'll review fast and guide you through any changes
| Contributor | Contribution |
|---|---|
| @mingmaaa | Spanish & French README translations |
| @ravindharann | Dashboard & UI screenshots |
| @kaminimangal | RAG markdown/txt file ingestion (in progress) |
| @rookepoole | Windows open_app PATH resolution |
| @surajthedev | Async persistence for long-term memory |
| @NikhilVedak | API route versioning with /api/v1 prefix |
| @MasRama | Cleaned up unused CSS and animation classes |
If Friday is useful to you, consider supporting the project:
- ⭐ Star the repo — it helps others discover it
- 🐛 Report bugs or request features via issues
- 🤝 Contribute code via pull requests
- 💰 Sponsor via GitHub Sponsors
MIT — use it, modify it, ship it. See LICENSE for details.
Built with ❤️ by alimaandev · Discussions · Issues