AI Learning lab

.dev.vars.example

@@ -0,0 +1,3 @@
+GEMINI_API_KEY=replace-with-a-real-google-gemini-key
+TOKEN_SECRET=replace-with-at-least-32-random-characters
+ENCRYPTION_KEY=replace-with-at-least-32-random-characters

.gitignore

@@ -0,0 +1,7 @@
+.dev.vars*
+!.dev.vars.example
+.wrangler/
+__pycache__/
+*.pyc
+node_modules/
+dist/

README.md

@@ -1,2 +1,197 @@
-# learnpilot
-AI-powered personalized learning lab that adapts to each learner in real time. Combines natural language processing, adaptive curricula, intelligent tutoring, and progress tracking to create a dynamic educational experience with interactive explanations, guided practice, and continuous feedback.
+# Mentora
+
+Mentora is an adaptive AI tutor built with Cloudflare Python Workers. It combines a conversational tutor, retrieval-augmented generation (RAG), progress tracking, prerequisite suggestions, and spaced review.
+
+## How it works
+
+```text
+PDF or pasted notes
+        |
+        v
+Browser extracts PDF text with PDF.js
+        |
+        v
+Workers AI creates 768-dimensional embeddings
+        |
+        +--> Vectorize stores embeddings and searchable metadata
+        +--> D1 stores the full text chunks and ownership records
+
+Chat question
+        |
+        v
+Workers AI embeds the question
+        |
+        v
+Vectorize retrieves matching, user-owned chunks
+        |
+        v
+Gemini generates the tutor response using the retrieved context
+```
+
+The original PDF file is not stored. PDF text is extracted in the browser and sent to the Worker. The full extracted chunks are stored in D1, while embeddings and chunk metadata are stored in Vectorize.
+
+## Cloudflare resources
+
+The resource bindings are defined in [`wrangler.toml`](wrangler.toml).
+
+- Worker: `mentora`
+- D1 database: `mentora_db`
+- Vectorize index: `mentora-embeddings`
+- Vectorize dimensions: `768`
+- Embedding model: `@cf/baai/bge-base-en-v1.5`
+- KV namespace: configured in `wrangler.toml`
+- Static assets: `public/`
+
+R2 is not configured or used. No PDF bucket is required.
+
+## Local setup
+
+1. Install Node.js and Wrangler.
+
+2. Create the local secrets file:
+
+   PowerShell:
+
+   ```powershell
+   Copy-Item .dev.vars.example .dev.vars
+   ```
+
+   Set these values in `.dev.vars`:
+
+   ```text
+   GEMINI_API_KEY=your-gemini-api-key
+   TOKEN_SECRET=at-least-32-random-characters
+   ENCRYPTION_KEY=at-least-32-random-characters
+   ```
+
+   Never commit `.dev.vars`.
+
+3. Apply local D1 migrations:
+
+   ```powershell
+   npx wrangler d1 migrations apply mentora_db --local
+   ```
+
+   Or, from Bash/Git Bash/WSL:
+
+   ```bash
+   bash migrate.sh --local
+   ```
+
+4. For a fully local Worker preview, run:
+
+   ```powershell
+   npx wrangler dev --local
+   ```
+
+   For real Workers AI, Vectorize, and remote D1 resources, run:
+
+   ```powershell
+   npx wrangler dev --remote
+   ```
+
+   Remote development uses the configured remote resources and may incur Cloudflare and Gemini usage.
+
+## Vectorize setup
+
+The Vectorize index must remain configured with 768 dimensions for the current embedding model. Do not recreate it with a different dimension.
+
+Check the index:
+
+```powershell
+npx wrangler vectorize list
+```
+
+Metadata filtering requires indexes for the fields used by retrieval. Check them:
+
+```powershell
+npx wrangler vectorize list-metadata-index mentora-embeddings
+```
+
+If `user_id` or `concept_id` is missing, create the missing index once:
+
+```powershell
+npx wrangler vectorize create-metadata-index mentora-embeddings --propertyName=user_id --type=string
+npx wrangler vectorize create-metadata-index mentora-embeddings --propertyName=concept_id --type=string
+```
+
+If metadata indexes were created after material was uploaded, re-upload those materials so their metadata is available for filtered retrieval.
+
+## Production setup
+
+### 1. Verify production resources
+
+Make sure these resources exist in the Cloudflare account referenced by `wrangler.toml`:
+
+- `mentora_db`
+- `mentora-embeddings`
+- The configured KV namespace
+
+### 2. Apply production migrations
+
+First check migration status:
+
+```powershell
+npx wrangler d1 migrations list mentora_db --remote --env production
+```
+
+The Wrangler build hook runs `bash migrate.sh` before deployment and applies
+pending migrations to the remote database. To apply them manually instead:
+
+```powershell
+npx wrangler d1 migrations apply mentora_db --remote --env production
+```
+
+Do not run `schema.sql` manually against a database managed by migrations.
+`migrate.sh --local` targets local D1; running `migrate.sh` without arguments
+targets the remote D1 database.
+
+If the database was previously changed by manually executing SQL files, inspect the migration status and database schema before applying pending migrations.
+
+### 3. Configure production secrets
+
+`.dev.vars` is only for local development. Set production secrets with Wrangler:
+
+```powershell
+npx wrangler secret put GEMINI_API_KEY --env production
+npx wrangler secret put TOKEN_SECRET --env production
+npx wrangler secret put ENCRYPTION_KEY --env production
+```
+
+Use different strong values for production authentication and encryption secrets. `TOKEN_SECRET` and `ENCRYPTION_KEY` must each be at least 32 characters.
+
+### 4. Validate and deploy
+
+Build-check the production configuration without deploying:
+
+```powershell
+npx wrangler deploy --dry-run --env production
+```
+
+Deploy the Worker:
+
+```powershell
+npx wrangler deploy --env production
+```
+
+The Wrangler build hook applies pending D1 migrations before deployment.
+
+
+## Data lifecycle
+
+- The browser temporarily holds the selected PDF while extracting text.
+- D1 stores the authenticated user's full text chunks and source information.
+- Vectorize stores the embedding, ownership metadata, concept metadata, and chunk text used for retrieval.
+- Gemini receives the chat message and retrieved excerpts to generate the response.
+- Deleting a source removes its Vectorize vectors and D1 rows.
+- The original PDF binary is not retained anywhere by this project.
+
+## Tests
+
+Run the full test suite from the repository root:
+
+```powershell
+python -m unittest discover -s tests -v
+```
+
+The tests cover authentication, chunk ordering, batch embeddings, Vectorize metadata, D1 inserts, retrieval ownership, deletion batching, and ingestion cleanup.

migrate.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+set -e
+
+if [ "$1" = "--remote" ]; then
+  wrangler d1 migrations apply mentora_db --remote --env production
+elif [ "$1" = "--local" ] || [ -z "$1" ]; then
+  wrangler d1 migrations apply mentora_db --local
+else
+  echo "Usage: $0 [--local|--remote]" >&2
+  exit 2
+fi

migrations/0001_schema.sql

@@ -0,0 +1,72 @@
+CREATE TABLE IF NOT EXISTS users (
+    id TEXT PRIMARY KEY,
+    username_hash TEXT UNIQUE,
+    email_hash TEXT UNIQUE,
+    name TEXT,
+    username TEXT,
+    email TEXT,
+    password_hash TEXT,
+    role TEXT DEFAULT 'member' CHECK(role IN ('member', 'host', 'admin')),
+    email_verified INTEGER DEFAULT 0,
+    created_at TEXT DEFAULT (datetime('now'))
+);
+
+CREATE TABLE IF NOT EXISTS concept_node (
+    id TEXT PRIMARY KEY,
+    user_id TEXT REFERENCES users(id) ON DELETE CASCADE,
+    concept_id TEXT,
+    label TEXT,
+    activity_id TEXT,
+    mastery REAL DEFAULT 0.0 CHECK(mastery >= 0.0 AND mastery <= 1.0),
+    easiness REAL DEFAULT 2.5,
+    interval INTEGER DEFAULT 1,
+    due_date TEXT,
+    struggling INTEGER DEFAULT 0,
+    engage_pref TEXT DEFAULT '{}',
+    last_seen TEXT,
+    UNIQUE(user_id, concept_id)
+);
+
+CREATE TABLE IF NOT EXISTS learner_edge (
+    id TEXT PRIMARY KEY,
+    user_id TEXT REFERENCES users(id) ON DELETE CASCADE,
+    source_id TEXT REFERENCES concept_node(id) ON DELETE CASCADE,
+    target_id TEXT REFERENCES concept_node(id) ON DELETE CASCADE,
+    edge_type TEXT CHECK(edge_type IN ('requires-prereq', 'mastered', 'struggling-with')),
+    confidence REAL DEFAULT 1.0,
+    created_at TEXT DEFAULT (datetime('now'))
+);
+
+CREATE TABLE IF NOT EXISTS tutor_sessions (
+    id TEXT PRIMARY KEY,
+    user_id TEXT REFERENCES users(id) ON DELETE CASCADE,
+    concept_id TEXT REFERENCES concept_node(id) ON DELETE CASCADE,
+    mode TEXT CHECK(mode IN ('explain', 'socratic', 'practice')),
+    message_count INTEGER DEFAULT 0,
+    started_at TEXT DEFAULT (datetime('now')),
+    ended_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS tutor_messages (
+    id TEXT PRIMARY KEY,
+    session_id TEXT REFERENCES tutor_sessions(id) ON DELETE CASCADE,
+    role TEXT CHECK(role IN ('user', 'assistant')),
+    content TEXT,
+    created_at TEXT DEFAULT (datetime('now'))
+);
+
+CREATE TABLE IF NOT EXISTS content_chunks (
+    id TEXT PRIMARY KEY,
+    concept_id TEXT,
+    chunk_text TEXT,
+    vectorize_id TEXT,
+    source_label TEXT DEFAULT 'lesson',
+    created_at TEXT DEFAULT (datetime('now'))
+);
+
+CREATE INDEX IF NOT EXISTS idx_concept_user ON concept_node(user_id);
+CREATE INDEX IF NOT EXISTS idx_concept_due ON concept_node(user_id, due_date);
+CREATE INDEX IF NOT EXISTS idx_messages_session ON tutor_messages(session_id);
+CREATE INDEX IF NOT EXISTS idx_chunks_concept ON content_chunks(concept_id);
+CREATE INDEX IF NOT EXISTS idx_sessions_user ON tutor_sessions(user_id);
+CREATE INDEX IF NOT EXISTS idx_sessions_concept ON tutor_sessions(concept_id);

migrations/0002_hardening.sql

@@ -0,0 +1,17 @@
+-- Add ownership to newly ingested content. Existing rows remain NULL and are
+-- intentionally hidden by the API because their original owner cannot be
+-- determined safely; re-ingest those materials after upgrading.
+ALTER TABLE content_chunks ADD COLUMN user_id TEXT REFERENCES users(id) ON DELETE CASCADE;
+
+CREATE TABLE IF NOT EXISTS concept_reviews (
+    id TEXT PRIMARY KEY,
+    user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+    concept_node_id TEXT NOT NULL REFERENCES concept_node(id) ON DELETE CASCADE,
+    quality INTEGER NOT NULL CHECK(quality >= 0 AND quality <= 5),
+    mastery REAL NOT NULL CHECK(mastery >= 0.0 AND mastery <= 1.0),
+    created_at TEXT DEFAULT (datetime('now'))
+);
+
+CREATE INDEX IF NOT EXISTS idx_chunks_user_source ON content_chunks(user_id, concept_id, source_label);
+CREATE INDEX IF NOT EXISTS idx_reviews_user_date ON concept_reviews(user_id, created_at);
+CREATE INDEX IF NOT EXISTS idx_reviews_concept_date ON concept_reviews(concept_node_id, created_at);

migrations/0003_study_tools.sql

@@ -0,0 +1,31 @@
+CREATE TABLE IF NOT EXISTS tutor_quizzes (
+    id TEXT PRIMARY KEY,
+    user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+    concept_node_id TEXT NOT NULL REFERENCES concept_node(id) ON DELETE CASCADE,
+    mastery_before REAL NOT NULL CHECK(mastery_before >= 0.0 AND mastery_before <= 1.0),
+    question_count INTEGER NOT NULL CHECK(question_count > 0),
+    score INTEGER CHECK(score >= 0 AND score <= 100),
+    correct_count INTEGER CHECK(correct_count >= 0),
+    quality INTEGER CHECK(quality >= 0 AND quality <= 5),
+    mastery_after REAL CHECK(mastery_after >= 0.0 AND mastery_after <= 1.0),
+    created_at TEXT DEFAULT (datetime('now')),
+    completed_at TEXT
+);
+
+CREATE TABLE IF NOT EXISTS tutor_quiz_questions (
+    id TEXT PRIMARY KEY,
+    quiz_id TEXT NOT NULL REFERENCES tutor_quizzes(id) ON DELETE CASCADE,
+    question_order INTEGER NOT NULL,
+    prompt TEXT NOT NULL,
+    options_json TEXT NOT NULL,
+    correct_index INTEGER NOT NULL CHECK(correct_index >= 0),
+    explanation TEXT,
+    difficulty TEXT,
+    source_label TEXT,
+    UNIQUE(quiz_id, question_order)
+);
+
+CREATE INDEX IF NOT EXISTS idx_quizzes_user_concept
+    ON tutor_quizzes(user_id, concept_node_id, created_at);
+CREATE INDEX IF NOT EXISTS idx_quiz_questions_quiz
+    ON tutor_quiz_questions(quiz_id, question_order);

migrations/0004_progress_dashboard.sql

@@ -0,0 +1,5 @@
+CREATE TABLE IF NOT EXISTS user_streaks (
+    user_id TEXT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
+    streak_days INTEGER NOT NULL DEFAULT 0 CHECK(streak_days >= 0),
+    last_study TEXT
+);