Lumen

A lightweight, AI-native data platform. Connect databases, model metrics in code, ask questions in natural language — get answers grounded in a governed semantic layer.

See PRODUCT_REPORT.md for product strategy and IMPLEMENTATION_PLAN.md for engineering detail.

Quick start (local dev)

Prereqs: Docker + Docker Compose, Node 20+, Python 3.12+. No AWS credentials needed for the smoke test (mock LLM mode).

Option A — Docker (recommended)

make install            # Python venv + npm install (host-side, for tests/lint)
make seed-lending       # consumer-lending fixture (~1.9 GB) — bind-mounted into the containers
make docker-up          # builds + starts gateway, ai, query, workspace, frontend
                        # → http://localhost:5173

make docker-logs        # tail logs
make docker-down        # stop the stack

Option B — Run on host

make install            # backend venv + frontend deps
make seed-lending       # or: make seed-lending-small (5%, ~100 MB)
make backend            # 4 uvicorn services in the background, logs in /tmp/lumen-logs/
make frontend           # Vite on http://localhost:5173

make smoke              # AI smoke test (mock LLM, no API keys needed)

Configure secrets (optional, only for real LLM calls)

# config/secrets.local.yaml is gitignored. Fill in whichever provider you'll use:
#   llm.bedrock.aws_access_key_id / aws_secret_access_key
#   llm.anthropic.api_key
#   llm.alibaba.api_key
# Default `make smoke` and `make docker-up` use USE_MOCK_LLM=true so this is optional.

Repo layout

lumen/
├── PRODUCT_REPORT.md           # Strategy
├── IMPLEMENTATION_PLAN.md      # Engineering plan
├── config/
│   ├── settings.yaml           # Defaults (committed)
│   ├── secrets.yaml            # Schema for required secrets, with empty values (committed)
│   ├── settings.local.yaml     # Local overrides (gitignored)
│   └── secrets.local.yaml      # Local API keys (gitignored)
├── backend/                    # The platform — connect any frontend here
│   ├── shared/                 # Internal SDK (LLM client, auth, audit, observability)
│   ├── services/               # api_gateway, ai_service, query_service, ...
│   └── cube/                   # Cube semantic layer (config + schemas)
├── frontend/                   # Reference web app (React + Vite + Observable Plot)
├── local_test/                 # Local test scheme with consumer-lending fixture
└── docs/                       # Tutorials, API ref, runbooks

Configuration

Lumen uses two YAML files in config/, each with an optional .local.yaml override:

• settings.yaml — non-secret config: LLM provider tier mapping, task routing, workspace presets, feature flags, observability.
• secrets.yaml — schema for required secrets (API keys, JWT signing key, etc.) with placeholder values. Real values go in secrets.local.yaml for development.

The *.local.yaml files are gitignored. The platform merges local over base on load.

LLM providers

The platform supports multiple model providers, configurable in settings.yaml:

Provider	Models we map
AWS Bedrock	Claude (Opus / Sonnet / Haiku)
Anthropic API	Claude (direct API)
Alibaba DashScope	Qwen (qwen-max / qwen-plus / qwen-turbo)

At service startup, each configured provider is health-checked with a minimal call. Providers that fail authentication are marked unavailable and won't be routed to until restart.

Architecture (one-liner)

User → React → API Gateway (Python/FastAPI) → AI Service (multi-provider LLM) → Cube semantic layer → MySQL/Postgres/MongoDB/DuckDB.

The backend is fully decoupled from the frontend — third-party UIs can connect to the API gateway directly.

For the long version see IMPLEMENTATION_PLAN.md §3.

License

Proprietary.