From 71ca229d76c718b55c42552328bfecd3e520510e Mon Sep 17 00:00:00 2001 From: Dan Chen Date: Wed, 20 May 2026 19:38:50 -0400 Subject: [PATCH] refine README overview --- README.md | 41 +++--- .../images/primary-control-execution-flow.svg | 134 ++++++++++++++++++ 2 files changed, 153 insertions(+), 22 deletions(-) create mode 100644 docs/images/primary-control-execution-flow.svg diff --git a/README.md b/README.md index 525d6c0..dcc0492 100644 --- a/README.md +++ b/README.md @@ -29,36 +29,33 @@ This platform map shows the major control-plane, execution-plane, persistence, and observability domains. The public demo path is intentionally narrower than -the full platform surface. +the full platform surface, but the same boundaries carry through the production +shape of the system. ![MLIS platform map](docs/images/platform-map.svg) +The diagram below connects those domains along the primary request, scheduling, +execution, and persistence path. + +![Primary control and execution flow](docs/images/primary-control-execution-flow.svg) + `/internal/workers` and `/internal/assignments` are worker/data-plane protocol routes, not public user APIs. Users submit and inspect work through `/v1/jobs`, -the CLI, or the console; worker containers use `/internal/*` to register -capacity, poll assignments, and report execution state. See the +the CLI, or the console. Worker containers use `/internal/*` to register +capacity, claim assignments, heartbeat lease ownership, and report execution +state. See the [worker protocol runbook](docs/runbooks/worker-protocol.md). -```mermaid -stateDiagram-v2 - [*] --> PENDING: admission accepted - PENDING --> RUNNING: assignment claimed - RUNNING --> SUCCEEDED: worker finish - RUNNING --> FAILED: worker/report failure - RUNNING --> PENDING: lease expired / reclaim - PENDING --> CANCELLED: user cancel - RUNNING --> CANCELLED: user cancel - SUCCEEDED --> [*] - FAILED --> [*] - CANCELLED --> [*] -``` - -This diagram shows the public `job` lifecycle. Execution attempts are modeled -separately as `job_assignments`, which is what makes lease expiry and reclaim -possible without collapsing user intent and worker execution into one state -machine. +The public lifecycle is modeled at two levels: `job` captures durable user +intent, while `job_assignment` captures individual execution attempts. That +separation makes lease expiry, reclaim, and worker recovery possible without +collapsing user intent and worker execution into one state machine. See +[job lifecycle](docs/design/job-lifecycle.md) and +[lease-based recovery](docs/design/lease-recovery.md) for the full state model. -Authentication and tenant binding happen in middleware before route handlers. Tenant context comes from the verified token; database queries are filtered by `tenant_id` from `request.state.identity`, not from request bodies. +Authentication and tenant binding happen in middleware before route handlers. +Tenant context comes from the verified token; database queries are filtered by +`tenant_id` from `request.state.identity`, not from request bodies. diff --git a/docs/images/primary-control-execution-flow.svg b/docs/images/primary-control-execution-flow.svg new file mode 100644 index 0000000..c4c0052 --- /dev/null +++ b/docs/images/primary-control-execution-flow.svg @@ -0,0 +1,134 @@ + + Primary control and execution flow + A layered diagram showing how requests enter MLIS through user interfaces, flow into the FastAPI control plane, pass admission and tenant policy, reach the central scheduler, dispatch to CPU, GPU, and IO workers, and persist state, artifacts, and observability outputs. + + + + + + MLIS primary control and execution flow + + + + + + + + User Interfaces + + Web Console + + API / CLI + + + + + + Control Plane + + + FastAPI Control Plane + + + Admission & Validation + schema, auth, rate limits + + + Tenant Policy / Quota / Priority + tenant-aware policy gates + + + Central Scheduler + + + + + + Execution Plane + + + CPU Workers + general compute + + + GPU Workers + accelerated compute + + + IO Workers + data movement + + + + + + State and Outputs + + + Postgres Job State + jobs, assignments, workers + + + Artifact Store + inputs and outputs + + + Metrics & Logs + observability outputs + + + + + + + Observability + all tiers emit metrics asynchronously + + + + + + + primary request / state flow + + observability emitted (async) + +