One-command Kubernetes + AWS deployment, multi-arch images

.github/workflows/docker.yml

@@ -0,0 +1,56 @@
+name: Docker image
+
+# Publishes ghcr.io/runcaptain/compass on every release tag — the image the
+# deploy/kubernetes overlays reference. Multi-arch (amd64 + arm64; the
+# builder base is trixie specifically so arm64 compiles). Also runnable
+# manually to publish an ad-hoc tag.
+
+on:
+  push:
+    tags:
+      - "v*"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  packages: write
+
+jobs:
+  publish:
+    runs-on: ubuntu-24.04
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: docker/setup-qemu-action@v3
+      - uses: docker/setup-buildx-action@v3
+
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ghcr.io/${{ github.repository }}
+          tags: |
+            type=semver,pattern=v{{version}}
+            type=semver,pattern=v{{major}}.{{minor}}
+            type=raw,value=latest,enable={{is_default_branch}}
+            type=raw,value=manual-${{ github.run_number }},enable=${{ github.event_name == 'workflow_dispatch' }}
+
+      # The overlays assume the serverless topology, so the published image
+      # carries the object-storage backend. Local-first users typically build
+      # from source anyway (README quickstart).
+      - uses: docker/build-push-action@v6
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          build-args: |
+            CARGO_FEATURES=object-storage
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max

.gitignore

@@ -24,3 +24,10 @@ beir_data/
 
 # Committed template — .env.* above must not swallow it
 !.env.example
+
+# Terraform (deploy/terraform)
+.terraform/
+*.tfstate
+*.tfstate.*
+*.tfvars
+.terraform.lock.hcl

CHANGELOG.md

@@ -4,6 +4,13 @@ All notable changes to this project are documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- **One-command deployment** (`deploy/`): a kustomize tree for any Kubernetes cluster — serving StatefulSet with a PVC per replica (warm restarts), stateless writer Deployment, optional cold-serving tier, hardened pod security (non-root, seccomp, no capabilities), startup probes sized for cold rebuilds — plus Terraform for the AWS storage half (private KMS-encrypted S3 bucket with a TLS-only policy, least-privilege IAM, IRSA role or access key). A self-contained `minio-dev` overlay brings the whole serverless topology up on kind/k3s/minikube in one `kubectl apply -k`. Verified end-to-end on a live k3s cluster (all tiers serving, cross-tier read-your-writes, PVC persistence across pod replacement) and `terraform apply` verified against an AWS API emulator (idempotent re-apply; destroy refuses while data exists).
+- **Published container images**: `ghcr.io/runcaptain/compass` (multi-arch amd64+arm64) on every release tag via `.github/workflows/docker.yml`. The Docker builder base moved to `rust:1.88-trixie` — bookworm's gcc-12 could not compile a dependency's ARM feature probes, which had made arm64 images impossible.
+
 ## [0.4.0] - 2026-07-04
 
 ### Added — serve-from-storage ("true serverless")

Dockerfile

@@ -6,7 +6,9 @@
 # ── Stage 1: Build ────────────────────────────────────────────────────────────
 # Pin builder toolchain so deploys are reproducible and a compromised
 # rust:latest tag can't silently land in our image.
-FROM rust:1.88-bookworm AS builder
+# trixie (gcc-14) rather than bookworm: bookworm's gcc-12 fails to build
+# the numkong dependency's ARM feature probes, which blocked arm64 images.
+FROM rust:1.88-trixie AS builder
 
 WORKDIR /app
 

README.md

@@ -419,6 +419,22 @@ docker build -t compass .
 docker run -p 4001:4001 -v ./data:/app/data compass
 ```
 
+## Deploy to Kubernetes / AWS (one command)
+
+[`deploy/`](deploy/README.md) ships the full serverless topology as code: a
+kustomize tree (serving StatefulSet with a PVC per replica, stateless writer
+Deployment, optional cold tier) plus Terraform for the AWS storage half
+(private encrypted S3 bucket + least-privilege IAM, IRSA-ready). Try the whole
+thing on any local cluster:
+
+```bash
+kubectl apply -k deploy/kubernetes/overlays/minio-dev
+```
+
+Production: `terraform apply` in [`deploy/terraform/aws`](deploy/terraform/aws),
+point [`deploy/kubernetes/overlays/aws`](deploy/kubernetes/overlays/aws) at the
+bucket, `kubectl apply -k`. Details and operational notes: [deploy/README.md](deploy/README.md).
+
 ## Object storage (S3 / GCS / Azure)
 
 By default Compass persists to local disk — zero config, no credentials. Optionally, it can persist to your own cloud object storage instead (a hard either/or, chosen at startup):

deploy/README.md

@@ -0,0 +1,93 @@
+# Deploying Compass
+
+Three paths, in increasing order of ceremony. All of them are the same
+binary; topology background lives in [docs/deployment.md](../docs/deployment.md).
+
+## 1. One machine (docker compose)
+
+```bash
+docker compose up          # local-first: data on the local volume, zero config
+```
+
+## 2. Kubernetes (any cluster) — the serverless topology
+
+`deploy/kubernetes` is a kustomize tree:
+
+```
+base/                serving StatefulSet (PVC per replica) + writer Deployment
+                     + optional cold Deployment + services + config
+overlays/minio-dev   self-contained dev stack (in-cluster MinIO) — try it on
+                     kind/k3s/minikube in one command
+overlays/aws         production: real S3, IRSA or access-key auth
+```
+
+Try the whole topology on a local cluster:
+
+```bash
+kubectl apply -k deploy/kubernetes/overlays/minio-dev
+kubectl -n compass-dev get pods        # serving-0, writer, cold, minio
+kubectl -n compass-dev port-forward svc/compass-read 4001:4001
+curl localhost:4001/health
+```
+
+Production on AWS:
+
+```bash
+# 1. Storage half: bucket + least-privilege IAM
+cd deploy/terraform/aws
+terraform init && terraform apply -var bucket_name=my-compass-data
+#    EKS?    add -var eks_oidc_provider_arn=… -var eks_oidc_provider_url=…
+#    no EKS? add -var create_access_key=true and create the Secret it hints at
+
+# 2. Compute half: point the overlay at your bucket + image, then
+vi deploy/kubernetes/overlays/aws/kustomization.yaml   # bucket, region, image, IRSA arn
+kubectl apply -k deploy/kubernetes/overlays/aws
+```
+
+What you get:
+
+| workload | kind | storage | scale advice |
+|---|---|---|---|
+| `compass-serving` | StatefulSet | PVC per replica (warm restarts = seconds) | scale for read QPS; each replica converges independently |
+| `compass-writer` | Deployment | none (stateless) | scale for ingest; safe to kill any time |
+| `compass-cold` | Deployment (optional) | none | scale-to-many for bursty semantic reads on rarely-touched collections |
+
+Traffic contract: **reads → `compass-read`, writes → `compass-write`**,
+optional cold reads → `compass-cold`. Writers refuse reads by design;
+serving nodes accept both but you keep clean scaling curves by splitting.
+
+## 3. Terraform (AWS storage half)
+
+`deploy/terraform/aws` provisions the bucket (private, encrypted,
+lifecycle-managed) and least-privilege IAM — IRSA role for EKS, or an IAM
+user + access key for anything else. It deliberately does NOT create a
+cluster; bring any Kubernetes (or run the containers on VMs).
+
+## Operational notes (read before production)
+
+- **The bucket is the database.** PVCs are a warm cache — losing one costs a
+  rebuild, never data. Bucket deletion is data loss; `force_destroy` stays
+  false for a reason.
+- **Consistency**: writes through writers are durable immediately and
+  visible on serving nodes within `COMPASS_REFRESH_INTERVAL` (default 5s).
+  Pass a write's `seq` as `min_seq` on search for read-your-writes; cold
+  pods have read-your-writes by construction.
+- **Version skew**: never run pre-v0.4 and v0.4+ writers against one bucket.
+  Roll writers first, then serving nodes.
+- **Auth**: the API is unauthenticated until you set `COMPASS_API_KEY`. The
+  manifests already envFrom the `compass-aws` secret, so add the key there:
+  `kubectl -n compass create secret generic compass-aws --from-literal=COMPASS_API_KEY=… [--from-literal=AWS_…]`.
+  `/health` and `/metrics` stay unauthenticated by design; keep them
+  cluster-internal (no NetworkPolicy ships here — add one if your cluster
+  doesn't default-deny).
+- **PVC lifecycle**: `kubectl delete -k …` removes the pods but RETAINS the
+  StatefulSet PVCs (Kubernetes default) — re-applying reuses the warm cache.
+  Delete PVCs explicitly to reclaim disk; that costs a rebuild, never data.
+- **Single-replica updates**: with `replicas: 1`, a rolling update has a
+  brief read-downtime window while the pod restarts (writes keep flowing via
+  writers). Run ≥2 serving replicas if reads must never blip.
+- **Sizing**: serving-node RAM tracks attached collections (bound it with
+  `COMPASS_LAZY_ATTACH` + `COMPASS_MAX_ATTACHED`); PVC size tracks the same
+  data as the bucket per attached collection. Cold pods run in ~tens of MiB.
+- **Upgrades**: StatefulSet updates roll one pod at a time; readiness gating
+  keeps traffic off a pod until its indexes serve. Writers roll freely.

deploy/kubernetes/base/common.yaml

@@ -0,0 +1,24 @@
+# Shared plumbing: namespace-agnostic service account + engine config.
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: compass
+  # On EKS with IRSA, the aws overlay annotates this with the role ARN
+  # from the Terraform output — no long-lived keys in the cluster.
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: compass-config
+data:
+  PORT: "4001"
+  RUST_LOG: "compass=info"
+  # The bucket. Overlays set this (s3://…, gs://…, az://…). Empty = the
+  # engine falls back to LOCAL disk mode — a bare `apply -k base` still
+  # boots, it just isn't the serverless topology.
+  COMPASS_STORAGE: ""
+  # Serving nodes converge on other nodes' writes every N seconds.
+  COMPASS_REFRESH_INTERVAL: "5"
+  # Uncomment to bound serving RAM to the hot collection set:
+  # COMPASS_LAZY_ATTACH: "true"
+  # COMPASS_MAX_ATTACHED: "64"

deploy/kubernetes/base/kustomization.yaml

@@ -0,0 +1,10 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - common.yaml
+  - serving.yaml
+  - writer.yaml
+labels:
+  - pairs:
+      app.kubernetes.io/part-of: compass
+    includeSelectors: false