test: load + chaos drill before launch

[jonmatum]

Priority: P2
Parent: #34

Context

All performance numbers today come from benchmarks at 178 nodes and nominal traffic. We have no evidence of behavior under load or failure.

Requirements

Load drill

• Script scripts/loadtest.sh using k6 or artillery
• Scenarios:
  • Ramp to 20 req/s on POST /capture for 10 min
  • Ramp to 100 req/s on GET /search for 10 min
  • Sustained 5 req/s on POST /capture with 10 % node-chat for 1 h
• Run against dev (or a throwaway env)
• Record: p50/p95/p99 latency, error rate, Bedrock token burn, DDB throttles
• Verify p95 meets SLO from ops: SLOs, CloudWatch dashboard, alert routing #38

Chaos drill

• Manual drill with a run-log:
  • Inject a 100 % failure on enrich — verify DLQ captures, alarm fires, runbook works
  • Throttle DDB (temporarily cap WCU/RCU or use fault injection) — verify backoff and user-visible error
  • Trigger a WAF rule — verify 403 path and runbook
  • Kill an in-flight capture (simulate timeout) — verify async enrich is not blocked

Acceptance criteria

• Load-test report committed to docs/benchmarks/load-YYYY-MM-DD.md
• Chaos-drill log committed to docs/ops/chaos-YYYY-MM-DD.md
• Any gap surfaced gets a follow-up issue before launch

References

• Production readiness — consolidated plan (tracking) #34 (parent)