Skip to content

ops: SLOs, CloudWatch dashboard, alert routing #38

Description

@jonmatum

Priority: P0 — launch blocker
Parent: #34

Context

Nine CloudWatch alarms exist today, all pointing to the ssb-{env}-daily-digest SNS topic. There is no dashboard, no published SLO, and no severity-based routing. A Bedrock throttle at 03:00 and a stale-node surfacing event both go to the same place.

Requirements

SLOs (publish in docs/ops/slo.md)

  • API Gateway availability (e.g. 99.5 % monthly)
  • POST /capture p95 latency (e.g. ≤ 1.5 s)
  • Enrich success rate (e.g. ≥ 99 % over 24h)
  • Error budget + burn-rate alarms

Dashboard

Alert routing

  • Two SNS topics: ssb-{env}-alerts-critical and ssb-{env}-alerts-info
  • Critical topic: API 5xx burst, DDB throttle, any DLQ depth > 0, Bedrock cost alarm, WAF block spike
  • Info topic: daily digest, surfacing results
  • Decide integration target (email vs PagerDuty vs Slack webhook) and document in docs/ops/oncall.md

Acceptance criteria

  • SLO doc published
  • Dashboard reachable and linked from docs
  • Two alert topics exist with correct subscriptions; manual alarm test triggers the critical path

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions