Skip to content

Post-Mortem: Resolved 5xx outage on sandbox service #34

Description

@yannvr

Incident Summary

A monitor alert reported 5xx responses from the sandbox service. Live probes confirmed that the service returned HTTP 500.

Actions Taken

Ran the heal_service runbook for the sandbox service, which successfully healed the service.

Verification Evidence

A public probe of sandbox via get_app_detail returned HTTP 200 with the body snippet "sandbox: ok":

{
  "app": "sandbox",
  "about": "drill target — a real service the red button really breaks",
  "http": 200,
  "bytes": 11,
  "server": "Google Frontend",
  "cache": "",
  "final_url": "https://helm-294160018950.europe-west1.run.app/sandbox",
  "body_snippet": "sandbox: ok"
}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions