Add default readiness/liveness probes to the RabbitMQ StatefulSet

[henzigo]

Refs #6

What

Gives the RabbitMQ StatefulSet default readiness/liveness probes (the probes bullet of #6).
The other tasks of #6 (port names/5672 declaration, startupProbe, preStop
rabbitmqctl stop_app + 90 s grace, memory request/limit defaults, explicit
updateStrategy) are not in this PR — hence Refs, not Closes.

Why

statefulset-rabbitmq.yaml rendered probes only from values via shopsys.containerSettings,
and charts/shopsys-infra/values.yaml defaults them to livenessProbe: {} /
readinessProbe: {} — so no probes reached the cluster. The broker pod counted as Ready
the moment the container process started, not when it accepted AMQP connections on 5672.

The hook ordering rests on the premise that the infra release (installed with wait: true)
is really ready before the app release's migration hook runs — see the comment in
helmfile.yaml.gotmpl and the #44 discussion referenced from #6. For RabbitMQ that premise
was hollow; with a readinessProbe it holds. Redis in the same chart already had template-side
default probes.

Chosen defaults and sources

readinessProbe:
  tcpSocket: { port: 5672 }
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 3
livenessProbe:
  exec: { command: [rabbitmq-diagnostics, -q, ping] }
  initialDelaySeconds: 60
  periodSeconds: 30
  timeoutSeconds: 15
  failureThreshold: 3

• Readiness = TCP on the AMQP port. RabbitMQ monitoring / health
  checks states verbatim: "RabbitMQ
  Kubernetes Operator configures a TCP port check on the AMQP port as the readinessProbe and
  defines no livenessProbe at all. This should be considered the best practice." The same
  page explains CLI-based checks are unsuitable as readiness probes (performance overhead,
  they assume a fully booted node). The AMQP listener opens as one of the last boot steps, so
  the TCP check is precisely the signal we need — and it is cheap. The timings match the
  Cluster Operator's defaults (10/10/5, failureThreshold 3),
  cf. cluster-operator#413.
• Liveness = stage-1 rabbitmq-diagnostics -q ping. Stage 1 in the docs' health-check
  ladder: validates the runtime is operating and CLI authentication works, with "nearly zero
  false positives outside maintenance windows". It spawns an Erlang VM, so it is not cheap and
  the budget is deliberately generous (60 s initial delay + 3 × 30 s ⇒ ~150 s before a restart,
  15 s timeout because a loaded node needs it). Heavier stage-3+ checks (check_running,
  check_local_alarms) are intentionally not used for liveness — an alarmed-but-alive
  broker must not be killed. Upstream even recommends no liveness probe at all; a stage-1
  ping is the conservative middle ground and matches what
  shopsys/deployment#75 proposed.

Divergence from #6 (please review)

#6 (and deployment#75) specifies readiness as rabbitmq-diagnostics -q check_running. I went
with the TCP check on 5672 instead, because the official docs explicitly discourage CLI checks
as readiness probes and name the TCP-on-AMQP check the best practice (quote above). It is also
strictly cheaper and a later boot signal than check_running. Liveness follows #6/#75 with
the softened budget from the issue comment (failureThreshold: 3 instead of 5), keeping
timeoutSeconds: 15 — a too-short timeout is itself a false-positive source for a check that
boots an Erlang VM.

No startupProbe here: it is a separate #6 task item, and without it the liveness probe's
60 s initial delay + 90 s failure budget already covers a slow boot. Adding a startupProbe
key would also introduce a rabbitmq-only value key, inconsistent with the standard component
keys — better done together with the rest of #6.

Changes

• charts/shopsys-infra/templates/statefulset-rabbitmq.yaml: probes rendered with the
  existing redis if/else pattern (deployment-redis.yaml) rather than
  shopsys.containerSettings, so a probe set in values replaces the default wholesale —
  map deep-merge would otherwise make the defaults impossible to remove. The container
  securityContext (the helper's only other output here) is rendered inline; verified there
  is exactly one livenessProbe/readinessProbe block in the rendered output, no duplicate
  via the helper. Rationale lives in Go-template comments, so nothing leaks into the manifest.
• charts/shopsys-infra/values.yaml: rabbitmq.livenessProbe/readinessProbe stay {} and
  are now documented (empty = chart defaults, set = wholesale replace). No new keys ⇒ no
  values.schema.json change.
• Tests: 3 new helm-unittest cases (defaults render exactly; explicit values replace both
  probes fully; securityContext still renders next to the probes) — 62 tests pass.
• Docs: new deviation 21 in docs/migrating-from-shopsys-deployment.md; the standard
  component keys table in docs/values.md now says probes replace wholesale.
• Golden snapshots regenerated: all 15 snapshots, rabbitmq container only (+ the stray blank
  line the empty containerSettings include used to leave behind).

Notes

• helm unittest 62 passed · golden 15/15 · helm lint both charts clean · kubeconform
  531/531 valid (k8s 1.31.0) · shellcheck clean.
• Merge note: the parallel PR on drop-sleep-and-bound-hooks rewords deviation 9 with
  "RabbitMQ has no default readinessProbe, so the wait guarantees a Running broker container,
  not yet an accepting broker". Once both land, that clause is stale and should be dropped
  (the [infra] RabbitMQ hardening: ports, probes, graceful restart #6 task item about entry 9) — whichever merges second should fix it. Golden
  expected/*.yaml conflicts with that PR are expected and are pure snapshot regenerations.

🤖 Generated with Claude Code