Retire Jenkins for vmt: build & deploy both mt-test tiers from GitHub Actions

[htres736]

Summary

Replaces both vmt Jenkins pipelines with GitHub Actions workflows. The diff is CI only — no application code changes.

Both deploy to the same mt-test host and the same mt-test stack, and both share the vmt-test-deploy concurrency group so they can never touch the stack at once.

What changed versus the Jenkins pipelines

Deploys one service instead of the whole stack. Jenkins ran docker service rm followed by a full docker stack deploy. That reconciles every service in the stack file, and with the default --resolve-image=always it re-resolved the moving *-staging-latest tags, so a development push restarted the staging tier. Now it is a targeted docker service update --image, with a stack deploy fallback only when the service does not exist.

Deploys by digest, not by moving tag. The image is dev-latest@ / staging-latest@, the digest coming from the build step in the same run. The digest makes it a genuine spec change, so swarm does a rolling restart without the service rm hack, and the live service spec still matches the tag the stack file declares — a manual docker stack deploy on mt-test resolves to the same digest and leaves vmt alone. The *-build-<run#> tags are still pushed as named rollback targets.

Waits for real convergence instead of sleep 10. The loop reads the desired replica count from the service spec, counts running tasks, prints a timestamped heartbeat every 10s, and dumps docker service ps --no-trunc before failing at 300s.

Survives a slow deploy. ssh keepalives, BatchMode, ConnectTimeout, a runner-side timeout 900, and --detach=true on the monitoring_prom force-update, which otherwise blocks on a service in a different stack.

Other differences from Jenkins: buildx with GHA layer caching; Slack via an Incoming Webhook rather than the Jenkins Slack plugin (notify steps are continue-on-error and skip themselves when the webhook secret is unset); no workspace cleanup step, which GHA does not need.

Deliberate scope change

The Jenkins jobs checked out the devops repo and scp'd the stack file, traefik.yml and the .env-* files to /mtdocker on every deploy. These workflows do not — /mtdocker is assumed already provisioned, which keeps every secret out of GitHub. The tradeoff is that changes to docker-compose-mt-apps-test.yml and the env files need their own delivery path once Jenkins stops running. Tracked separately; it does not block this merge.

Testing

The development workflow has been running against the real development branch and deploying mt-test_vmt successfully, including with the digest pin.

The staging workflow is unrun — it cannot fire until it is on main, which this merge does.

Note on merging

Merging this will immediately trigger the staging workflow, because the push to main that lands the file is itself a push: main event. That build deploys mt-test_vmt-staging. Worth confirming beforehand that mt-test_vmt-staging exists on the host — the old Jenkinsfile-staging ran docker service rm with no || true, so if a past run died mid-deploy the service may be missing, in which case the first run takes the stack deploy bootstrap path and reconciles the whole stack file once.