Skip to content

fix: converge asynchronous supervisor transitions - #75

Merged
fuller-stack-dev merged 1 commit into
openclaw:mainfrom
fuller-stack-dev:fix/async-supervisor-convergence
Aug 13, 2026
Merged

fix: converge asynchronous supervisor transitions#75
fuller-stack-dev merged 1 commit into
openclaw:mainfrom
fuller-stack-dev:fix/async-supervisor-convergence

Conversation

@fuller-stack-dev

Copy link
Copy Markdown
Member

Summary

  • treat desired service policy and observed child state as separate inputs at the snapshot safety boundary
  • wait for a requested start/restart to converge on an observed child and HTTP health before deep gateway verification
  • preserve existing bounded stop/start timeouts, restore prior policy on snapshot failure, and fail closed when convergence or gateway readiness does not arrive

Root cause

Supervisor actions publish desired policy before the child transition is necessarily observable. Snapshot creation treated a still-running advisory stop result as final before its existing quiescence wait. Upgrade did the inverse: it skipped health convergence when start/restart returned desiredRunning=true with running=false, then probed the gateway early and rolled back on ECONNREFUSED.

The owner-level invariant is now: policy decides what state OCM must converge toward; observed child and gateway state decide when maintenance is safe or successful.

Compatibility and failure boundary

  • synchronous supervisors retain their existing fast path
  • already-stopped enabled services and disabled services retain their requested policy
  • a stale observed child under desired-running=false is quiesced without re-enabling the service
  • snapshot/runtime mutation remains blocked until the supervisor reports no planned, runtime-child, or runtime-service state
  • upgrade success still requires observed child state, HTTP health, and the existing deep gateway RPC
  • no retry was added; the existing 3s advisory stop wait, 5s snapshot quiescence wait, and 90s gateway convergence deadline are unchanged
  • rollback restoration remains unchanged apart from using the requested policy to decide whether convergence is required

This PR intentionally excludes the separate cold-finalization/long-outage transaction-ordering work in #60.

Proof

Canonical base: f6dea9f

On that exact unmodified base, the delayed-stop fixture aborted before bounded quiescence with managed service remained running after the snapshot safety stop; the delayed-start fixture rolled back after an early ECONNREFUSED probe. Canonical main remains that same SHA, and #60 current head retains both broken gates.

On this head:

  • exact delayed-stop and delayed-start regression commands passed
  • cargo test --test env_snapshot_tests -- --test-threads=1 - 35 passed
  • cargo test --test upgrade_command_tests -- --test-threads=1 - 46 passed
  • cargo check --workspace --all-targets --locked
  • cargo fmt --all -- --check
  • git diff --check
  • source-blind CLI validation: delayed stop converged in 3.52s with running/stopped/disabled policies preserved; delayed start converged on new, made a real health request, and did not roll back
  • structured autoreview: clean, no accepted/actionable P0 findings

No Crabbox was used; the local fake launchd/supervisor contract exercises the same JSON policy/runtime boundary without platform-specific behavior.

@fuller-stack-dev
fuller-stack-dev merged commit 5d8ef11 into openclaw:main Aug 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant