Skip to content

Confirm the enclosure permit before serving requests after boot - #642

Merged
xmap merged 1 commit into
mainfrom
fix-enclosure-monitor-startup-race
Aug 10, 2026
Merged

Confirm the enclosure permit before serving requests after boot#642
xmap merged 1 commit into
mainfrom
fix-enclosure-monitor-startup-race

Conversation

@xmap

@xmap xmap commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • Boot used to start the enclosure permit monitor task and immediately serve requests, so a start_run/start_procedure preflight landing in the first few seconds after a restart could pass on a stale permit_status left over from before the outage.
  • enclosure_permit_monitor_lifespan now waits, bounded (default 8.0s, matching EpicsCaControlPort's own connect timeout plus headroom), for every configured enclosure's first settled observation before yielding, then drains the enclosure projection once, since the wait alone only proves an EnclosurePermitObserved event was appended and permit_status is a denormalized column the ProjectionWorker catches up to on its own poll cadence.
  • Both steps degrade gracefully: a PV or a slow drain that never settles logs a warning and lets boot proceed rather than blocking or crashing it. The existing (possibly stale) reading stands, gated the same as any other stale reading.
  • Item 1 (coverage-window commissioning ladder, item 3 originally scoped in [enclosure coverage-window design]) split out as its own small slice ahead of the larger coverage-trail design lock, since it's a live correctness bug rather than a record-honesty feature.

No migration, no aggregate/decider/event change. New Settings.enclosure_permit_monitor_startup_timeout_seconds field (default 8.0), all new function parameters keyword-only and optional so no existing caller signature broke.

Test plan

  • uv run pyright clean on all changed files
  • uv run ruff check / ruff format --check clean
  • Full architecture fitness suite: 30400 passed, 621 skipped
  • Full unit tier: 12477 passed, 3 skipped
  • 14 tests in tests/integration/test_enclosure_permit_monitor.py (5 new unit tests plus 2 added during review: a cancellation-during-startup-wait regression test verified against a standalone repro of the pre-fix structure, and a drain-timeout-degrades-gracefully test)
  • Reviewed across 3 adversarial passes (task-leak on cancellation, read-model-lag not covered by the wait alone, timeout racing the adapter's own connect timeout, an unguarded drain exception that would have crashed boot, and a dead-constant bump that left the production default unfixed — all found and fixed)
  • test_lifespan_drains_projection_before_yielding (integration, needs Postgres) not run locally — no Docker daemon in the dev sandbox; will run in CI

🤖 Generated with Claude Code

Boot used to start the permit monitor task and immediately serve
requests, so a start_run/start_procedure preflight landing in the
first few seconds after a restart could pass on a stale permit_status
left over from before the outage. enclosure_permit_monitor_lifespan
now waits, bounded, for every configured enclosure's first settled
observation, then drains the enclosure projection once, since the wait
alone only proves an event was appended and permit_status is a
denormalized column the ProjectionWorker catches up to separately. A
PV or a slow drain that never settles does not block or fail boot past
its timeout; the existing reading stands, gated the same as any other
stale reading. Item 1 of the 2-BM coverage-window commissioning ladder.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/enclosure
  _monitor.py
  apps/api/src/cora/infrastructure
  config.py
Project Total  

This report was generated by python-coverage-comment-action

@xmap
xmap merged commit 28c2146 into main Aug 10, 2026
19 checks passed
@xmap
xmap deleted the fix-enclosure-monitor-startup-race branch August 10, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant