Skip to content

fix(reactor-cli): keep serving the other reactors when one reactor's poll rejects - #5

Open
samuelmbabhazi wants to merge 1 commit into
openprose:mainfrom
samuelmbabhazi:fix/host-reactor-isolation
Open

fix(reactor-cli): keep serving the other reactors when one reactor's poll rejects#5
samuelmbabhazi wants to merge 1 commit into
openprose:mainfrom
samuelmbabhazi:fix/host-reactor-isolation

Conversation

@samuelmbabhazi

@samuelmbabhazi samuelmbabhazi commented Aug 28, 2026

Copy link
Copy Markdown

Summary

Fixes #4.

The host awaited every reactor's continuity poll, gateway poll and shutdown with Promise.all. One reactor's rejection made the combined promise reject at once: the other reactors' tasks kept running in the pool, but their results were discarded, a second failure in the same cycle was never surfaced, and the serve loop, which has no handler around those awaits, exited with code 1 without draining anyone. Every per-reactor task now settles before failures are handled:

  • bootHost gains an optional onReactorError handler receiving { name, phase, error }. With it the host keeps serving and reports the failing reactor; without it the first failure is still rethrown, but only once every reactor has settled (existing callers keep their contract).
  • pollGatewaysAll returns the outcomes of the reactors that succeeded. The gateway phase shares the same settling helper: a connector fetch throwing at poll time (the http connector throws on a non 2xx response, and the poller has no handling around it) is reported like a continuity poll failure. Render failures were never the problem, they already commit as failed receipts.
  • shutdown drains every reactor, then the pool, before surfacing a failed shutdown.
  • reactor serve wires the handler: an operator visible line per failed cycle (reactor <name>: poll failed: <message>, or a new reactor-error record under --json) and the categorized error event sampled once per reactor, so a persistent fault on a 60s cadence cannot flood the backend.

This changes host behavior on failure, including for a single reactor host, which now keeps polling and reporting instead of exiting on a failed cycle (Reactor is alpha, so per CONTRIBUTING it is called out here and recorded under [Unreleased] in CHANGELOG.md). The API is additive.

Tests

Two cases join the multi-reactor isolation block of serve-host.test.ts, injecting the failure through the simplest seam, a freshness reader that arms at boot and throws on the poll's read: with a handler, alpha's rejected poll is reported while beta keeps polling and triggering across cycles; without a handler, the rejection still reaches the caller, and beta's poll is observed to have run before the rejection is surfaced (serial pool, alpha first). Reverting pollAll to Promise.all makes both cases fail.

Gates run: pnpm test:reactor:offline (reactor 496 passed, reactor-cli 212 passed, reactor-devtools 96 passed, 0 failed) and pnpm test:examples (16 files, 202 tests passed). No LIVE test was run.

…poll rejects

The host awaited every reactor's poll, gateway poll and shutdown with
Promise.all, so a single reactor's rejection aborted the whole cycle and
the serve loop exited, taking the healthy reactors down with it. Every
per-reactor task now settles before failures are handled: with the new
onReactorError handler the host keeps serving and reports the failing
reactor, and without one the first failure is rethrown only once every
reactor has settled. The serve command logs each failed cycle and emits
the categorized error event once per reactor.
@samuelmbabhazi
samuelmbabhazi force-pushed the fix/host-reactor-isolation branch from 9d2c075 to 333b97e Compare August 28, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reactor serve: one reactor's rejected poll takes every hosted reactor down

1 participant