Skip to content

A worker that opens a stream and never speaks is invisible to both detectors #1275

Description

@gHashTag

Specimen, not a hypothesis. Task 3165EF5A on #1273 sat in .running with streamOutcome == .open, no lastStreamByteAt and no completedTurns for 24 minutes.

Neither detector could ever have taken it:

  • wasNeverStarted requires streamOutcome == nil. This one is .open.
  • hasGoneSilent opened with guard !isStreamOpen(task) else { return false }, so an open stream was never silent — not at 24 minutes, not at 24 hours.

The same shield stood in three places independently: the policy, reapStalledWorkers, and the test helper that mirrors it. A mute worker had to survive three coincidences and survived all three.

The doc comment above hasGoneSilent described a conjunction — "no byte for threshold, and the stream is not open" — while the code short-circuited before ever consulting the bytes. The byte test was dead on the open path.

Fix

An open stream that has spoken keeps its full hour; that rule is untouched, and the scenario defending it stays green. An open stream that has never spoken is judged against a first-byte deadline instead. Waiting for a first token and pausing between tokens are different quantities with different scales.

firstByteDeadline = 600s, derived from the transport rather than chosen: SSETransport gives up on a fully mute connection after 3 x 120s request-inactivity plus 1s and 2s backoff = 363s. The reaper must not cut in ahead of the layer that actually knows (the Falcon argument already cited on WorkerStreamOutcome). 600s also coincides with timeoutIntervalForResource.

New field DelegatedTask.streamOpenedAt, because time-to-first-byte was unmeasurable without it — both quantities were being read off the same updatedAt.

Likely proximate cause, not yet proven

Both ~/.trios-dev/config.json and ~/.trios/config.json hold TRIOS_ZAI_API_KEY as an empty string. If the runner's failure path does not record .cut, a credential failure leaves the stream fact at .open forever. That is a separate defect and is not what this issue fixes — the deadline is the end-to-end backstop that holds whether or not every error path remembers to report.

Proof

Reverting the predicate turns exactly two new assertions red and leaves the 90-byte "thinking" scenario green.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions