Skip to content

PTY_LIST_TIMEOUT spans getting the child running, not just the pty work #360

Description

@schickling-assistant

Problem

Four run tests fail during the full Nix package suite on one Apple Silicon workstation,
all on the same production constant — PTY_LIST_TIMEOUT (2s, src/run.rs:46):

metadata_patch_uses_exact_id_and_one_json_stdin_payload
  → `pty metadata patch --id` failed: timed out after 2.0s
removed_and_recreated_pty_root_is_indeterminate_not_absent
  → `pty list --json` failed: timed out after 2.0s
pty_task_observation_preserves_exact_generation_and_closed_states
  → `pty list --json` failed: timed out after 2.0s
running_pty_without_complete_generation_is_indeterminate
  → index out of bounds: the len is 0 but the index is 0   (downstream of the same empty result)

Every one of these drives a stub #!/bin/sh script, not the real pty binary. So the 2s
budget expired while a shell ran three trivial builtins — which means the budget is being
spent on getting the child running, not on the work the child does.

The same constant governs production: patch_presentation and list_entries both pass
PTY_LIST_TIMEOUT to a child process. If 2s can expire around a stub shell, it can expire
around a real pty list --json on a reconcile pass, which surfaces as a spurious list
failure and a skipped pass.

This is the same defect class as #356 and #255 — a budget that has to cover getting the
child running, not just the behaviour being measured — but landing on a production constant
rather than a test-local deadline.

What is NOT the cause

Reported alongside the failure because the obvious explanations were tested and do not hold.
Whoever picks this up should not re-run these.

Not #358 / not a recent regression. The change that surfaced it is test-only — its entire
diff to src/run.rs is inside mod tests, and it touches neither PTY_LIST_TIMEOUT nor
run_captured's logic.

Not CPU load. Standalone fork+exec latency stays far below 2s even under deliberate
oversubscription:

Host Load trivial /bin/sh pty list --json (empty root)
16-core workstation 10 p90 0.011s, max 0.369s p90 0.187s, max 0.231s
16-core workstation 68 p90 0.004s, max 0.261s p90 0.063s, max 0.101s
10-core builder 10 p90 0.027s, max 0.176s p90 0.230s, max 0.295s
10-core builder 50–58 p90 0.039s, max 0.223s p90 0.569s, max 0.710s

Not memory pressure. 93% free at the time of the failing run.

Not simply in-suite spawn concurrency. run_captured installs a pre_exec hook
(setsid), which forces real fork() instead of posix_spawn(), and cargo test runs the
suite as ~ncpu threads of one large binary all spawning children. fork() cost does scale
with parent heap on Darwin — 16 threads at load 56 measured p90 0.121s / max 0.537s with a
2 GB heap, versus p90 0.041s / max 0.131s at 256 MB — but that is an order of magnitude short
of 2s. More decisively, the hosted 3-core macOS CI runner runs the same suite and passes;
if concurrent spawning were the mechanism, fewer cores should be worse, not better.

Where it has and has not been seen

Host Cores Condition Result
Hosted macos-15 CI runner 3 CI pass ×2
10-core builder 10 load 18–56 pass ×4
16-core workstation 16 load ~99, a file-provider daemon pegged at 100% these 4 fail
16-core workstation 16 load ~75, same host ~30 min later these 4 pass

Intermittent, not deterministic. A later full-suite run on the same host did not
reproduce them, so this is a tail event tied to transient host state rather than a property
of the host. Load average was ~99 while CPU demand was low and memory was 93% free, which is
consistent with processes blocked in the filesystem layer rather than starved of CPU — but a
standalone probe on that same host during that same state still could not produce a slow
child, so this is a lead, not a conclusion.

Why this is worth fixing anyway

The mechanism is unestablished, but the shape of the budget is wrong regardless: 2s is
charged for the child's scheduling, execve, and interpreter startup as well as the actual
pty work, so the margin available to the operation depends on host state the caller cannot
see. That is the same design error #356 fixed in the tests.

Deliberately not proposing a new number here. Raising the constant buys margin
proportional to nothing — the in-tree comments already call that the wrong answer — and the
measurements above are not a basis for choosing one.

Suggested next step

Reproduce inside the suite context rather than from a standalone probe: instrument
run_captured to record spawn→exit latency per child and run the full suite on the affected
host, so the distribution is captured at the moment it goes bad. That distinguishes "the
child was slow to start" from "the child was slow to run" — which decides whether the fix is
a separate startup allowance or something else entirely.

References

Posted on behalf of @schickling
field value
agent_identity unknown
agent_persona generalist
agent_supervisor unavailable
agent_tool Claude Code
agent_tool_version 2.1.237
agent_runtime Claude Code 2.1.237
tooling_profile dotfiles@11eaf2d-dirty

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions