From 8191ff841ce7b37a5e78b7278174ff6d23af5cae Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Sun, 23 Aug 2026 18:59:03 +0200 Subject: [PATCH 1/5] feat(agents): join observed harness state into the roster, doctor, and human output The chosen $E surface from #268: an appended observedState object on both roster JSON shapes (null = no record, distinct from derived unknown), a compact observed column in human output, and advisory-only doctor lines. The session-liveness cross-check demands positive evidence of death (ding::session_liveness); an unreadable registry downgrades nothing. Row 22's wording now names the third independent axis with its proof, and two stale stream proof references from the #300/#313 renames are corrected. Co-Authored-By: Claude Fable 5 --- INVARIANTS.md | 4 +- src/agents.rs | 115 +++++++++++++++++++++++++++++++++++++++-- src/ding/mod.rs | 28 ++++++++++ src/main.rs | 50 +++++++++++++++++- tests/status_agents.rs | 2 +- 5 files changed, 191 insertions(+), 8 deletions(-) diff --git a/INVARIANTS.md b/INVARIANTS.md index bd053af9..a12f3c32 100644 --- a/INVARIANTS.md +++ b/INVARIANTS.md @@ -11,7 +11,7 @@ materialization, messaging, DING, or presence must preserve them. | **Transport-decoupled lifecycle** | Each task is isolated from a supervisor/transport process-group or cgroup cascade. | `tests/transport_isolation.rs`; `tests/transport_isolation_macos.rs` | | **Clean exec teardown** | Killing an exec task reaps its whole process group. | `tests/exec_backend.rs::exec_kill_reaps_the_whole_process_group_not_just_the_leader` | | **Bounded restart diagnostics** | Relaunching an exec task preserves the just-finished log as one prior generation while bounding retained diagnostics to current plus prior. Final retirement removes the PID and both logs. | `tests/exec_backend.rs::exec_restart_reap_keeps_bounded_diagnostics_and_final_remove_cleans_them`; `tests/run.rs::up_once_finally_removes_dead_retired_tasks_without_restarting_them` | -| **Derived companion lifecycle** | A generated DING or launched stream starts only with an eligible canonical agent, is suppressed or stopped while that target is held, suspended, unavailable, retired, or terminally parked, and remains coupled without changing explicit sibling task behavior. A stream lowers its authored adapter launch directly, parks and surfaces independently, and neither makes an otherwise empty agent runnable nor claims a delivery transport. Stream authoring is serialized, authority-scoped, source-preserving, and fail-closed for Nix ownership and invalid declarations. Healthy compact startup launches companions in one pass, and targeted reconciliation holds a missing generated companion rather than broadening to its agent. | `crates/agent-spec/tests/discovery.rs::streams_are_typed_and_only_launched_streams_lower_to_derived_exec_tasks`; `crates/agent-spec/tests/discovery.rs::stream_names_launches_and_task_collisions_fail_closed`; `src/agent_author.rs::stream_add_supports_external_command_and_argv_and_remove_is_idempotent`; `src/agent_author.rs::stream_authoring_enforces_authority_nix_ownership_and_canonical_validation`; `tests/run.rs::fresh_compact_agent_launches_with_its_derived_ding`; `tests/run.rs::held_adopt_only_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::parked_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::retired_compact_agent_stops_agent_and_derived_ding`; `tests/run.rs::selected_missing_derived_ding_is_held_without_broadening_to_its_agent`; `tests/run.rs::fresh_compact_agent_launches_with_its_derived_stream`; `tests/run.rs::retired_compact_agent_stops_agent_and_derived_stream`; `tests/run.rs::suspended_compact_agent_stops_its_derived_stream_without_touching_a_sibling`; `tests/run.rs::held_adopt_only_compact_agent_stops_its_live_derived_stream`; `tests/run.rs::a_crash_looping_stream_parks_and_surfaces_without_disturbing_its_agent`; `tests/run.rs::parked_compact_agent_stops_its_live_derived_stream`; `tests/run.rs::selected_missing_derived_stream_is_held_without_broadening_to_its_agent`; `tests/run.rs::a_stream_alone_does_not_make_an_agent_runnable`; `tests/run.rs::a_stream_does_not_claim_a_delivery_transport` | +| **Derived companion lifecycle** | A generated DING or launched stream starts only with an eligible canonical agent, is suppressed or stopped while that target is held, suspended, unavailable, retired, or terminally parked, and remains coupled without changing explicit sibling task behavior. A stream lowers its authored adapter launch directly, parks and surfaces independently, and neither makes an otherwise empty agent runnable nor claims a delivery transport. Stream authoring is serialized, authority-scoped, source-preserving, and fail-closed for Nix ownership and invalid declarations. Healthy compact startup launches companions in one pass, and targeted reconciliation holds a missing generated companion rather than broadening to its agent. | `crates/agent-spec/tests/discovery.rs::streams_are_typed_and_only_launched_streams_lower_to_derived_exec_tasks`; `crates/agent-spec/tests/discovery.rs::stream_names_launches_and_task_collisions_fail_closed`; `src/agent_author.rs::stream_add_supports_external_command_and_argv_and_external_remove_is_idempotent`; `src/agent_author.rs::stream_authoring_enforces_authority_nix_ownership_and_canonical_validation`; `tests/run.rs::fresh_compact_agent_launches_with_its_derived_ding`; `tests/run.rs::held_adopt_only_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::parked_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::retired_compact_agent_stops_agent_and_derived_ding`; `tests/run.rs::selected_missing_derived_ding_is_held_without_broadening_to_its_agent`; `tests/run.rs::fresh_compact_agent_launches_with_its_derived_stream`; `tests/run.rs::retired_compact_agent_stops_agent_and_derived_stream`; `tests/run.rs::suspended_compact_agent_stops_its_derived_stream_without_touching_a_sibling`; `tests/run.rs::held_adopt_only_compact_agent_stops_its_live_derived_stream`; `tests/run.rs::a_crash_looping_stream_parks_and_surfaces_without_disturbing_its_agent`; `tests/run.rs::parked_compact_agent_stops_its_live_derived_stream`; `tests/run.rs::selected_missing_derived_stream_is_held_without_broadening_to_its_agent`; `tests/run.rs::a_launched_stream_alone_is_rejected_before_reconciliation`; `tests/run.rs::a_stream_does_not_claim_a_delivery_transport` | | **Exactly-once-safe native bus** | Messages use stable `-.md` files. An archive filename is a durable receipt that shadows and cleans restored inbox replicas and makes repeated archive cleanup idempotent. | `src/message.rs::filename_grammar`; `src/message.rs::archive_receipt_suppresses_and_idempotently_cleans_a_restored_inbox_copy`; `tests/message.rs` | | **Idempotent service requests** | A declared non-agent service principal publishes one exact JSON request per caller-supplied idempotency key to a canonical Agent Spec inbox. Concurrent or crash-replayed publication reuses the reserved filename; conflicting key reuse fails. The typed reply routes to the principal's canonical inbox without an Agent Spec identity or orphan mailbox. | `tests/request_cli.rs::stable_request_key_atomically_deduplicates_one_canonical_agent_message`; `tests/request_cli.rs::concurrent_replays_publish_exactly_one_request`; `tests/request_cli.rs::typed_reply_routes_to_the_principal_and_status_is_a_tagged_json_union`; `tests/request_cli.rs::request_api_rejects_agent_impersonation_and_unknown_flat_principals` | | **Bounded idempotent stream ingress** | A running agent accepts events only for a declared stream. Within the retained 128-receipt ring, concurrent or crash-replayed `(stream, event-id)` publication reuses one canonical filename, conflicting content fails, and supersession archives only the matching keyed predecessor or the stream-wide head through ordinary archive semantics. State remains bounded and honestly treats an identity evicted from the ring as new without searching inbox or archive history. Events do not write the Sent ledger and DING marks them as stream work. | `tests/event_e2e.rs::stable_event_identity_publishes_exactly_one_canonical_message`; `tests/event_e2e.rs::concurrent_replays_publish_exactly_one_event`; `tests/event_e2e.rs::conflicting_reuse_and_undeclared_or_suspended_ingress_fail_closed`; `tests/event_e2e.rs::supersede_collapses_only_the_matching_key_and_preserves_archive_receipts`; `tests/event_e2e.rs::keyless_supersede_replaces_the_stream_wide_head`; `tests/event_e2e.rs::crash_replay_honors_an_archive_receipt_and_never_restores_the_inbox_copy`; `tests/event_e2e.rs::subject_frontmatter_injection_is_refused_before_any_write`; `tests/event_e2e.rs::stream_state_is_bounded_and_forgets_only_beyond_its_honest_horizon`; `tests/event_e2e.rs::event_emit_cli_returns_a_stable_json_receipt_and_ding_marks_the_record`; `tests/stream_authoring_cli.rs::a_direct_adapter_launch_executes_the_exact_event_cli_contract`; `tests/run.rs::suspend_and_resume_relaunch_the_agent_and_stream_together` | @@ -19,7 +19,7 @@ materialization, messaging, DING, or presence must preserve them. | **Mutation-only filesystem wakeups** | Supervisor and DING filesystem watchers ignore read/open access events and wake early only for create, modify, rename, or remove events. Their own catalog and inbox reads therefore cannot bypass the bounded timer cadence or form a Linux inotify CPU loop. | `src/watch.rs::only_mutations_wake_watch_loops`; `src/watch.rs::linux_reads_are_silent_but_real_mutations_wake`; `src/ding/mod.rs::idle_ding_does_not_spin_on_its_own_inbox_reads`; `src/run.rs::idle_supervisor_does_not_spin_on_its_own_catalog_reads` | | **Bounded DING PTY probe churn** | An unsafe or active composer retains its FIFO notice but deferred delivery retries use a bounded backoff, so each inbox poll cannot spawn another short-lived PTY probe. | `src/ding/mod.rs::deferred_delivery_backoff_bounds_short_lived_pty_attempts` | | **Agent-declared presence discipline** | The shipped bus contract requires agents to declare `busy` before executing work, use `available` only while yielding or ready, and reserve `dnd` for an explicit hold. Both native harnesses materialize that contract. Busy remains observable but does not suppress DING; fresh `dnd` is the only delivery gate. | `tests/native_only.rs::clean_path_executes_the_maintained_native_authoring_guide`; `src/ding/mod.rs::pending_delivery_ignores_busy_but_respects_fresh_dnd_archive_and_retry` | -| **Stable roster JSON** | `st2 agents --json [--enrich]` preserves field names, order, null handling, presence, typed desired state and rationale, the retirement compatibility projection, opaque declared Resource descriptors, origin-timed activity, and inbox counts. Presence remains independent from desired lifecycle. | `src/agents.rs::agents_json_has_stable_wire_shape`; `src/agents.rs::agents_json_preserves_opaque_declared_resource_descriptors`; `tests/status_agents.rs::roster_json_and_human_output_distinguish_retirement_from_presence`; `tests/status_agents.rs::roster_keeps_presence_separate_from_suspended_desired_state`; `tests/status_agents.rs::roster_uses_version_1_origin_time_for_last_activity` | +| **Stable roster JSON** | `st2 agents --json [--enrich]` preserves field names, order, null handling, presence, typed desired state and rationale, the retirement compatibility projection, opaque declared Resource descriptors, origin-timed activity, inbox counts, and the appended `observedState` object. Declared presence, desired lifecycle, and observed harness state are three independent axes: none is derived from another, and a missing observed record is `null`, never a derived `unknown`. | `src/agents.rs::agents_json_has_stable_wire_shape`; `src/agents.rs::agents_json_preserves_opaque_declared_resource_descriptors`; `src/agents.rs::observed_state_joins_declared_presence_without_touching_either`; `tests/status_agents.rs::roster_json_and_human_output_distinguish_retirement_from_presence`; `tests/status_agents.rs::roster_keeps_presence_separate_from_suspended_desired_state`; `tests/status_agents.rs::roster_uses_version_1_origin_time_for_last_activity` | | **Agent-declared presence** | Refresh preserves non-DND declared status and advances the version 1 heartbeat. A missing status starts as `available`. Legacy DND migrates without renewing its hold. Version 1 DND is not refreshed. Stale, malformed, or implausibly future heartbeats read as `unknown`. The outer Codex, Claude, and pi session wrappers own a five-minute heartbeat while their provider remains alive. | `src/status.rs::refresh_preserves_value_and_changes_heartbeat_bytes`; `src/status.rs::refresh_upgrades_legacy_dnd_without_renewing_the_hold`; `src/status.rs::refresh_missing_writes_available_default`; `src/status.rs::version_1_staleness_and_future_skew_are_bounded`; `src/status.rs::malformed_versioned_record_is_unknown_without_mtime_fallback`; `src/claude_session.rs::idle_provider_refreshes_presence_without_mcp_input`; `src/pi_session.rs::idle_pi_provider_refreshes_presence_without_channel_input`; `src/codex_app_server.rs::inbox_fallback_does_not_write_a_fifteen_second_presence_heartbeat` | | **Retirement health** | A retired declaration is healthy only after every declared task ID is absent. Any live or dead declared task record reports incomplete retirement; retired declarations do not require presence. Live declarations retain their existing task and presence checks. | `tests/doctor.rs::retired_declaration_is_healthy_when_tasks_and_presence_are_absent`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_declared_task_is_alive`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_dead_task_record_remains` | | **Suspension health** | A suspended declaration is healthy when no declared task is live and every retained dead record is explicitly keep-pinned. It requires no presence, but this weaker result never proves retirement. Resume preserves ordinary keep and adopt-only policy. | `tests/doctor.rs::suspended_declaration_is_healthy_when_tasks_are_absent_without_presence`; `tests/doctor.rs::suspended_declaration_distinguishes_live_dead_keep_and_dead_nonkeep`; `tests/reconcile.rs::resuming_uses_ordinary_reconcile_and_does_not_override_keep` | diff --git a/src/agents.rs b/src/agents.rs index a2424b46..8e5ea284 100644 --- a/src/agents.rs +++ b/src/agents.rs @@ -10,7 +10,7 @@ use serde::Serialize; use crate::message; use crate::status::{self, State}; -use crate::{Discovered, Resource}; +use crate::{AgentSpec, Discovered, Resource, harness_state}; /// One roster row: everything `st2 agents [--enrich]` can report about an agent. #[derive(Debug, Clone)] @@ -36,6 +36,10 @@ pub struct AgentRow { pub last_activity_ms: Option, /// Count of canonical message files in the agent's inbox. `--enrich` only. pub inbox: usize, + /// Observed harness state — the driver-owned signal of what the harness is seen doing, a third + /// axis independent from declared presence and from desired lifecycle. `None` means no driver + /// has ever published a record for this agent, which is different from a derived `unknown`. + pub observed: Option, } /// Every agent in the catalog, sorted by bus id, with presence + enrich data computed. Read-only: @@ -64,6 +68,7 @@ pub fn roster_from_discovered(found: &Discovered, this_host: &str) -> Vec Vec Option { + let path = harness_state::harness_state_path(agent_dir); + if spec.resolved_host(this_host) == this_host { + let probe: &dyn Fn(&str) -> harness_state::SessionLiveness = &crate::ding::session_liveness; + harness_state::read(&path, Some(probe)) + } else { + harness_state::read(&path, None) + } +} + +/// The `observedState` object inside a roster row. Vocabulary words are the record's own +/// (`as_str`), never Rust identifier spellings. +#[derive(Serialize)] +#[serde(rename_all = "camelCase")] +struct ObservedJson<'a> { + state: &'a str, + blocked_on: &'a str, + input_buffer: &'a str, + harness: Option<&'a str>, + since: Option, + #[serde(skip_serializing_if = "Option::is_none")] + exit: Option<&'a str>, + #[serde(skip_serializing_if = "Option::is_none")] + reason: Option<&'a str>, +} + +impl<'a> ObservedJson<'a> { + fn from_row(observed: Option<&'a harness_state::Observed>) -> Option { + observed.map(|observed| ObservedJson { + state: observed.state.as_str(), + blocked_on: observed.blocked_on.as_str(), + input_buffer: observed.input_buffer.as_str(), + harness: observed.harness.as_deref(), + since: observed.since_ms, + exit: observed.exit.as_deref(), + reason: observed.reason.as_deref(), + }) + } +} + /// `st2 agents --json` row. Field order and names are the stable wire contract. #[derive(Serialize)] struct SummaryJson<'a> { @@ -84,6 +136,8 @@ struct SummaryJson<'a> { desired_state: &'a str, #[serde(rename = "desiredStateReason")] desired_state_reason: Option<&'a str>, + #[serde(rename = "observedState")] + observed_state: Option>, } /// `st2 agents --json --enrich` row (adds `lastActivity` and `inbox`). @@ -102,6 +156,8 @@ struct EnrichedJson<'a> { desired_state: &'a str, #[serde(rename = "desiredStateReason")] desired_state_reason: Option<&'a str>, + #[serde(rename = "observedState")] + observed_state: Option>, } /// Serialize a roster to the stable JSON emitted by `st2 agents --json [--enrich]`. @@ -120,6 +176,7 @@ pub fn to_json(rows: &[AgentRow], enrich: bool) -> String { desired_state_reason: r.desired_state_reason.as_deref(), last_activity: r.last_activity_ms, inbox: r.inbox, + observed_state: ObservedJson::from_row(r.observed.as_ref()), }) .collect(); serde_json::to_string(&out).unwrap_or_else(|_| "[]".to_string()) @@ -135,6 +192,7 @@ pub fn to_json(rows: &[AgentRow], enrich: bool) -> String { resources: &r.resources, desired_state: &r.desired_state, desired_state_reason: r.desired_state_reason.as_deref(), + observed_state: ObservedJson::from_row(r.observed.as_ref()), }) .collect(); serde_json::to_string(&out).unwrap_or_else(|_| "[]".to_string()) @@ -199,6 +257,7 @@ mod tests { resources: Vec::new(), last_activity_ms: last, inbox, + observed: None, } } @@ -219,11 +278,11 @@ mod tests { assert_eq!( to_json(&rows, false), - r#"[{"identity":"hetz.cos-claude","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null},{"identity":"hetz.st2-claude","status":"busy","name":"owner","description":null,"retired":true,"resources":[],"desiredState":"retired","desiredStateReason":null}]"# + r#"[{"identity":"hetz.cos-claude","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":null},{"identity":"hetz.st2-claude","status":"busy","name":"owner","description":null,"retired":true,"resources":[],"desiredState":"retired","desiredStateReason":null,"observedState":null}]"# ); assert_eq!( to_json(&rows, true), - r#"[{"identity":"hetz.cos-claude","status":"available","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":1,"desiredState":"running","desiredStateReason":null},{"identity":"hetz.st2-claude","status":"busy","name":"owner","description":null,"retired":true,"resources":[],"lastActivity":null,"inbox":0,"desiredState":"retired","desiredStateReason":null}]"# + r#"[{"identity":"hetz.cos-claude","status":"available","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":1,"desiredState":"running","desiredStateReason":null,"observedState":null},{"identity":"hetz.st2-claude","status":"busy","name":"owner","description":null,"retired":true,"resources":[],"lastActivity":null,"inbox":0,"desiredState":"retired","desiredStateReason":null,"observedState":null}]"# ); // Empty roster is `[]`, not `null`. assert_eq!(to_json(&[], true), "[]"); @@ -243,7 +302,55 @@ mod tests { assert_eq!( to_json(&[resource_row], false), - r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[{"name":"work","uri":"vendor+thing://authority/exact%20identity","reason":"Current implementation task."}],"desiredState":"running","desiredStateReason":null}]"# + r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[{"name":"work","uri":"vendor+thing://authority/exact%20identity","reason":"Current implementation task."}],"desiredState":"running","desiredStateReason":null,"observedState":null}]"# + ); + } + + /// Declared presence and observed harness state are independent axes in one payload: a + /// declared `busy` sits beside an observed `idle` (the wedged-agent signal), and + /// `lastActivity` keeps its existing meaning untouched by the new field. + #[test] + fn observed_state_joins_declared_presence_without_touching_either() { + let mut wedged = row( + "hetz.worker", + State::Busy, + None, + false, + Some(1784653027733.6138), + 0, + ); + wedged.observed = Some(harness_state::Observed { + state: harness_state::Activity::Idle, + blocked_on: harness_state::BlockedOn::None, + input_buffer: harness_state::InputBuffer::Empty, + harness: Some("codex".to_string()), + since_ms: Some(1784653000000), + exit: None, + reason: None, + }); + + assert_eq!( + to_json(&[wedged.clone()], false), + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000}}]"# + ); + assert_eq!( + to_json(&[wedged], true), + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":0,"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000}}]"# + ); + + let mut derived = row("hetz.worker", State::Available, None, false, None, 0); + derived.observed = Some(harness_state::Observed { + state: harness_state::Activity::Unknown, + blocked_on: harness_state::BlockedOn::Unknown, + input_buffer: harness_state::InputBuffer::Unknown, + harness: Some("codex".to_string()), + since_ms: None, + exit: None, + reason: Some("session-dead".to_string()), + }); + assert_eq!( + to_json(&[derived], false), + r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"unknown","blockedOn":"unknown","inputBuffer":"unknown","harness":"codex","since":null,"reason":"session-dead"}}]"# ); } } diff --git a/src/ding/mod.rs b/src/ding/mod.rs index b7650104..d8c7ca8e 100644 --- a/src/ding/mod.rs +++ b/src/ding/mod.rs @@ -709,6 +709,34 @@ pub fn session_alive(session: &str) -> bool { pid > 0 && unsafe { libc::kill(pid, 0) == 0 } } +/// Positive-evidence session liveness for observed-harness-state readers. Unlike +/// [`session_alive`], whose delivery callers must fail closed ("any miss means gone"), a reader +/// deriving `unknown` needs proof of death: an unreadable or unparseable pidfile is +/// `Indeterminate` — the reader may not share the writer's PTY root — and a pid that exists but +/// is not signalable (EPERM) is still alive. +pub fn session_liveness(session: &str) -> crate::harness_state::SessionLiveness { + use crate::harness_state::SessionLiveness; + let pidfile = pty_session_dir().join(format!("{session}.pid")); + let Ok(raw) = std::fs::read_to_string(&pidfile) else { + return SessionLiveness::Indeterminate; + }; + let Ok(pid) = raw.trim().parse::() else { + return SessionLiveness::Indeterminate; + }; + if pid <= 0 { + return SessionLiveness::Indeterminate; + } + if unsafe { libc::kill(pid, 0) } == 0 { + return SessionLiveness::Alive; + } + match std::io::Error::last_os_error().raw_os_error() { + Some(code) if code == libc::ESRCH => SessionLiveness::Dead, + // EPERM proves existence; anything else proves nothing. + Some(code) if code == libc::EPERM => SessionLiveness::Alive, + _ => SessionLiveness::Indeterminate, + } +} + /// The `pty` session registry dir. This must mirror the sibling tool's resolution order. fn pty_session_dir() -> PathBuf { for var in ["PTY_ROOT", "PTY_SESSION_DIR"] { diff --git a/src/main.rs b/src/main.rs index 94a618fd..1e4a20c5 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1813,6 +1813,35 @@ fn doctor_cmd(root: &Path, host: Option, require_supervisor: bool) -> Re "rotted to `unknown` — is its session owner refreshing presence?", ); } + // Observed harness state is advisory-only in doctor: absence names a driver gap and + // a derived `unknown` names its reason, but neither fails the exit code. + let observed_path = st2::harness_state::harness_state_path(dir); + let probe: &dyn Fn(&str) -> st2::harness_state::SessionLiveness = + &st2::ding::session_liveness; + match st2::harness_state::read(&observed_path, Some(probe)) { + None => report_advisory( + &format!("{bus_id} observed harness state absent"), + "no driver has published a harness-state record for this agent", + ), + Some(observed) if observed.state == st2::harness_state::Activity::Unknown => { + report_advisory( + &format!("{bus_id} observed harness state indeterminate"), + &format!( + "derived `unknown` ({}) — is its driver still observing the harness?", + observed.reason.as_deref().unwrap_or("unstated") + ), + ) + } + Some(observed) => report_check( + &mut problems, + true, + &format!( + "{bus_id} observed harness state fresh (is `{}`)", + observed.state.as_str() + ), + "", + ), + } } } @@ -2151,9 +2180,10 @@ fn agents_cmd( ) }; println!( - "{}\t{}\t{}\t{}{}", + "{}\t{}\t{}\t{}\t{}{}", r.identity, r.status.as_str(), + observed_column(r.observed.as_ref()), r.name.as_deref().unwrap_or(""), r.description.as_deref().unwrap_or(""), lifecycle, @@ -2163,6 +2193,24 @@ fn agents_cmd( Ok(()) } +/// The compact observed-harness-state column for human `st2 agents` output. `-` means no record +/// exists (no driver has published one), which is distinct from a derived `unknown`. +fn observed_column(observed: Option<&st2::harness_state::Observed>) -> String { + let Some(observed) = observed else { + return "-".to_string(); + }; + let mut column = observed.state.as_str().to_string(); + if observed.blocked_on == st2::harness_state::BlockedOn::Human { + column.push_str("+human"); + } + if observed.state == st2::harness_state::Activity::Unknown + && let Some(reason) = observed.reason.as_deref() + { + column = format!("unknown({reason})"); + } + column +} + fn ding_cmd( session: Option, identity: Option, diff --git a/tests/status_agents.rs b/tests/status_agents.rs index 44cb3220..41fc5bd5 100644 --- a/tests/status_agents.rs +++ b/tests/status_agents.rs @@ -286,7 +286,7 @@ fn roster_json_and_human_output_distinguish_retirement_from_presence() { ); assert_eq!( String::from_utf8(human.stdout).unwrap(), - "h.live\tavailable\t\t\nh.retired\tbusy\t\t\t[retired]\n" + "h.live\tavailable\t-\t\t\nh.retired\tbusy\t-\t\t\t[retired]\n" ); } From 78320c0d401af792e8de960d4443413f545640fa Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Sun, 23 Aug 2026 21:00:22 +0200 Subject: [PATCH 2/5] fix(agents): catalog-derived probe root, explicit nulls, and labeled observed output Review-pass fixes for the exposure slice: the session-liveness probe derives the runner's own pty root from the catalog (PTY_ROOT and PTY_SESSION_DIR still win) so operator shells get the cross-check without exporting anything, with the errno mapping now unit-tested; observedState serializes exit and reason as explicit null per the roster's null-not-omit convention; Doctor gains the ended-while-desired-running crashed-seat advisory and tests covering all four observed-state arms; and the human observed column is prefixed obs: so two bare state words never sit ambiguously side by side. Co-Authored-By: Claude Fable 5 --- src/agents.rs | 38 ++++++++++++++++------ src/ding/mod.rs | 50 ++++++++++++++++++++++++++++- src/main.rs | 26 ++++++++++++---- tests/doctor.rs | 71 ++++++++++++++++++++++++++++++++++++++++++ tests/status_agents.rs | 2 +- 5 files changed, 169 insertions(+), 18 deletions(-) diff --git a/src/agents.rs b/src/agents.rs index 8e5ea284..540a2941 100644 --- a/src/agents.rs +++ b/src/agents.rs @@ -46,12 +46,17 @@ pub struct AgentRow { /// walks discovered specs and each agent's resources, mutating nothing. pub fn roster(catalog_root: &Path, this_host: &str) -> Vec { let found = crate::discover(catalog_root); - roster_from_discovered(&found, this_host) + roster_from_discovered(&found, catalog_root, this_host) } /// Project a roster from one immutable discovery result. Exact selectors use this after proving /// discovery complete so the uniqueness check and returned metadata describe the same snapshot. -pub fn roster_from_discovered(found: &Discovered, this_host: &str) -> Vec { +pub fn roster_from_discovered( + found: &Discovered, + catalog_root: &Path, + this_host: &str, +) -> Vec { + let pty_root = probe_pty_root(catalog_root); let mut rows: Vec = found .specs .iter() @@ -68,7 +73,7 @@ pub fn roster_from_discovered(found: &Discovered, this_host: &str) -> Vec Vec Option { let path = harness_state::harness_state_path(agent_dir); if spec.resolved_host(this_host) == this_host { - let probe: &dyn Fn(&str) -> harness_state::SessionLiveness = &crate::ding::session_liveness; - harness_state::read(&path, Some(probe)) + let probe = |session: &str| crate::ding::session_liveness_in(pty_root, session); + harness_state::read(&path, Some(&probe)) } else { harness_state::read(&path, None) } } +/// The pty registry root the probe reads. Operator shells rarely export PTY_ROOT, while +/// supervised tasks run under the catalog's own pty root — so the reader derives the runner's +/// default from the catalog, with the ambient overrides still winning. +pub fn probe_pty_root(catalog_root: &Path) -> PathBuf { + for var in ["PTY_ROOT", "PTY_SESSION_DIR"] { + if let Some(dir) = std::env::var_os(var) + && !dir.is_empty() + { + return PathBuf::from(dir); + } + } + crate::run::effective_pty_root(catalog_root) +} + /// The `observedState` object inside a roster row. Vocabulary words are the record's own /// (`as_str`), never Rust identifier spellings. #[derive(Serialize)] @@ -103,9 +123,7 @@ struct ObservedJson<'a> { input_buffer: &'a str, harness: Option<&'a str>, since: Option, - #[serde(skip_serializing_if = "Option::is_none")] exit: Option<&'a str>, - #[serde(skip_serializing_if = "Option::is_none")] reason: Option<&'a str>, } @@ -331,11 +349,11 @@ mod tests { assert_eq!( to_json(&[wedged.clone()], false), - r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000}}]"# + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# ); assert_eq!( to_json(&[wedged], true), - r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":0,"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000}}]"# + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":0,"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# ); let mut derived = row("hetz.worker", State::Available, None, false, None, 0); @@ -350,7 +368,7 @@ mod tests { }); assert_eq!( to_json(&[derived], false), - r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"unknown","blockedOn":"unknown","inputBuffer":"unknown","harness":"codex","since":null,"reason":"session-dead"}}]"# + r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"unknown","blockedOn":"unknown","inputBuffer":"unknown","harness":"codex","since":null,"exit":null,"reason":"session-dead"}}]"# ); } } diff --git a/src/ding/mod.rs b/src/ding/mod.rs index d8c7ca8e..e81f6a54 100644 --- a/src/ding/mod.rs +++ b/src/ding/mod.rs @@ -715,8 +715,18 @@ pub fn session_alive(session: &str) -> bool { /// `Indeterminate` — the reader may not share the writer's PTY root — and a pid that exists but /// is not signalable (EPERM) is still alive. pub fn session_liveness(session: &str) -> crate::harness_state::SessionLiveness { + session_liveness_in(&pty_session_dir(), session) +} + +/// [`session_liveness`], probing an explicit registry root instead of the ambient environment — +/// readers that know the catalog derive the runner's own root rather than requiring PTY_ROOT in +/// the shell. +pub fn session_liveness_in( + root: &std::path::Path, + session: &str, +) -> crate::harness_state::SessionLiveness { use crate::harness_state::SessionLiveness; - let pidfile = pty_session_dir().join(format!("{session}.pid")); + let pidfile = root.join(format!("{session}.pid")); let Ok(raw) = std::fs::read_to_string(&pidfile) else { return SessionLiveness::Indeterminate; }; @@ -3779,4 +3789,42 @@ Enter to select · ↑/↓ to navigate · Esc to cancel"; "idle DING must sleep at its configured cadence" ); } + + #[test] + fn session_liveness_probe_reports_only_positive_evidence() { + use crate::harness_state::SessionLiveness; + let tmp = tempfile::tempdir().unwrap(); + + // An unreadable registry proves nothing: missing or garbled pidfiles downgrade nothing. + assert_eq!( + session_liveness_in(tmp.path(), "absent"), + SessionLiveness::Indeterminate + ); + std::fs::write(tmp.path().join("garbled.pid"), "not-a-pid\n").unwrap(); + assert_eq!( + session_liveness_in(tmp.path(), "garbled"), + SessionLiveness::Indeterminate + ); + + // Our own pid is positive evidence of life. + std::fs::write( + tmp.path().join("live.pid"), + format!("{}\n", std::process::id()), + ) + .unwrap(); + assert_eq!( + session_liveness_in(tmp.path(), "live"), + SessionLiveness::Alive + ); + + // A reaped child is positive evidence of death (ESRCH). + let mut child = std::process::Command::new("true").spawn().unwrap(); + let pid = child.id(); + child.wait().unwrap(); + std::fs::write(tmp.path().join("dead.pid"), format!("{pid}\n")).unwrap(); + assert_eq!( + session_liveness_in(tmp.path(), "dead"), + SessionLiveness::Dead + ); + } } diff --git a/src/main.rs b/src/main.rs index 1e4a20c5..2bfd6c75 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1816,9 +1816,9 @@ fn doctor_cmd(root: &Path, host: Option, require_supervisor: bool) -> Re // Observed harness state is advisory-only in doctor: absence names a driver gap and // a derived `unknown` names its reason, but neither fails the exit code. let observed_path = st2::harness_state::harness_state_path(dir); - let probe: &dyn Fn(&str) -> st2::harness_state::SessionLiveness = - &st2::ding::session_liveness; - match st2::harness_state::read(&observed_path, Some(probe)) { + let pty_root = st2::agents::probe_pty_root(&catalog); + let probe = |session: &str| st2::ding::session_liveness_in(&pty_root, session); + match st2::harness_state::read(&observed_path, Some(&probe)) { None => report_advisory( &format!("{bus_id} observed harness state absent"), "no driver has published a harness-state record for this agent", @@ -1832,6 +1832,18 @@ fn doctor_cmd(root: &Path, host: Option, require_supervisor: bool) -> Re ), ) } + Some(observed) + if observed.state == st2::harness_state::Activity::Ended + && spec.desired_state.as_str() == "running" => + { + report_advisory( + &format!("{bus_id} observed harness state ended"), + &format!( + "session ended ({}) while desired state is running — crashed seat?", + observed.exit.as_deref().unwrap_or("exit unstated") + ), + ) + } Some(observed) => report_check( &mut problems, true, @@ -2145,7 +2157,7 @@ fn agents_cmd( found.errors.len() ); } - let mut rows = st2::agents::roster_from_discovered(&found, &host); + let mut rows = st2::agents::roster_from_discovered(&found, &root, &host); if let Some(identity) = &identity { rows.retain(|row| row.identity == *identity); anyhow::ensure!( @@ -2196,8 +2208,10 @@ fn agents_cmd( /// The compact observed-harness-state column for human `st2 agents` output. `-` means no record /// exists (no driver has published one), which is distinct from a derived `unknown`. fn observed_column(observed: Option<&st2::harness_state::Observed>) -> String { + // Prefixed so the observed word is never mistaken for the declared presence beside it — + // two bare state words in one row is the exact ambiguity the ontology's collision rules name. let Some(observed) = observed else { - return "-".to_string(); + return "obs:-".to_string(); }; let mut column = observed.state.as_str().to_string(); if observed.blocked_on == st2::harness_state::BlockedOn::Human { @@ -2208,7 +2222,7 @@ fn observed_column(observed: Option<&st2::harness_state::Observed>) -> String { { column = format!("unknown({reason})"); } - column + format!("obs:{column}") } fn ding_cmd( diff --git a/tests/doctor.rs b/tests/doctor.rs index b0da7b42..ab2f946f 100644 --- a/tests/doctor.rs +++ b/tests/doctor.rs @@ -442,3 +442,74 @@ fn missing_delivery_is_advisory_while_an_invalid_delivery_is_a_catalog_problem() "{stdout}" ); } + +#[test] +fn observed_harness_state_arms_are_advisory_except_a_fresh_live_record() { + use st2::harness_state::{Activity, BlockedOn, InputBuffer, Observation, Writer}; + + let tmp = tempfile::tempdir().unwrap(); + let catalog = tmp.path().join("catalog"); + let declaration = catalog.join("agents/h/worker/agent.kdl"); + let bin = tmp.path().join("bin"); + fs::create_dir_all(declaration.parent().unwrap()).unwrap(); + fs::create_dir_all(&bin).unwrap(); + fs::write( + &declaration, + r#"agent "worker" { host "h"; command "true"; deliver "mcp" }"#, + ) + .unwrap(); + let agent_dir = declaration.parent().unwrap().to_path_buf(); + fs::write(agent_dir.join("status"), "available\n").unwrap(); + executable( + &bin.join("pty"), + "#!/bin/sh\nif [ \"$1\" = list ]; then printf '[{\"name\":\"h.worker\",\"status\":\"running\"}]\\n'; fi\n", + ); + + // No record: a driver gap is an advisory, never a failing check. + let absent = doctor(&catalog, &bin, &tmp.path().join("state")); + let stdout = String::from_utf8_lossy(&absent.stdout); + assert!(absent.status.success(), "{stdout}"); + assert!( + stdout.contains("⚠ h.worker observed harness state absent"), + "{stdout}" + ); + + // A fresh live record is a passing check. + let mut writer = Writer::new(&agent_dir, "h.worker", "codex", None); + writer + .observe(Observation::new( + Activity::Active, + BlockedOn::None, + InputBuffer::Unknown, + )) + .unwrap(); + let live = doctor(&catalog, &bin, &tmp.path().join("state")); + let stdout = String::from_utf8_lossy(&live.stdout); + assert!(live.status.success(), "{stdout}"); + assert!( + stdout.contains("h.worker observed harness state fresh (is `active`)"), + "{stdout}" + ); + + // A terminal record while the declaration wants the seat running is the crashed-seat signal. + writer.ended("signal 9").unwrap(); + let ended = doctor(&catalog, &bin, &tmp.path().join("state")); + let stdout = String::from_utf8_lossy(&ended.stdout); + assert!(ended.status.success(), "{stdout}"); + assert!( + stdout.contains("⚠ h.worker observed harness state ended"), + "{stdout}" + ); + assert!(stdout.contains("session ended (signal 9)"), "{stdout}"); + + // A record that derives `unknown` names its reason, still advisory. + fs::write(agent_dir.join("harness-state"), "garbage").unwrap(); + let indeterminate = doctor(&catalog, &bin, &tmp.path().join("state")); + let stdout = String::from_utf8_lossy(&indeterminate.stdout); + assert!(indeterminate.status.success(), "{stdout}"); + assert!( + stdout.contains("⚠ h.worker observed harness state indeterminate"), + "{stdout}" + ); + assert!(stdout.contains("(malformed-record)"), "{stdout}"); +} diff --git a/tests/status_agents.rs b/tests/status_agents.rs index 41fc5bd5..66af0af0 100644 --- a/tests/status_agents.rs +++ b/tests/status_agents.rs @@ -286,7 +286,7 @@ fn roster_json_and_human_output_distinguish_retirement_from_presence() { ); assert_eq!( String::from_utf8(human.stdout).unwrap(), - "h.live\tavailable\t-\t\t\nh.retired\tbusy\t-\t\t\t[retired]\n" + "h.live\tavailable\tobs:-\t\t\nh.retired\tbusy\tobs:-\t\t\t[retired]\n" ); } From b549bb41babca52567f91f0a7c52479e9849f4ae Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Sun, 23 Aug 2026 21:51:31 +0200 Subject: [PATCH 3/5] fix(agents): probe exactly the runner's pty root and expose the ask kind PTY_SESSION_DIR pointed the probe at a directory st2-managed sessions never use (provable deaths read indeterminate); the probe now shares run::effective_pty_root verbatim. observedState gains the machine-readable ask axis, pinned literals updated. Co-Authored-By: Claude Fable 5 --- src/agents.rs | 24 +++++++++++------------- tests/doctor.rs | 7 ++++++- 2 files changed, 17 insertions(+), 14 deletions(-) diff --git a/src/agents.rs b/src/agents.rs index 540a2941..c322277c 100644 --- a/src/agents.rs +++ b/src/agents.rs @@ -99,17 +99,11 @@ fn observed_state( } } -/// The pty registry root the probe reads. Operator shells rarely export PTY_ROOT, while -/// supervised tasks run under the catalog's own pty root — so the reader derives the runner's -/// default from the catalog, with the ambient overrides still winning. +/// The pty registry root the probe reads: exactly the runner's own resolution, so the reader and +/// the sessions it probes can never disagree. The runner honors `PTY_ROOT` and nothing else — a +/// legacy `PTY_SESSION_DIR` here would point the probe at a directory st2-managed sessions never +/// use, turning provable deaths into indeterminate reads. pub fn probe_pty_root(catalog_root: &Path) -> PathBuf { - for var in ["PTY_ROOT", "PTY_SESSION_DIR"] { - if let Some(dir) = std::env::var_os(var) - && !dir.is_empty() - { - return PathBuf::from(dir); - } - } crate::run::effective_pty_root(catalog_root) } @@ -121,6 +115,7 @@ struct ObservedJson<'a> { state: &'a str, blocked_on: &'a str, input_buffer: &'a str, + ask: &'a str, harness: Option<&'a str>, since: Option, exit: Option<&'a str>, @@ -133,6 +128,7 @@ impl<'a> ObservedJson<'a> { state: observed.state.as_str(), blocked_on: observed.blocked_on.as_str(), input_buffer: observed.input_buffer.as_str(), + ask: observed.ask.as_str(), harness: observed.harness.as_deref(), since: observed.since_ms, exit: observed.exit.as_deref(), @@ -341,6 +337,7 @@ mod tests { state: harness_state::Activity::Idle, blocked_on: harness_state::BlockedOn::None, input_buffer: harness_state::InputBuffer::Empty, + ask: harness_state::Ask::None, harness: Some("codex".to_string()), since_ms: Some(1784653000000), exit: None, @@ -349,11 +346,11 @@ mod tests { assert_eq!( to_json(&[wedged.clone()], false), - r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","ask":"none","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# ); assert_eq!( to_json(&[wedged], true), - r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":0,"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":0,"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","ask":"none","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# ); let mut derived = row("hetz.worker", State::Available, None, false, None, 0); @@ -361,6 +358,7 @@ mod tests { state: harness_state::Activity::Unknown, blocked_on: harness_state::BlockedOn::Unknown, input_buffer: harness_state::InputBuffer::Unknown, + ask: harness_state::Ask::Unknown, harness: Some("codex".to_string()), since_ms: None, exit: None, @@ -368,7 +366,7 @@ mod tests { }); assert_eq!( to_json(&[derived], false), - r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"unknown","blockedOn":"unknown","inputBuffer":"unknown","harness":"codex","since":null,"exit":null,"reason":"session-dead"}}]"# + r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"unknown","blockedOn":"unknown","inputBuffer":"unknown","ask":"unknown","harness":"codex","since":null,"exit":null,"reason":"session-dead"}}]"# ); } } diff --git a/tests/doctor.rs b/tests/doctor.rs index ab2f946f..9357f5ce 100644 --- a/tests/doctor.rs +++ b/tests/doctor.rs @@ -475,7 +475,12 @@ fn observed_harness_state_arms_are_advisory_except_a_fresh_live_record() { ); // A fresh live record is a passing check. - let mut writer = Writer::new(&agent_dir, "h.worker", "codex", None); + let mut writer = Writer::new( + &agent_dir, + "h.worker", + "codex", + Some("h.worker".to_string()), + ); writer .observe(Observation::new( Activity::Active, From 4cff5f74aac9d5176c8ab3e25925a095ac5cb10f Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Mon, 24 Aug 2026 00:44:05 +0200 Subject: [PATCH 4/5] fix(agents): observedState field order matches the documented contract reason precedes exit, as the spec example pins; pinned literals updated in the same change. Co-Authored-By: Claude Fable 5 --- src/agents.rs | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/agents.rs b/src/agents.rs index c322277c..b1031906 100644 --- a/src/agents.rs +++ b/src/agents.rs @@ -118,8 +118,8 @@ struct ObservedJson<'a> { ask: &'a str, harness: Option<&'a str>, since: Option, - exit: Option<&'a str>, reason: Option<&'a str>, + exit: Option<&'a str>, } impl<'a> ObservedJson<'a> { @@ -131,8 +131,8 @@ impl<'a> ObservedJson<'a> { ask: observed.ask.as_str(), harness: observed.harness.as_deref(), since: observed.since_ms, - exit: observed.exit.as_deref(), reason: observed.reason.as_deref(), + exit: observed.exit.as_deref(), }) } } @@ -346,11 +346,11 @@ mod tests { assert_eq!( to_json(&[wedged.clone()], false), - r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","ask":"none","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","ask":"none","harness":"codex","since":1784653000000,"reason":null,"exit":null}}]"# ); assert_eq!( to_json(&[wedged], true), - r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":0,"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","ask":"none","harness":"codex","since":1784653000000,"exit":null,"reason":null}}]"# + r#"[{"identity":"hetz.worker","status":"busy","name":null,"description":null,"retired":false,"resources":[],"lastActivity":1784653027733.6138,"inbox":0,"desiredState":"running","desiredStateReason":null,"observedState":{"state":"idle","blockedOn":"none","inputBuffer":"empty","ask":"none","harness":"codex","since":1784653000000,"reason":null,"exit":null}}]"# ); let mut derived = row("hetz.worker", State::Available, None, false, None, 0); @@ -366,7 +366,7 @@ mod tests { }); assert_eq!( to_json(&[derived], false), - r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"unknown","blockedOn":"unknown","inputBuffer":"unknown","ask":"unknown","harness":"codex","since":null,"exit":null,"reason":"session-dead"}}]"# + r#"[{"identity":"hetz.worker","status":"available","name":null,"description":null,"retired":false,"resources":[],"desiredState":"running","desiredStateReason":null,"observedState":{"state":"unknown","blockedOn":"unknown","inputBuffer":"unknown","ask":"unknown","harness":"codex","since":null,"reason":"session-dead","exit":null}}]"# ); } } From 493cb8c83a77a2e2a90cf2247684da42988fbeed Mon Sep 17 00:00:00 2001 From: schickling-assistant <261620128+schickling-assistant@users.noreply.github.com> Date: Mon, 24 Aug 2026 15:54:29 +0200 Subject: [PATCH 5/5] fix(agents): doctor reports a reason-only terminal instead of exit unstated MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An ended record can carry only a reason — Codex's observed systemError writes reason without an exit — and the catch-all advisory discarded it, leaving the operator 'exit unstated' with nothing to act on. The advisory now falls back to the recorded reason when no exit is present. Co-Authored-By: Claude Fable 5 agent-identity: unknown agent-persona: generalist agent-supervisor: unavailable agent-tool: OMP agent-tool-version: 18.0.3 agent-runtime: OMP 18.0.3 tooling-profile: dotfiles@f33cd9c-dirty --- src/main.rs | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/src/main.rs b/src/main.rs index 2bfd6c75..11586395 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1840,7 +1840,17 @@ fn doctor_cmd(root: &Path, host: Option, require_supervisor: bool) -> Re &format!("{bus_id} observed harness state ended"), &format!( "session ended ({}) while desired state is running — crashed seat?", - observed.exit.as_deref().unwrap_or("exit unstated") + observed + .exit + .as_deref() + .map(|exit| format!("exit {exit}")) + // A terminal record can carry only a reason — Codex's + // observed systemError writes reason without an exit — and + // discarding it leaves the operator nothing to act on. + .or_else(|| { + observed.reason.as_deref().map(|reason| format!("{reason}")) + }) + .unwrap_or_else(|| "exit unstated".to_string()) ), ) }