docs(vrs): record observed harness state experiments - #263
Closed
schickling wants to merge 2 commits into
Closed
Conversation
Four experiment records and one draft decision record for observed harness state: a coarse per-agent lifecycle signal, produced by a harness driver and consumed generically, kept separate from agent-declared presence. These are records of what was measured, not a specification. No requirements text, no ontology entry, no code. Measured and captured: - Codex already carries blocked-on-human on the wire. `ThreadStatus.active` has a required `activeFlags` field; `observe_thread_status` reads only the status word. Read from the 0.147.0 schema, which st2 does not yet support. - Claude can prove the same. The binary carries 31 hook events against 10 documented; `PermissionRequest` fires only when a human is about to be asked. Nine captures are included. - Claude death is silent for SIGTERM and SIGKILL alike, and a wrapper-owned write does not close the gap: `stop_provider_group` escalates into the wrapper's own process group. - A shared append-only history conflicts under Git; host-named segments do not. Retention disagreement between hosts resolves to the minimum. - The transport was not identified. Fabric is unobservable on this host, so no cost model is claimed. The decision record is `Status: proposed` and asserts only the half that survived independent review: observed state is inadmissible as transport authorization. The scheduling half is withdrawn to an open question. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The experiment recorded that `activeFlags` was verified only on codex-cli 0.147.0, and inferred that consuming it required admitting that version. Schemas generated from 0.145.0 and 0.146.0 binaries — both already admitted — carry `ActiveThreadStatus` with `required: ["activeFlags","type"]` and the same two-value `ThreadActiveFlag`. The field is available on a supported version today, so no finding depends on the version pin. What is unchanged: no populated `activeFlags` has been observed on the wire on any version, so the steer-into-approval consequence remains inferred. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
schickling
marked this pull request as ready for review
August 18, 2026 07:00
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
schickling
marked this pull request as draft
August 18, 2026 07:02
This was referenced Aug 18, 2026
Contributor
|
Superseded by the merged observed-harness-state implementation (#319–#326). The experimental conclusions are now represented by the accepted architecture. Posted on behalf of @schickling
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
st2 cannot see what a harness is doing. Presence is agent-declared —
status::refreshproves only that a process is alive and preserves whatever the model last wrote — so a wedged agent and a working one are indistinguishable for up to fifteen minutes. MeanwhileCodexObservedState(src/codex_app_server.rs:166) already models exactly this concept for one harness, typed and persisted underst2.codex-control-state.v1, in a hash-keyed host-local file that nothing outside that module reads. This is the activity-lease slice of #162 — "activity leases (idle,active,child,unknown)" — measured before being specified, asDQ3requires.These are records of what was measured, not a specification. No requirements text, no ontology entry, no code.
What
ThreadStatus.activehas a requiredactiveFlagsfield, andobserve_thread_status(:770) reads only the status word.PermissionRequestfires only when a human is about to be asked — in 2 of 9 captures, and in zerobypassPermissions/autoruns.stop_provider_groupescalates into the wrapper's own process group.0004, asserting only the half that survived independent adversarial review: observed state is inadmissible as transport authorization.Impact
Nothing changes. This adds five documents and their captures under
docs/vrs/, and nothing else.INVARIANTS.mdis unchanged; no invariant is claimed, relaxed, or added.compoundingtech/evals.0004isStatus: proposedand explicitly says it is not accepted.Checks
No Darwin run: no code changed.
Limits, stated up front
Corrected 2026-08-18: schemas generated from 0.145.0 and 0.146.0 binaries — both already admitted byactiveFlagsis unverified on a supported version.SUPPORTED_CODEX_CLI_VERSIONS— carryActiveThreadStatuswithrequired: ["activeFlags","type"]and an identical two-valueThreadActiveFlag. The field is available on a supported version today and no finding here depends on admitting 0.147.0. What stands: a populatedactiveFlagshas never been observed on the wire on any version — the live capture confirmed only"activeFlags":[]before the account hit its usage limit — so the steer-into-approval consequence remains inferred, not observed.expected = ""collides with the Claude arm's representation of an empty composer, so it provesidleonce and reports unproven thereafter.DQ3's supervisor-following half is unproven. Only stale-state behavior was measured.CodexHoldReasonmapping is lossy, not residue-free: four of six rows reportunknown, two of which Codex positively reported as active.Ready for review.
Related
DQ3requires that ordering.observe_thread_statustakes only the status word, so a requiredThreadStatusfield is parsed and discarded at all three call sites #265,exitedReviewModeis never matched, soHeld{Review}clears only when the thread reports idle #266 — shipped-code defects found while measuring. Independent of the proposal.