Skip to content

eval: cover pty attach outcomes and roles - #69

Open
schickling-assistant wants to merge 8 commits into
compoundingtech:mainfrom
schickling-assistant:schickling-assistant/2026-07-31-pty-attach-corrections
Open

eval: cover pty attach outcomes and roles#69
schickling-assistant wants to merge 8 commits into
compoundingtech:mainfrom
schickling-assistant:schickling-assistant/2026-07-31-pty-attach-corrections

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Why

The machine attach stream needs to distinguish intentional local detach from transport truncation, and one socket must be able to replace its read-only or writable role without reconnecting. Source-level unit tests do not prove either behavior through the shipped launcher and live daemon protocol boundary.

What

Adds one model-free pty-attach-outcomes-and-roles cell covering:

  • one terminal empty DETACH, clean EOF, and status zero after Ctrl+\;
  • administrative session loss as EOF without EXIT or DETACH, with an integer nonzero attach exit code and no terminating signal;
  • same-socket PEEK -> ATTACH restoring DATA, RESIZE, and min-grid participation;
  • same-socket ATTACH -> PEEK suppressing DATA and RESIZE and removing its geometry constraint;
  • cleanup commands succeeding, an empty session catalog, and captured eval-owned daemon and child PIDs being dead before root removal.

The eval VRS spec traces the cell to existing R01, R04, R05, R07, and R11. No requirements text changes are needed because those requirements already define executable, mutation-valid, hermetic eval contracts.

Oracle design

Negative assertions use protocol barriers rather than timing guesses. Synthetic mutations prove that the truncation oracle rejects both a successful status and code: null with a non-null signal. Cleanup mutations prove that an empty catalog cannot conceal a live captured daemon or child PID.

Merged-artifact receipt

  • Exact upstream PTY main: 9eb958c5aae026d5c05690ab72b528662c55708d
  • This includes merged #128, #145, #146, and #147; there is no validation-only runtime dependency
  • nix build github:compoundingtech/pty/9eb958c5aae026d5c05690ab72b528662c55708d#pty --no-link --print-out-paths: /nix/store/qii7qblg90qnwch6jq2pp5rbyrpdra66-pty-0.1.0; executable SHA-256 8c7f4d8076f03ff68756d76b0f89613c0de2e6bf9e89e08798361b4a18660d4c
  • That exact packaged PTY through st2 eval: 7 PASS / 0 FAIL / 7 gating judges
  • Deterministic oracle mutation self-test: passed
  • JavaScript syntax, st2 semantic load, model policy, harness exclusions, corpus inventory, catalog freshness, scoped PII, VRS trace, and diff checks: passed
  • vision.md and requirements.md: unchanged
  • Historical PTY base d5fabc3: expected red, 1 PASS / 6 FAIL

Publication metadata

Source files and commit messages are sanitized. The managed GitHub wrapper injects the remaining PR footer and re-adds it after edits; removing that infrastructure-owned metadata requires a human or tooling-owner action.

Posted on behalf of @schickling
field value
agent_name co1-sage
agent_session_id ed878dac-3735-4276-b3e0-ea1b1cd65291
agent_tool Codex CLI
agent_tool_version 0.145.0
agent_runtime Codex CLI 0.145.0
agent_model unknown
runtime_profile /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
worktree evals/schickling-assistant/2026-07-31-pty-attach-corrections
machine dev3
tooling_profile dotfiles@0fb7e03

myobie commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Review gap at exact head 9e7d07a: validateTruncation() uses assert.notEqual(result.code, 0), but Node reports code === null when the child terminates by signal, so a signal-killed attach currently passes the claimed nonzero-exit oracle. Please require an integer nonzero exit code and signal === null (or explicitly narrow the written contract if signal termination is acceptable), and add the missing code: null / non-null signal mutation.

@schickling-assistant

Copy link
Copy Markdown
Contributor Author

Addressed at f18e653: the truncation judge now requires an integer nonzero exit code and signal === null, and the mutation matrix includes code: null with SIGKILL.

The deterministic self-test passes, and the packaged current PTY composition at 4dd713e still passes all 7/7 gating judges.

Posted on behalf of @schickling
field value
agent_name co1-sage
agent_session_id ed878dac-3735-4276-b3e0-ea1b1cd65291
agent_tool Codex CLI
agent_tool_version 0.145.0
agent_runtime Codex CLI 0.145.0
agent_model unknown
runtime_profile /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
worktree evals/schickling-assistant/2026-07-31-pty-attach-corrections
machine dev3
tooling_profile dotfiles@0fb7e03

@schickling-assistant
schickling-assistant marked this pull request as ready for review July 31, 2026 19:53
@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-07-31-pty-attach-corrections branch from cb8d064 to cb7e3cd Compare July 31, 2026 20:15
@schickling-assistant

Copy link
Copy Markdown
Contributor Author

Final review receipt at exact head cb7e3cd80cc1c7bc0727993eb59a942edc9429de:

  • merged PTY main 7effe7b8c4ee22796f62188d321445027c8f5096: 7/7 executable gates
  • your truncation feedback is enforced: integer nonzero code, signal === null, and signal-only mutation
  • cleanup now checks kill/remove results, captures daemon/child identities, proves both PIDs dead before root removal, and rejects empty-catalog/live-process mutations
  • mutation, static, VRS, catalog, and independent rereview: green/clear
  • all commit bodies are sanitized; only the managed GitHub wrapper's infrastructure-owned PR footer remains
  • ready, current, CLEAN, no review threads

This is ready for your review. No canonical merge was performed by the author lane.

Posted on behalf of @schickling
field value
agent_name co1-sage
agent_session_id ed878dac-3735-4276-b3e0-ea1b1cd65291
agent_tool Codex CLI
agent_tool_version 0.145.0
agent_runtime Codex CLI 0.145.0
agent_model unknown
runtime_profile /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
worktree dotfiles/schickling/2026-07-20-cos-misc
machine dev3
tooling_profile dotfiles@0fb7e03

@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-07-31-pty-attach-corrections branch from 666eb27 to 7aef55c Compare July 31, 2026 22:41
@myobie

myobie commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Nathan authorized merge if the eval remains sound and green. I reviewed current head 7aef55c: the attach outcome/role contract, truncation oracle, signal-only mutation, and cleanup proof look good, and the recorded exact PTY run is 7/7. GitHub now reports a conflict with current main, so I cannot merge this head. Please merge current main into the branch with a normal merge commit, regenerate CATALOG.md, and rerun the static gates and focused 7/7 cell. If the semantic cell remains unchanged and those gates are green, it is approved for merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants