From 100ad68e3af921642a13d74de2a1f48d4f50cc37 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 11:26:08 +0200 Subject: [PATCH 01/17] test(agent-spec): prove declared event streams --- AGENT-SPEC.md | 58 ++++++++- CATALOG.md | 6 +- README.md | 4 +- bin/check-corpus.sh | 8 +- cells/agent-spec-streams/README.md | 9 ++ .../agent-spec-streams/agent-spec-streams.kdl | 30 +++++ cells/agent-spec-streams/fixture/adapter.sh | 13 ++ cells/agent-spec-streams/fixture/contract.sh | 116 ++++++++++++++++++ .../net/agents/stream/worker/agent.kdl | 7 ++ evidence/harness-exclusions.tsv | 1 + 10 files changed, 241 insertions(+), 11 deletions(-) create mode 100644 cells/agent-spec-streams/README.md create mode 100644 cells/agent-spec-streams/agent-spec-streams.kdl create mode 100755 cells/agent-spec-streams/fixture/adapter.sh create mode 100755 cells/agent-spec-streams/fixture/contract.sh create mode 100644 cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index 1d4886c..c97cf5f 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`0fed14bb5653b67e1d64f1199e240c4c5c612bf7`](https://github.com/compoundingtech/st2/commit/0fed14bb5653b67e1d64f1199e240c4c5c612bf7) -(`0.1.0`, source `0fed14b`). The pin identifies the implementation and version the corpus currently proves; it +[`002859de3598d471b3cdbba401fa1502d797bb64`](https://github.com/compoundingtech/st2/commit/002859de3598d471b3cdbba401fa1502d797bb64) +(`0.1.0+002859d`, source `002859d`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -64,6 +64,10 @@ agent "" { resource "work" uri="github-issue://example/project/123" + stream "webhook" {} + stream "ci" { command #"exec ci-adapter --emit-to "$ST_AGENT""# } + stream "deploy" { argv "deploy-adapter" "--json" } + restart { attempts 3 interval "60s" @@ -102,6 +106,9 @@ Supported agent children are: | `retired #true` | Legacy read-compatible spelling of retirement. New lifecycle transitions use `desired-state` with a rationale. | | `keep #true` | Freeze dead evidence and suppress collection/restart for every task; retirement still stops live tasks. | | `resource "name" uri="absolute-uri"` | Binds one uniquely named, externally identified Resource as declaration metadata. | +| `stream "name" {}` | Declares a named external event ingress endpoint. | +| `stream "name" { command "..." }` | Declares an ingress endpoint and a runner-supervised shell adapter. | +| `stream "name" { argv "program" "arg"... }` | Declares an ingress endpoint and a runner-supervised direct-argv adapter. | | `restart { … }` | Optional service restart policy. | | `env { KEY "value" }` | Environment inherited by the compact agent task and sidecars. | | `command "…"` | Compact interactive task named `agent`. | @@ -124,6 +131,47 @@ the window reset; `mode "fail"` parks the task after attempts are exhausted and crash-loop message to `supervisor`. Invalid restart subfields currently fall back to defaults; authors must not rely on that permissiveness. + + +## Event streams + +An agent may declare zero or more named streams. A stream name is 1-40 ASCII characters, contains only +lowercase letters, digits, and hyphens, and neither begins nor ends with a hyphen. Names are unique within one +agent. The stream envelope is closed: the node has exactly one +positional name, no properties, and zero or one launch child. A launch is either exactly one positional +`command` string or a non-empty positional-string `argv`; declaring both, extra arguments or properties, +typed nodes, child blocks, unknown children, and `every` all fail validation. Scheduling remains the separate, +reserved `schedule` contract; streams are long-running event sources. + +A launch-bearing stream lowers directly to a derived, terminal-free exec task named `stream-` with +runtime id `..stream-`. `command` runs through `sh -c`; `argv` executes element zero +directly with the remaining elements unchanged. The derived task inherits agent environment, cwd, restart +policy, suspension, resumption, retirement, and cleanup. An explicit task named `stream-` is therefore a +collision and fails validation. A stream adapter is a sibling companion: it does not make an otherwise +unrunnable agent runnable, and its failure does not relaunch a healthy owning agent. + +A commandless stream declares external ingress and creates no task. Neither form is a delivery transport: +streams publish ordinary durable inbox records but do not wake an agent unless it separately declares a +delivery transport such as `ding`. Producers publish with a stable identity: + +```sh +st2 event emit . \ + --stream --event-id \ + [--key ] [--supersede] [--subject ] \ + --message --host --json +``` + +The recipient must be active and must declare the stream. A first identity/content pair creates one canonical +inbox record; an identical replay returns the same filename as deduplicated; reuse with different content +fails closed. `--supersede` archives the unread predecessor for the same key, or the stream-wide predecessor +without a key, before publishing its successor. Per-stream state is durable and bounded to the latest 128 +identities; an identity older than that honest horizon is new again. Event ids, keys, stream names, and subjects +reject control characters before any write. + +Executable evidence: [`agent-spec-streams`](cells/agent-spec-streams/) proves strict declaration failures, +direct lowering and runtime identity, command/argv execution, external ingress without a task, stable +deduplication and conflicting-reuse refusal, and suspend/resume coupling through the real st2 CLI and runtime. + ## Resource bindings @@ -542,7 +590,11 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -For source `0fed14b`, the accepted published Linux executable has SHA256 +The latest independently pinned candidate executable for source `002859d` has SHA256 +`5995aafcd5a7f4e8a25ae557386068c7d6978123fe69c448a90eeb3458092178`. It is built from the exact source +above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. + +For the older source `0fed14b`, the accepted published Linux executable has SHA256 `d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed`; its published archive has SHA256 `d14404ae678bbe3f2a5ad8580cde1e4b8f6009067c46555f392c6e0957b8a2da`, and the downloaded `SHA256SUMS` asset has SHA256 `50cfd8722e58d1c74fdc543f3e3bb3bac768decd04575fde2360ea838ec5e9d3`. The immutable diff --git a/CATALOG.md b/CATALOG.md index e37255e..571c302 100644 --- a/CATALOG.md +++ b/CATALOG.md @@ -1,7 +1,7 @@ # Current eval catalog and evidence -This is the sole current corpus surface: **60 included cells** (20 model-free, +This is the sole current corpus surface: **61 included cells** (21 model-free, 40 model-backed) and **4 retired exclusions**. Inventory fields are derived from the active KDL and executable judge scripts. Latest accepted PASS evidence remains distinct from the append-only run history: a failed last run is visible without being advertised as accepted evidence, and cells with no @@ -18,6 +18,7 @@ harness-native loader plus canonical hook file. |---|---|---|---:|---|---|---:|---|---| | `adopt-only-migration` | model-free | — | 0 | none | `90s` | 6 | — | **NO STRUCTURED RUN** | | `agent-spec-resource-bindings` | model-free | — | 0 | none | `60s` | 4 | — | **NO STRUCTURED RUN** | +| `agent-spec-streams` | model-free | — | 0 | none | `90s` | 6 | — | **NO STRUCTURED RUN** | | `assignment-contract-cold-assignment` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | | `assignment-contract-cold-focus` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | | `assignment-contract-cold-resources` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | @@ -98,6 +99,7 @@ while the last-run column makes a recorded failure distinct from a cell with no - `adopt-only-migration` / `cell` — deterministic native adoption/replacement lifecycle probe; no harness seat - `agent-spec-resource-bindings` / `cell` — deterministic native Agent Spec Resource-envelope probe; no harness seat +- `agent-spec-streams` / `cell` — deterministic native Agent Spec stream lifecycle and ingress probe; no harness seat - `context-resource-continuity` / `cell` — deterministic native context/resource restart probe; no harness seat - `docs` / `judge:cold-reader` — one-shot offline Claude print grader; no bus identity, DING, or hook surface - `exec-lifecycle-logging` / `cell` — deterministic native exec lifecycle probe; no harness seat @@ -118,7 +120,7 @@ while the last-run column makes a recorded failure distinct from a cell with no - `targeted-reconcile-isolation` / `cell` — deterministic selected reconcile isolation probe; no bus-connected agent - `two-networks-coexist` / `cell` — deterministic catalog-isolation probe; no harness seat -These **21** rows are generated from structured exclusions. A cell-level row is required for +These **22** rows are generated from structured exclusions. A cell-level row is required for every derived model-free cell. The one-shot docs cold reader is a model-backed grader, but not a bus-connected agent: it has no st2 identity, DING sidecar, or lifecycle hooks. diff --git a/README.md b/README.md index 80fbbae..31a7f46 100644 --- a/README.md +++ b/README.md @@ -96,8 +96,8 @@ SCORE: N PASS / 0 FAIL / K gating judges VERDICT: PASS ``` -Requirements are `st2 0.1.0` from source -[`0fed14b`](https://github.com/compoundingtech/st2/commit/0fed14bb5653b67e1d64f1199e240c4c5c612bf7), +Requirements are `st2 0.1.0+002859d` from source +[`002859d`](https://github.com/compoundingtech/st2/commit/002859de3598d471b3cdbba401fa1502d797bb64), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index b3a66ac..bc28769 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="0fed14b" -expected_binary_sha256="d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" -expected_version_regex='^st2 0\.1\.0 — running from local source \(0fed14b, .+ ago\)$' +expected_source="002859d" +expected_binary_sha256="5995aafcd5a7f4e8a25ae557386068c7d6978123fe69c448a90eeb3458092178" +expected_version_regex='^st2 0\.1\.0\+002859d — committed .+$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { @@ -19,7 +19,7 @@ actual_binary_sha256="$(sha256sum "$st2_path" | awk '{ print $1 }')" echo "FAIL: expected st2 binary sha256 $expected_binary_sha256, found $actual_binary_sha256 at $st2_path" >&2 exit 1 } -echo "PASS: pinned published runner source $expected_source ($actual_version; sha256 $actual_binary_sha256)" +echo "PASS: pinned runner source $expected_source ($actual_version; sha256 $actual_binary_sha256)" mapfile -d '' shell_files < <( find bin cells -type f -name '*.sh' -not -path '*/_git/*' -print0 | sort -z diff --git a/cells/agent-spec-streams/README.md b/cells/agent-spec-streams/README.md new file mode 100644 index 0000000..6d41d4f --- /dev/null +++ b/cells/agent-spec-streams/README.md @@ -0,0 +1,9 @@ +# agent-spec-streams + +Model-free E2E acceptance for the canonical Agent Spec stream contract introduced by +[`compoundingtech/st2#288`](https://github.com/compoundingtech/st2/pull/288). + +The cell crosses the public parser, reconciliation, process, event-ingress, authoring, inbox, and task +inventory boundaries. Its negative controls prove the contract is fail-closed for invalid names, unsupported +intervals, ambiguous launch shapes, task collisions, undeclared streams, suspended recipients, and conflicting +event identity reuse. No model seat or paid provider is started. diff --git a/cells/agent-spec-streams/agent-spec-streams.kdl b/cells/agent-spec-streams/agent-spec-streams.kdl new file mode 100644 index 0000000..70d5089 --- /dev/null +++ b/cells/agent-spec-streams/agent-spec-streams.kdl @@ -0,0 +1,30 @@ +// Native Agent Spec stream declaration, lifecycle, adapter, and ingress acceptance. +eval { + copy "./fixture" + max-timeout "90s" + + run "contract" { + command "bash ./contract.sh" + } + + judges { + judge "STRICT SHAPE - names, launch envelope, interval, and task collisions fail closed" { + exec "grep -Fqx STREAM-STRICT-SHAPE-GREEN-83a7 $RUNS_DIR/contract.out" + } + judge "LOWERING - shell and argv adapters have exact derived exec task identities; external ingress has none" { + exec "grep -Fqx STREAM-LOWERING-GREEN-83a7 $RUNS_DIR/contract.out" + } + judge "ADAPTERS - both supervised launch forms publish through the public event CLI" { + exec "grep -Fqx STREAM-ADAPTERS-GREEN-83a7 $RUNS_DIR/contract.out" + } + judge "INGRESS - external events deduplicate stably and conflicting identity reuse fails closed" { + exec "grep -Fqx STREAM-INGRESS-GREEN-83a7 $RUNS_DIR/contract.out" + } + judge "LIFECYCLE - suspend removes owner plus adapters and refuses ingress; resume relaunches them" { + exec "grep -Fqx STREAM-LIFECYCLE-GREEN-83a7 $RUNS_DIR/contract.out" + } + judge "CLEANUP - no eval-owned PTY or exec task remains" { + exec "grep -Fqx STREAM-CLEANUP-GREEN-83a7 $RUNS_DIR/contract.out" + } + } +} diff --git a/cells/agent-spec-streams/fixture/adapter.sh b/cells/agent-spec-streams/fixture/adapter.sh new file mode 100755 index 0000000..02ee3d6 --- /dev/null +++ b/cells/agent-spec-streams/fixture/adapter.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +set -euo pipefail + +stream="${1:?stream name required}" +marker="${CATALOG:?CATALOG required}/../adapter-$stream.json" +st2 event emit "${ST_AGENT:?ST_AGENT required}" \ + --stream "$stream" \ + --event-id "$stream-delivery-1" \ + --subject "$stream adapter" \ + --message "payload from $stream" \ + --host stream \ + --json >"$marker" +exec sleep 300 diff --git a/cells/agent-spec-streams/fixture/contract.sh b/cells/agent-spec-streams/fixture/contract.sh new file mode 100755 index 0000000..dd30d59 --- /dev/null +++ b/cells/agent-spec-streams/fixture/contract.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +set -euo pipefail + +root="${CATALOG:?CATALOG must be set}" +net="$root/net" +spec="$net/agents/stream/worker/agent.kdl" +original="$root/agent-spec-streams.original" +export CATALOG="$net" +export ST_ROOT="$net" +export PTY_ROOT="$net/pty" +export XDG_STATE_HOME="$root/state" +cp "$spec" "$original" + +cleanup() { + cp "$original" "$spec" 2>/dev/null || true + st2 down --catalog "$net" --host stream >/dev/null 2>&1 || true + PTY_ROOT="$PTY_ROOT" pty rm stream.worker >/dev/null 2>&1 || true +} +trap cleanup EXIT + +check_invalid() { + local name="$1" + local declaration="$2" + local expected="$3" + local catalog="$root/invalid-$name" + mkdir -p "$catalog/agents/stream/worker" + printf '%s\n' "$declaration" >"$catalog/agents/stream/worker/agent.kdl" + set +e + st2 validate --catalog "$catalog" --host stream --strict >"$root/invalid-$name.out" 2>&1 + local status="$?" + set -e + test "$status" -ne 0 + grep -Fq "$expected" "$root/invalid-$name.out" +} + +check_invalid uppercase \ + 'agent "worker" { host "stream"; command "true"; stream "Bad" {} }' \ + 'must match' +check_invalid too-long \ + 'agent "worker" { host "stream"; command "true"; stream "abcdefghijklmnopqrstuvwxyzabcdefghijklmno" {} }' \ + 'must be 1..=40 characters' +check_invalid two-launches \ + 'agent "worker" { host "stream"; command "true"; stream "ci" { command "a"; argv "b" } }' \ + 'at most one of `command` or `argv`' +check_invalid interval \ + 'agent "worker" { host "stream"; command "true"; stream "ci" { every "1m" } }' \ + 'stream `every` is reserved' +check_invalid collision \ + 'agent "worker" { host "stream"; command "true"; stream "ci" {}; exec "stream-ci" { command "true" } }' \ + 'declares both `stream "ci"` and a task named `stream-ci`' +echo "STREAM-STRICT-SHAPE-GREEN-83a7" + +st2 validate --catalog "$net" --host stream --strict >/dev/null +st2 tasks --catalog "$net" --host stream --json >"$root/tasks-before.json" +jq -e ' + [.tasks[] | select(.task | startswith("stream-")) | {task, runtimeId, kind}] == [ + {"task":"stream-argv","runtimeId":"stream.worker.stream-argv","kind":"exec"}, + {"task":"stream-shell","runtimeId":"stream.worker.stream-shell","kind":"exec"} + ] and + ([.tasks[].task] | index("stream-external")) == null +' "$root/tasks-before.json" >/dev/null +echo "STREAM-LOWERING-GREEN-83a7" + +st2 up --once --catalog "$net" --host stream >"$root/up.out" +grep -Fq 'stream.worker.stream-argv' "$root/up.out" +grep -Fq 'stream.worker.stream-shell' "$root/up.out" +for marker in "$root/adapter-argv.json" "$root/adapter-shell.json"; do + for _ in $(seq 1 100); do + test -s "$marker" && break + sleep 0.05 + done + jq -e '.status == "created" and .recipient == "stream.worker"' "$marker" >/dev/null +done +echo "STREAM-ADAPTERS-GREEN-83a7" + +first="$(st2 event emit stream.worker --stream external --event-id ext-1 --subject external --message payload --host stream --json)" +replay="$(st2 event emit stream.worker --stream external --event-id ext-1 --subject external --message payload --host stream --json)" +jq -e '.status == "created"' <<<"$first" >/dev/null +jq -e '.status == "deduplicated"' <<<"$replay" >/dev/null +test "$(jq -r .filename <<<"$first")" = "$(jq -r .filename <<<"$replay")" +set +e +st2 event emit stream.worker --stream external --event-id ext-1 --message changed --host stream --json >"$root/conflict.out" 2>&1 +conflict_status="$?" +st2 event emit stream.worker --stream missing --event-id missing-1 --message payload --host stream --json >"$root/undeclared.out" 2>&1 +undeclared_status="$?" +set -e +test "$conflict_status" -ne 0 +test "$undeclared_status" -ne 0 +grep -Fq 'reused with different content' "$root/conflict.out" +grep -Fq "does not declare stream 'missing'" "$root/undeclared.out" +echo "STREAM-INGRESS-GREEN-83a7" + +st2 agent desired-state stream.worker suspended --reason "Acceptance hold" --host stream --json >"$root/suspend.json" +st2 up --once --catalog "$net" --host stream >"$root/suspended-up.out" +st2 tasks --catalog "$net" --host stream --json >"$root/tasks-suspended.json" +jq -e '.tasks | all(.desiredState == "absent" and .runtime.state != "running")' "$root/tasks-suspended.json" >/dev/null +set +e +st2 event emit stream.worker --stream external --event-id held-1 --message payload --host stream --json >"$root/suspended-event.out" 2>&1 +suspended_status="$?" +set -e +test "$suspended_status" -ne 0 +grep -Fq 'eyes are closed' "$root/suspended-event.out" +st2 agent desired-state stream.worker running --host stream --json >"$root/resume.json" +st2 up --once --catalog "$net" --host stream >"$root/resumed-up.out" +grep -Fq 'stream.worker.stream-argv' "$root/resumed-up.out" +grep -Fq 'stream.worker.stream-shell' "$root/resumed-up.out" +echo "STREAM-LIFECYCLE-GREEN-83a7" + +st2 down --catalog "$net" --host stream >/dev/null +PTY_ROOT="$PTY_ROOT" pty rm stream.worker >/dev/null 2>&1 || true +cp "$original" "$spec" +trap - EXIT +test "$(PTY_ROOT="$PTY_ROOT" pty list --json | jq 'length')" -eq 0 +st2 tasks --catalog "$net" --host stream --json >"$root/tasks-clean.json" +jq -e '.tasks | all(.runtime.state != "running")' "$root/tasks-clean.json" >/dev/null +echo "STREAM-CLEANUP-GREEN-83a7" diff --git a/cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl b/cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl new file mode 100644 index 0000000..e082acb --- /dev/null +++ b/cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl @@ -0,0 +1,7 @@ +agent "worker" { + host "stream" + command "exec sleep 300" + stream "external" {} + stream "shell" { command #"exec bash "$CATALOG/../adapter.sh" shell"# } + stream "argv" { argv "bash" "$CATALOG/../adapter.sh" "argv" } +} diff --git a/evidence/harness-exclusions.tsv b/evidence/harness-exclusions.tsv index 0384505..6cd005c 100644 --- a/evidence/harness-exclusions.tsv +++ b/evidence/harness-exclusions.tsv @@ -1,6 +1,7 @@ cell surface reason adopt-only-migration cell deterministic native adoption/replacement lifecycle probe; no harness seat agent-spec-resource-bindings cell deterministic native Agent Spec Resource-envelope probe; no harness seat +agent-spec-streams cell deterministic native Agent Spec stream lifecycle and ingress probe; no harness seat context-resource-continuity cell deterministic native context/resource restart probe; no harness seat docs judge:cold-reader one-shot offline Claude print grader; no bus identity, DING, or hook surface exec-lifecycle-logging cell deterministic native exec lifecycle probe; no harness seat From 6e2e86e8cb590dd5b68a71ab0f00db17de866b3d Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 11:37:38 +0200 Subject: [PATCH 02/17] test(agent-spec): prove no-follow stream ingress --- AGENT-SPEC.md | 12 +++-- CATALOG.md | 2 +- README.md | 4 +- bin/check-corpus.sh | 6 +-- cells/agent-spec-streams/README.md | 3 +- .../agent-spec-streams/agent-spec-streams.kdl | 3 ++ cells/agent-spec-streams/fixture/contract.sh | 51 +++++++++++++++++++ 7 files changed, 70 insertions(+), 11 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index c97cf5f..6cd5abb 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`002859de3598d471b3cdbba401fa1502d797bb64`](https://github.com/compoundingtech/st2/commit/002859de3598d471b3cdbba401fa1502d797bb64) -(`0.1.0+002859d`, source `002859d`). The pin identifies the implementation and version the corpus currently proves; it +[`9be176f5ce06443296e38bc53760b64fd48ee3d5`](https://github.com/compoundingtech/st2/commit/9be176f5ce06443296e38bc53760b64fd48ee3d5) +(`0.1.0+9be176f`, source `9be176f`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -168,6 +168,10 @@ without a key, before publishing its successor. Per-stream state is durable and identities; an identity older than that honest horizon is new again. Event ids, keys, stream names, and subjects reject control characters before any write. +Ingress resolves the recipient through strict catalog discovery and retains no-follow capabilities for stream +state, inbox, and archive operations. An unobservable declaration entry or a symlinked state/inbox ancestor +therefore fails closed and cannot redirect event bytes outside the owning agent directory. + Executable evidence: [`agent-spec-streams`](cells/agent-spec-streams/) proves strict declaration failures, direct lowering and runtime identity, command/argv execution, external ingress without a task, stable deduplication and conflicting-reuse refusal, and suspend/resume coupling through the real st2 CLI and runtime. @@ -590,8 +594,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned candidate executable for source `002859d` has SHA256 -`5995aafcd5a7f4e8a25ae557386068c7d6978123fe69c448a90eeb3458092178`. It is built from the exact source +The latest independently pinned candidate executable for source `9be176f` has SHA256 +`e575e057d018be2b864eeb1aacd3f1d67332a7a2fbfcadccab1499b4511e5545`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/CATALOG.md b/CATALOG.md index 571c302..82b47c0 100644 --- a/CATALOG.md +++ b/CATALOG.md @@ -18,7 +18,7 @@ harness-native loader plus canonical hook file. |---|---|---|---:|---|---|---:|---|---| | `adopt-only-migration` | model-free | — | 0 | none | `90s` | 6 | — | **NO STRUCTURED RUN** | | `agent-spec-resource-bindings` | model-free | — | 0 | none | `60s` | 4 | — | **NO STRUCTURED RUN** | -| `agent-spec-streams` | model-free | — | 0 | none | `90s` | 6 | — | **NO STRUCTURED RUN** | +| `agent-spec-streams` | model-free | — | 0 | none | `90s` | 7 | — | **NO STRUCTURED RUN** | | `assignment-contract-cold-assignment` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | | `assignment-contract-cold-focus` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | | `assignment-contract-cold-resources` | Codex | `gpt-5.6-sol` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | diff --git a/README.md b/README.md index 31a7f46..663b5ba 100644 --- a/README.md +++ b/README.md @@ -96,8 +96,8 @@ SCORE: N PASS / 0 FAIL / K gating judges VERDICT: PASS ``` -Requirements are `st2 0.1.0+002859d` from source -[`002859d`](https://github.com/compoundingtech/st2/commit/002859de3598d471b3cdbba401fa1502d797bb64), +Requirements are `st2 0.1.0+9be176f` from source +[`9be176f`](https://github.com/compoundingtech/st2/commit/9be176f5ce06443296e38bc53760b64fd48ee3d5), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index bc28769..b653c9f 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="002859d" -expected_binary_sha256="5995aafcd5a7f4e8a25ae557386068c7d6978123fe69c448a90eeb3458092178" -expected_version_regex='^st2 0\.1\.0\+002859d — committed .+$' +expected_source="9be176f" +expected_binary_sha256="e575e057d018be2b864eeb1aacd3f1d67332a7a2fbfcadccab1499b4511e5545" +expected_version_regex='^st2 0\.1\.0\+9be176f — committed .+$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/agent-spec-streams/README.md b/cells/agent-spec-streams/README.md index 6d41d4f..c9648da 100644 --- a/cells/agent-spec-streams/README.md +++ b/cells/agent-spec-streams/README.md @@ -6,4 +6,5 @@ Model-free E2E acceptance for the canonical Agent Spec stream contract introduce The cell crosses the public parser, reconciliation, process, event-ingress, authoring, inbox, and task inventory boundaries. Its negative controls prove the contract is fail-closed for invalid names, unsupported intervals, ambiguous launch shapes, task collisions, undeclared streams, suspended recipients, and conflicting -event identity reuse. No model seat or paid provider is started. +event identity reuse. It also proves that strict discovery and retained no-follow state/inbox capabilities +prevent symlink redirection outside the agent directory. No model seat or paid provider is started. diff --git a/cells/agent-spec-streams/agent-spec-streams.kdl b/cells/agent-spec-streams/agent-spec-streams.kdl index 70d5089..34c4589 100644 --- a/cells/agent-spec-streams/agent-spec-streams.kdl +++ b/cells/agent-spec-streams/agent-spec-streams.kdl @@ -20,6 +20,9 @@ eval { judge "INGRESS - external events deduplicate stably and conflicting identity reuse fails closed" { exec "grep -Fqx STREAM-INGRESS-GREEN-83a7 $RUNS_DIR/contract.out" } + judge "CAPABILITIES - strict discovery and no-follow state/inbox operations prevent redirected writes" { + exec "grep -Fqx STREAM-CAPABILITIES-GREEN-83a7 $RUNS_DIR/contract.out" + } judge "LIFECYCLE - suspend removes owner plus adapters and refuses ingress; resume relaunches them" { exec "grep -Fqx STREAM-LIFECYCLE-GREEN-83a7 $RUNS_DIR/contract.out" } diff --git a/cells/agent-spec-streams/fixture/contract.sh b/cells/agent-spec-streams/fixture/contract.sh index dd30d59..4a6aa2a 100755 --- a/cells/agent-spec-streams/fixture/contract.sh +++ b/cells/agent-spec-streams/fixture/contract.sh @@ -48,6 +48,12 @@ check_invalid interval \ check_invalid collision \ 'agent "worker" { host "stream"; command "true"; stream "ci" {}; exec "stream-ci" { command "true" } }' \ 'declares both `stream "ci"` and a task named `stream-ci`' +check_invalid property \ + 'agent "worker" { host "stream"; command "true"; stream "ci" bogus=#true {} }' \ + 'no properties' +check_invalid misspelled \ + 'agent "worker" { host "stream"; command "true"; stream "ci" { comand "watch" } }' \ + 'unsupported field `comand`' echo "STREAM-STRICT-SHAPE-GREEN-83a7" st2 validate --catalog "$net" --host stream --strict >/dev/null @@ -90,6 +96,51 @@ grep -Fq 'reused with different content' "$root/conflict.out" grep -Fq "does not declare stream 'missing'" "$root/undeclared.out" echo "STREAM-INGRESS-GREEN-83a7" +nofollow_case() { + local name="$1" + local ancestor="$2" + local catalog="$root/no-follow-$name" + local agent="$catalog/agents/stream/worker" + local outside="$root/outside-$name" + mkdir -p "$agent/resources" "$outside" + cp "$original" "$agent/agent.kdl" + ln -s "$outside" "$agent/resources/$ancestor" + set +e + st2 event emit stream.worker \ + --catalog "$catalog" \ + --stream external \ + --event-id "no-follow-$name" \ + --message payload \ + --host stream \ + --json >"$root/no-follow-$name.out" 2>&1 + local status="$?" + set -e + test "$status" -ne 0 + test -z "$(find "$outside" -mindepth 1 -print -quit)" +} + +nofollow_case state streams +nofollow_case inbox inbox + +strict="$root/strict-discovery" +mkdir -p "$strict/agents/stream/worker" +cp "$original" "$strict/agents/stream/worker/agent.kdl" +ln -s "$strict/missing-agent.kdl" "$strict/concealed-agent.kdl" +set +e +st2 event emit stream.worker \ + --catalog "$strict" \ + --stream external \ + --event-id strict-discovery \ + --message payload \ + --host stream \ + --json >"$root/strict-discovery.out" 2>&1 +strict_status="$?" +set -e +test "$strict_status" -ne 0 +grep -Fq 'unobservable declaration entry' "$root/strict-discovery.out" +test ! -e "$strict/agents/stream/worker/resources/inbox" +echo "STREAM-CAPABILITIES-GREEN-83a7" + st2 agent desired-state stream.worker suspended --reason "Acceptance hold" --host stream --json >"$root/suspend.json" st2 up --once --catalog "$net" --host stream >"$root/suspended-up.out" st2 tasks --catalog "$net" --host stream --json >"$root/tasks-suspended.json" From 7b323593788df90438091c6503036f7ae15962c8 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 11:55:39 +0200 Subject: [PATCH 03/17] test(agent-spec): prove supersession and temp safety --- AGENT-SPEC.md | 15 +++--- README.md | 4 +- bin/check-corpus.sh | 6 +-- cells/agent-spec-streams/README.md | 5 +- cells/agent-spec-streams/fixture/contract.sh | 50 ++++++++++++++++++++ 5 files changed, 66 insertions(+), 14 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index 6cd5abb..f10bbfc 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`9be176f5ce06443296e38bc53760b64fd48ee3d5`](https://github.com/compoundingtech/st2/commit/9be176f5ce06443296e38bc53760b64fd48ee3d5) -(`0.1.0+9be176f`, source `9be176f`). The pin identifies the implementation and version the corpus currently proves; it +[`2fa21f12212777ee46857d6eb3eb13db1b4ba64b`](https://github.com/compoundingtech/st2/commit/2fa21f12212777ee46857d6eb3eb13db1b4ba64b) +(`0.1.0+2fa21f1`, source `2fa21f1`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -168,9 +168,10 @@ without a key, before publishing its successor. Per-stream state is durable and identities; an identity older than that honest horizon is new again. Event ids, keys, stream names, and subjects reject control characters before any write. -Ingress resolves the recipient through strict catalog discovery and retains no-follow capabilities for stream -state, inbox, and archive operations. An unobservable declaration entry or a symlinked state/inbox ancestor -therefore fails closed and cannot redirect event bytes outside the owning agent directory. +Ingress and stream authoring resolve their targets through strict catalog discovery. Ingress retains no-follow +capabilities for stream state, its create-new temporary, inbox, and archive operations. An unobservable +declaration entry or a symlinked state/inbox path therefore fails closed and cannot redirect event bytes or +authored declarations outside the owning agent directory. Executable evidence: [`agent-spec-streams`](cells/agent-spec-streams/) proves strict declaration failures, direct lowering and runtime identity, command/argv execution, external ingress without a task, stable @@ -594,8 +595,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned candidate executable for source `9be176f` has SHA256 -`e575e057d018be2b864eeb1aacd3f1d67332a7a2fbfcadccab1499b4511e5545`. It is built from the exact source +The latest independently pinned candidate executable for source `2fa21f1` has SHA256 +`fda5b37c26bc4a2cd7a2fe545186efc96bb741f620395cb7ac8f0aa66e295f91`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index 663b5ba..f5f74e5 100644 --- a/README.md +++ b/README.md @@ -96,8 +96,8 @@ SCORE: N PASS / 0 FAIL / K gating judges VERDICT: PASS ``` -Requirements are `st2 0.1.0+9be176f` from source -[`9be176f`](https://github.com/compoundingtech/st2/commit/9be176f5ce06443296e38bc53760b64fd48ee3d5), +Requirements are `st2 0.1.0+2fa21f1` from source +[`2fa21f1`](https://github.com/compoundingtech/st2/commit/2fa21f12212777ee46857d6eb3eb13db1b4ba64b), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index b653c9f..fd056b7 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="9be176f" -expected_binary_sha256="e575e057d018be2b864eeb1aacd3f1d67332a7a2fbfcadccab1499b4511e5545" -expected_version_regex='^st2 0\.1\.0\+9be176f — committed .+$' +expected_source="2fa21f1" +expected_binary_sha256="fda5b37c26bc4a2cd7a2fe545186efc96bb741f620395cb7ac8f0aa66e295f91" +expected_version_regex='^st2 0\.1\.0\+2fa21f1 — committed .+$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/agent-spec-streams/README.md b/cells/agent-spec-streams/README.md index c9648da..d0f3294 100644 --- a/cells/agent-spec-streams/README.md +++ b/cells/agent-spec-streams/README.md @@ -6,5 +6,6 @@ Model-free E2E acceptance for the canonical Agent Spec stream contract introduce The cell crosses the public parser, reconciliation, process, event-ingress, authoring, inbox, and task inventory boundaries. Its negative controls prove the contract is fail-closed for invalid names, unsupported intervals, ambiguous launch shapes, task collisions, undeclared streams, suspended recipients, and conflicting -event identity reuse. It also proves that strict discovery and retained no-follow state/inbox capabilities -prevent symlink redirection outside the agent directory. No model seat or paid provider is started. +event identity reuse. It also proves keyed-head to keyless supersession, strict discovery for ingress and +authoring, and retained no-follow state/inbox/temp-file capabilities that prevent symlink redirection outside +the agent directory. No model seat or paid provider is started. diff --git a/cells/agent-spec-streams/fixture/contract.sh b/cells/agent-spec-streams/fixture/contract.sh index 4a6aa2a..e018f61 100755 --- a/cells/agent-spec-streams/fixture/contract.sh +++ b/cells/agent-spec-streams/fixture/contract.sh @@ -94,6 +94,13 @@ test "$conflict_status" -ne 0 test "$undeclared_status" -ne 0 grep -Fq 'reused with different content' "$root/conflict.out" grep -Fq "does not declare stream 'missing'" "$root/undeclared.out" + +keyed="$(st2 event emit stream.worker --stream external --event-id keyed-old --key pr-1 --supersede --message old --host stream --json)" +keyless="$(st2 event emit stream.worker --stream external --event-id keyless-new --supersede --message new --host stream --json)" +keyed_filename="$(jq -r .filename <<<"$keyed")" +test "$(jq -r .superseded <<<"$keyless")" = "$keyed_filename" +test ! -e "$net/agents/stream/worker/resources/inbox/$keyed_filename" +test -e "$net/agents/stream/worker/resources/archive/$keyed_filename" echo "STREAM-INGRESS-GREEN-83a7" nofollow_case() { @@ -122,6 +129,36 @@ nofollow_case() { nofollow_case state streams nofollow_case inbox inbox +temporary_catalog="$root/no-follow-temporary" +temporary_agent="$temporary_catalog/agents/stream/worker" +temporary_state="$temporary_agent/resources/streams/external" +temporary_victim="$root/temporary-victim" +mkdir -p "$temporary_state" +cp "$original" "$temporary_agent/agent.kdl" +printf '%s' 'must remain unchanged' >"$temporary_victim" +set +e +bash -c ' + victim="$1" + state="$2" + catalog="$3" + for counter in $(seq 0 4095); do + ln -s "$victim" "$state/.state.tmp-$$-$counter" + done + exec st2 event emit stream.worker \ + --catalog "$catalog" \ + --stream external \ + --event-id no-follow-temporary \ + --message payload \ + --host stream \ + --json +' _ "$temporary_victim" "$temporary_state" "$temporary_catalog" >"$root/no-follow-temporary.out" 2>&1 +temporary_status="$?" +set -e +test "$temporary_status" -ne 0 +grep -Fq 'fresh stream state temporary' "$root/no-follow-temporary.out" +test "$(cat "$temporary_victim")" = 'must remain unchanged' +test ! -e "$temporary_state/state.json" + strict="$root/strict-discovery" mkdir -p "$strict/agents/stream/worker" cp "$original" "$strict/agents/stream/worker/agent.kdl" @@ -139,6 +176,19 @@ set -e test "$strict_status" -ne 0 grep -Fq 'unobservable declaration entry' "$root/strict-discovery.out" test ! -e "$strict/agents/stream/worker/resources/inbox" + +strict_original="$(sha256sum "$strict/agents/stream/worker/agent.kdl")" +set +e +st2 stream add concealed-check \ + --catalog "$strict" \ + --agent stream.worker \ + --host stream \ + --json >"$root/strict-authoring.out" 2>&1 +authoring_status="$?" +set -e +test "$authoring_status" -ne 0 +grep -Fq 'unobservable declaration entry' "$root/strict-authoring.out" +test "$(sha256sum "$strict/agents/stream/worker/agent.kdl")" = "$strict_original" echo "STREAM-CAPABILITIES-GREEN-83a7" st2 agent desired-state stream.worker suspended --reason "Acceptance hold" --host stream --json >"$root/suspend.json" From fe05c6d2dfe3cc6c88c740fd5189da4f4848da17 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 12:04:03 +0200 Subject: [PATCH 04/17] test(agent-spec): repin rebased stream implementation --- AGENT-SPEC.md | 8 ++++---- README.md | 4 ++-- bin/check-corpus.sh | 6 +++--- cells/agent-spec-streams/README.md | 2 +- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index f10bbfc..c383cf9 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`2fa21f12212777ee46857d6eb3eb13db1b4ba64b`](https://github.com/compoundingtech/st2/commit/2fa21f12212777ee46857d6eb3eb13db1b4ba64b) -(`0.1.0+2fa21f1`, source `2fa21f1`). The pin identifies the implementation and version the corpus currently proves; it +[`29dec2f3c9dc0f9eb980f913e9c972c3f10785f1`](https://github.com/compoundingtech/st2/commit/29dec2f3c9dc0f9eb980f913e9c972c3f10785f1) +(`0.1.0+29dec2f`, source `29dec2f`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -595,8 +595,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned candidate executable for source `2fa21f1` has SHA256 -`fda5b37c26bc4a2cd7a2fe545186efc96bb741f620395cb7ac8f0aa66e295f91`. It is built from the exact source +The latest independently pinned candidate executable for source `29dec2f` has SHA256 +`af4bf721977f9cd11992ba96ba8b5a0afdec1f8619ec2a20ccad6558768d7b2a`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index f5f74e5..d588a95 100644 --- a/README.md +++ b/README.md @@ -96,8 +96,8 @@ SCORE: N PASS / 0 FAIL / K gating judges VERDICT: PASS ``` -Requirements are `st2 0.1.0+2fa21f1` from source -[`2fa21f1`](https://github.com/compoundingtech/st2/commit/2fa21f12212777ee46857d6eb3eb13db1b4ba64b), +Requirements are `st2 0.1.0+29dec2f` from source +[`29dec2f`](https://github.com/compoundingtech/st2/commit/29dec2f3c9dc0f9eb980f913e9c972c3f10785f1), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index fd056b7..8782f27 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="2fa21f1" -expected_binary_sha256="fda5b37c26bc4a2cd7a2fe545186efc96bb741f620395cb7ac8f0aa66e295f91" -expected_version_regex='^st2 0\.1\.0\+2fa21f1 — committed .+$' +expected_source="29dec2f" +expected_binary_sha256="af4bf721977f9cd11992ba96ba8b5a0afdec1f8619ec2a20ccad6558768d7b2a" +expected_version_regex='^st2 0\.1\.0\+29dec2f — committed .+$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/agent-spec-streams/README.md b/cells/agent-spec-streams/README.md index d0f3294..0648a52 100644 --- a/cells/agent-spec-streams/README.md +++ b/cells/agent-spec-streams/README.md @@ -1,7 +1,7 @@ # agent-spec-streams Model-free E2E acceptance for the canonical Agent Spec stream contract introduced by -[`compoundingtech/st2#288`](https://github.com/compoundingtech/st2/pull/288). +[`compoundingtech/st2#300`](https://github.com/compoundingtech/st2/pull/300). The cell crosses the public parser, reconciliation, process, event-ingress, authoring, inbox, and task inventory boundaries. Its negative controls prove the contract is fail-closed for invalid names, unsupported From ab65e64c5114e877815a2342f9d1760f05f539b8 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 12:51:08 +0200 Subject: [PATCH 05/17] test(stream): prove real CI and Nix waiters --- AGENT-SPEC.md | 13 ++- CATALOG.md | 8 +- README.md | 4 +- bin/check-corpus.sh | 6 +- cells/stream-github-ci-waiter/README.md | 20 ++++ .../fixture/contract.sh | 90 ++++++++++++++++ cells/stream-github-ci-waiter/fixture/fake-gh | 25 +++++ .../stream-github-ci-waiter/fixture/flaky-st2 | 12 +++ .../net/agents/stream/worker/agent.kdl | 5 + .../fixture/wait-gh-pr-ci.sh | 75 +++++++++++++ .../stream-github-ci-waiter.kdl | 27 +++++ cells/stream-nix-build-waiter/README.md | 22 ++++ .../fixture/contract.sh | 102 ++++++++++++++++++ .../fixture/inject-event-failures.sh | 33 ++++++ .../net/agents/stream/nix-watcher/agent.kdl | 6 ++ .../fixture/nix-build-waiter.sh | 87 +++++++++++++++ .../stream-nix-build-waiter.kdl | 24 +++++ evidence/harness-exclusions.tsv | 2 + 18 files changed, 550 insertions(+), 11 deletions(-) create mode 100644 cells/stream-github-ci-waiter/README.md create mode 100755 cells/stream-github-ci-waiter/fixture/contract.sh create mode 100755 cells/stream-github-ci-waiter/fixture/fake-gh create mode 100755 cells/stream-github-ci-waiter/fixture/flaky-st2 create mode 100644 cells/stream-github-ci-waiter/fixture/net/agents/stream/worker/agent.kdl create mode 100755 cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh create mode 100644 cells/stream-github-ci-waiter/stream-github-ci-waiter.kdl create mode 100644 cells/stream-nix-build-waiter/README.md create mode 100755 cells/stream-nix-build-waiter/fixture/contract.sh create mode 100755 cells/stream-nix-build-waiter/fixture/inject-event-failures.sh create mode 100644 cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl create mode 100755 cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh create mode 100644 cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index c383cf9..b4dff17 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`29dec2f3c9dc0f9eb980f913e9c972c3f10785f1`](https://github.com/compoundingtech/st2/commit/29dec2f3c9dc0f9eb980f913e9c972c3f10785f1) -(`0.1.0+29dec2f`, source `29dec2f`). The pin identifies the implementation and version the corpus currently proves; it +[`493bc5cb70c680650a7a49de0d3b33cccd5b6810`](https://github.com/compoundingtech/st2/commit/493bc5cb70c680650a7a49de0d3b33cccd5b6810) +(`0.1.0`, source `493bc5c`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -176,6 +176,11 @@ authored declarations outside the owning agent directory. Executable evidence: [`agent-spec-streams`](cells/agent-spec-streams/) proves strict declaration failures, direct lowering and runtime identity, command/argv execution, external ingress without a task, stable deduplication and conflicting-reuse refusal, and suspend/resume coupling through the real st2 CLI and runtime. +[`stream-nix-build-waiter`](cells/stream-nix-build-waiter/) proves supervised adapters against real successful +and failed Nix builds, transient publication retry, restart replay, and deduplication. The maintained offline +lane of [`stream-github-ci-waiter`](cells/stream-github-ci-waiter/) proves pending-to-terminal polling, keyed +supersession, publication retry, timeout refusal, and cleanup; its opt-in authenticated lane exercises the same +boundary against a real GitHub Actions transition without making network state a merge gate. @@ -595,8 +600,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned candidate executable for source `29dec2f` has SHA256 -`af4bf721977f9cd11992ba96ba8b5a0afdec1f8619ec2a20ccad6558768d7b2a`. It is built from the exact source +The latest independently pinned release-mode executable for source `493bc5c` has SHA256 +`25f35c420a305badc4e13eb62555a371eadd01b21d02ad448f3e4d57c4233dd4`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/CATALOG.md b/CATALOG.md index 82b47c0..a9ffb39 100644 --- a/CATALOG.md +++ b/CATALOG.md @@ -1,7 +1,7 @@ # Current eval catalog and evidence -This is the sole current corpus surface: **61 included cells** (21 model-free, +This is the sole current corpus surface: **63 included cells** (23 model-free, 40 model-backed) and **4 retired exclusions**. Inventory fields are derived from the active KDL and executable judge scripts. Latest accepted PASS evidence remains distinct from the append-only run history: a failed last run is visible without being advertised as accepted evidence, and cells with no @@ -64,6 +64,8 @@ harness-native loader plus canonical hook file. | `skill-inheritance` | Claude | `claude-sonnet-5` / medium | 1 | low | `420s` | 3 | — | **NO STRUCTURED RUN** | | `st2-doctor-structure` | model-free | — | 0 | none | `60s` | 4 | — | **NO STRUCTURED RUN** | | `st2-network` | model-free | — | 0 | none | `60s` | 2 | — | **NO STRUCTURED RUN** | +| `stream-github-ci-waiter` | model-free | — | 0 | none | `25m` | 5 | — | **NO STRUCTURED RUN** | +| `stream-nix-build-waiter` | model-free | — | 0 | none | `120s` | 4 | — | **NO STRUCTURED RUN** | | `strict-validation-json` | model-free | — | 0 | none | `60s` | 4 | — | **NO STRUCTURED RUN** | | `targeted-reconcile-isolation` | model-free | — | 0 | none | `60s` | 7 | — | **NO STRUCTURED RUN** | | `test-writing` | Claude | `claude-sonnet-5` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | @@ -116,11 +118,13 @@ while the last-run column makes a recorded failure distinct from a cell with no - `shared-workspace-render-ownership` / `cell` — deterministic native shared-render admission probe; no harness seat - `st2-doctor-structure` / `cell` — deterministic catalog-health probe; no harness seat - `st2-network` / `cell` — deterministic catalog/message round-trip probe; no harness seat +- `stream-github-ci-waiter` / `cell` — deterministic GitHub CI waiter controls with an opt-in authenticated read-only smoke; no harness seat +- `stream-nix-build-waiter` / `cell` — deterministic real Nix build waiter and retry probe; no harness seat - `strict-validation-json` / `cell` — deterministic native validation-diagnostics probe; no harness seat - `targeted-reconcile-isolation` / `cell` — deterministic selected reconcile isolation probe; no bus-connected agent - `two-networks-coexist` / `cell` — deterministic catalog-isolation probe; no harness seat -These **22** rows are generated from structured exclusions. A cell-level row is required for +These **24** rows are generated from structured exclusions. A cell-level row is required for every derived model-free cell. The one-shot docs cold reader is a model-backed grader, but not a bus-connected agent: it has no st2 identity, DING sidecar, or lifecycle hooks. diff --git a/README.md b/README.md index d588a95..19532d2 100644 --- a/README.md +++ b/README.md @@ -96,8 +96,8 @@ SCORE: N PASS / 0 FAIL / K gating judges VERDICT: PASS ``` -Requirements are `st2 0.1.0+29dec2f` from source -[`29dec2f`](https://github.com/compoundingtech/st2/commit/29dec2f3c9dc0f9eb980f913e9c972c3f10785f1), +Requirements are `st2 0.1.0` from source +[`493bc5c`](https://github.com/compoundingtech/st2/commit/493bc5cb70c680650a7a49de0d3b33cccd5b6810), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index 8782f27..e415708 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="29dec2f" -expected_binary_sha256="af4bf721977f9cd11992ba96ba8b5a0afdec1f8619ec2a20ccad6558768d7b2a" -expected_version_regex='^st2 0\.1\.0\+29dec2f — committed .+$' +expected_source="493bc5c" +expected_binary_sha256="25f35c420a305badc4e13eb62555a371eadd01b21d02ad448f3e4d57c4233dd4" +expected_version_regex='^st2 0\.1\.0 — running from local source \(493bc5c, .+ ago\)$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/stream-github-ci-waiter/README.md b/cells/stream-github-ci-waiter/README.md new file mode 100644 index 0000000..25d8d28 --- /dev/null +++ b/cells/stream-github-ci-waiter/README.md @@ -0,0 +1,20 @@ +# stream-github-ci-waiter + +Use-case E2E for waiting on GitHub PR checks and delivering the terminal result through an st2 stream. + +The maintained model-free lane deterministically exercises two pending polls followed by success, stable replay +deduplication, terminal failure superseding success for the same PR key, bounded timeout with no event, and +cleanup. It uses the same `gh` JSON boundary as the live lane. + +The authenticated smoke lane waits through a real pending-to-terminal Nix transition on the immutable merged +[`compoundingtech/st2#285`](https://github.com/compoundingtech/st2/pull/285), verifies its exact head, records +every observed check state, and emits the result through `st2 event emit`. Rerun the existing successful workflow +immediately before starting it. It is opt-in because the maintained corpus must remain runnable offline and must +not make public GitHub state a flaky merge gate: + +```sh +STREAM_GH_LIVE=1 st2 eval ./cells/stream-github-ci-waiter/ --keep +``` + +The smoke test is read-only. It requires an authenticated `gh` session with repository read access and never +reruns, cancels, edits, comments on, or otherwise mutates the PR or its checks. diff --git a/cells/stream-github-ci-waiter/fixture/contract.sh b/cells/stream-github-ci-waiter/fixture/contract.sh new file mode 100755 index 0000000..afe688d --- /dev/null +++ b/cells/stream-github-ci-waiter/fixture/contract.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +set -euo pipefail + +root="${CATALOG:?CATALOG must be set}" +net="$root/net" +export CATALOG="$net" +export ST_ROOT="$net" +export PTY_ROOT="$net/pty" +export XDG_STATE_HOME="$root/state" +export ST_AGENT=stream.worker +export STREAM_KEEP_ALIVE=0 +real_gh="$(command -v gh)" +real_st2="$(command -v st2)" + +cleanup() { + st2 down --catalog "$net" --host stream >/dev/null 2>&1 || true + PTY_ROOT="$PTY_ROOT" pty rm stream.worker >/dev/null 2>&1 || true +} +trap cleanup EXIT + +st2 validate --catalog "$net" --host stream --strict >/dev/null +export STREAM_GH_REPO=compoundingtech/st2 STREAM_GH_PR=285 STREAM_GH_POLL_SECONDS=0 + +run_fake() { + local mode="$1" count_file="$2" + shift 2 + PATH="$root/fake-bin:$PATH" FAKE_GH_MODE="$mode" FAKE_GH_COUNT_FILE="$count_file" \ + "$root/wait-gh-pr-ci.sh" "$@" +} +mkdir -p "$root/fake-bin" +ln -s "$root/fake-gh" "$root/fake-bin/gh" +chmod +x "$root/fake-gh" "$root/flaky-st2" "$root/wait-gh-pr-ci.sh" + +created="$(STREAM_GH_MAX_ATTEMPTS=5 run_fake success "$root/success.count")" +jq -e '.status == "created" and .stream == "github-ci" and (.eventId | startswith("github-pr-285-1ca43e397f804277c553b08b4571408e82831bb4-success-"))' <<<"$created" >/dev/null +test "$(cat "$root/success.count")" -eq 3 +filename="$(jq -r .filename <<<"$created")" +jq -e '.kind == "github-pr-ci" and .conclusion == "success" and .checks[0].state == "SUCCESS"' \ + <<<"$(tail -n 1 "$net/agents/stream/worker/resources/inbox/$filename")" >/dev/null + +replay="$(STREAM_GH_MAX_ATTEMPTS=5 run_fake success "$root/replay.count")" +jq -e '.status == "deduplicated"' <<<"$replay" >/dev/null +test "$(jq -r .filename <<<"$replay")" = "$filename" +echo "GH-CI-RETRY-DEDUP-GREEN-b9e4" + +delivery="$(PATH="$root/fake-bin:$PATH" FAKE_GH_MODE=success FAKE_GH_LINK_SUFFIX=/delivery \ + FAKE_GH_COUNT_FILE="$root/delivery-gh.count" FAKE_ST2_COUNT_FILE="$root/delivery-st2.count" \ + REAL_ST2="$real_st2" STREAM_ST2_BIN="$root/flaky-st2" STREAM_EMIT_INITIAL_BACKOFF_SECONDS=0 \ + STREAM_GH_MAX_ATTEMPTS=5 "$root/wait-gh-pr-ci.sh")" +jq -e '.status == "created"' <<<"$delivery" >/dev/null +test "$(cat "$root/delivery-st2.count")" -eq 3 +delivery_file="$(jq -r .filename <<<"$delivery")" +test -e "$net/agents/stream/worker/resources/inbox/$delivery_file" +echo "GH-CI-DELIVERY-RETRY-GREEN-b9e4" + +failed="$(STREAM_GH_MAX_ATTEMPTS=5 run_fake failure "$root/failure.count")" +jq -e '.status == "created" and (.eventId | contains("-failure-"))' <<<"$failed" >/dev/null +failed_file="$(jq -r .filename <<<"$failed")" +test ! -e "$net/agents/stream/worker/resources/inbox/$filename" +test -e "$net/agents/stream/worker/resources/archive/$filename" +grep -Fq '"conclusion":"failure"' "$net/agents/stream/worker/resources/inbox/$failed_file" +echo "GH-CI-FAILURE-SUPERSESSION-GREEN-b9e4" + +set +e +STREAM_GH_MAX_ATTEMPTS=3 run_fake pending "$root/pending.count" >"$root/pending.out" 2>&1 +pending_status="$?" +set -e +test "$pending_status" -eq 75 +test "$(cat "$root/pending.count")" -eq 3 +grep -Fq 'did not reach a terminal state after 3 attempts' "$root/pending.out" +test "$(find "$net/agents/stream/worker/resources/inbox" -type f | wc -l)" -eq 1 +echo "GH-CI-TIMEOUT-NO-EVENT-GREEN-b9e4" + +if test "${STREAM_GH_LIVE:-0}" = 1; then + "$real_gh" auth status >/dev/null + rm -f "$root/live.count" + live="$(STREAM_GH_BIN="$real_gh" STREAM_GH_TRACE_FILE="$root/live-trace.jsonl" STREAM_GH_MAX_ATTEMPTS=700 STREAM_GH_POLL_SECONDS=2 "$root/wait-gh-pr-ci.sh")" + jq -se 'any(.pending > 0) and (last | .pending == 0 and .unknown == 0)' "$root/live-trace.jsonl" >/dev/null + jq -e '.status == "created" or .status == "deduplicated"' <<<"$live" >/dev/null + live_file="$(jq -r .filename <<<"$live")" + grep -Fq '"repo":"compoundingtech/st2"' "$net/agents/stream/worker/resources/inbox/$live_file" + grep -Fq '"head":"1ca43e397f804277c553b08b4571408e82831bb4"' "$net/agents/stream/worker/resources/inbox/$live_file" + grep -Fq '"conclusion":"success"' "$net/agents/stream/worker/resources/inbox/$live_file" + echo "GH-CI-LIVE-PENDING-TERMINAL-GREEN-b9e4" +fi + +cleanup +trap - EXIT +test "$(PTY_ROOT="$PTY_ROOT" pty list --json | jq 'length')" -eq 0 +echo "GH-CI-CLEANUP-GREEN-b9e4" diff --git a/cells/stream-github-ci-waiter/fixture/fake-gh b/cells/stream-github-ci-waiter/fixture/fake-gh new file mode 100755 index 0000000..90c0aa6 --- /dev/null +++ b/cells/stream-github-ci-waiter/fixture/fake-gh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash +set -euo pipefail + +case "${1:-} ${2:-}" in + 'pr view') + printf '%s\n' '{"number":285,"url":"https://github.com/compoundingtech/st2/pull/285","headRefOid":"1ca43e397f804277c553b08b4571408e82831bb4","state":"MERGED"}' + ;; + 'pr checks') + count_file="${FAKE_GH_COUNT_FILE:?FAKE_GH_COUNT_FILE is required}" + count=0 + test ! -f "$count_file" || count="$(cat "$count_file")" + count=$((count + 1)) + printf '%s' "$count" >"$count_file" + if test "${FAKE_GH_MODE:-success}" = pending; then + printf '%s\n' '[{"bucket":"pending","link":"https://example.invalid/run","name":"check","state":"IN_PROGRESS","workflow":"Nix"}]' + elif test "$count" -lt 3; then + printf '%s\n' '[{"bucket":"pending","link":"https://example.invalid/run","name":"check","state":"IN_PROGRESS","workflow":"Nix"}]' + elif test "${FAKE_GH_MODE:-success}" = failure; then + printf '%s\n' '[{"bucket":"fail","link":"https://example.invalid/run","name":"check","state":"FAILURE","workflow":"Nix"}]' + else + printf '[{"bucket":"pass","link":"https://example.invalid/run%s","name":"check","state":"SUCCESS","workflow":"Nix"}]\n' "${FAKE_GH_LINK_SUFFIX:-}" + fi + ;; + *) echo "unexpected fake gh invocation: $*" >&2; exit 64;; +esac diff --git a/cells/stream-github-ci-waiter/fixture/flaky-st2 b/cells/stream-github-ci-waiter/fixture/flaky-st2 new file mode 100755 index 0000000..da9d4ca --- /dev/null +++ b/cells/stream-github-ci-waiter/fixture/flaky-st2 @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail +count_file="${FAKE_ST2_COUNT_FILE:?FAKE_ST2_COUNT_FILE is required}" +count=0 +test ! -f "$count_file" || count="$(cat "$count_file")" +count=$((count + 1)) +printf '%s' "$count" >"$count_file" +if test "$count" -le "${FAKE_ST2_FAILURES:-2}"; then + echo "injected transient st2 ingress failure $count" >&2 + exit 69 +fi +exec "${REAL_ST2:?REAL_ST2 is required}" "$@" diff --git a/cells/stream-github-ci-waiter/fixture/net/agents/stream/worker/agent.kdl b/cells/stream-github-ci-waiter/fixture/net/agents/stream/worker/agent.kdl new file mode 100644 index 0000000..d1d759d --- /dev/null +++ b/cells/stream-github-ci-waiter/fixture/net/agents/stream/worker/agent.kdl @@ -0,0 +1,5 @@ +agent "worker" { + host "stream" + command "exec sleep 300" + stream "github-ci" {} +} diff --git a/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh new file mode 100755 index 0000000..2991f75 --- /dev/null +++ b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash +set -euo pipefail + +repo="${STREAM_GH_REPO:?STREAM_GH_REPO is required}" +pr="${STREAM_GH_PR:?STREAM_GH_PR is required}" +max_attempts="${STREAM_GH_MAX_ATTEMPTS:-30}" +poll_seconds="${STREAM_GH_POLL_SECONDS:-2}" +gh_bin="${STREAM_GH_BIN:-gh}" +st2_bin="${STREAM_ST2_BIN:-st2}" +emit_attempts="${STREAM_EMIT_MAX_ATTEMPTS:-5}" +emit_backoff="${STREAM_EMIT_INITIAL_BACKOFF_SECONDS:-1}" + +case "$max_attempts" in ''|*[!0-9]*) echo "STREAM_GH_MAX_ATTEMPTS must be a positive integer" >&2; exit 2;; esac +case "$poll_seconds" in ''|*[!0-9.]*) echo "STREAM_GH_POLL_SECONDS must be a non-negative number" >&2; exit 2;; esac +test "$max_attempts" -gt 0 || { echo "STREAM_GH_MAX_ATTEMPTS must be positive" >&2; exit 2; } + +pr_json="$("$gh_bin" pr view "$pr" --repo "$repo" --json number,url,headRefOid,state)" +number="$(jq -er '.number' <<<"$pr_json")" +url="$(jq -er '.url' <<<"$pr_json")" +head="$(jq -er '.headRefOid' <<<"$pr_json")" + +for attempt in $(seq 1 "$max_attempts"); do + checks="$("$gh_bin" pr checks "$pr" --repo "$repo" --json name,state,bucket,workflow,link)" + summary="$(jq -cer 'sort_by(.workflow, .name) | map({workflow,name,state,bucket,link})' <<<"$checks")" + count="$(jq 'length' <<<"$summary")" + pending="$(jq '[.[] | select(.bucket == "pending")] | length' <<<"$summary")" + failed="$(jq '[.[] | select(.bucket == "fail" or .bucket == "cancel")] | length' <<<"$summary")" + unknown="$(jq '[.[] | select(.bucket != "pass" and .bucket != "pending" and .bucket != "fail" and .bucket != "cancel" and .bucket != "skipping")] | length' <<<"$summary")" + if test -n "${STREAM_GH_TRACE_FILE:-}"; then + jq -cn --argjson attempt "$attempt" --argjson count "$count" --argjson pending "$pending" \ + --argjson failed "$failed" --argjson unknown "$unknown" --argjson checks "$summary" \ + '{attempt:$attempt,count:$count,pending:$pending,failed:$failed,unknown:$unknown,checks:$checks}' \ + >>"$STREAM_GH_TRACE_FILE" + fi + + if test "$count" -gt 0 && test "$pending" -eq 0 && test "$unknown" -eq 0; then + if test "$failed" -eq 0; then conclusion="success"; else conclusion="failure"; fi + message="$(jq -cn \ + --argjson pr "$number" --arg repo "$repo" --arg url "$url" --arg head "$head" \ + --arg conclusion "$conclusion" --argjson checks "$summary" \ + '{kind:"github-pr-ci",repo:$repo,pr:$pr,url:$url,head:$head,conclusion:$conclusion,checks:$checks}')" + content_id="$(sha256sum <<<"$message" | cut -c1-16)" + for emit_attempt in $(seq 1 "$emit_attempts"); do + set +e + emitted="$("$st2_bin" event emit "${ST_AGENT:?ST_AGENT is required}" \ + --stream github-ci \ + --event-id "github-pr-$number-$head-$conclusion-$content_id" \ + --key "github:$repo#$number" \ + --supersede \ + --subject "GitHub PR #$number CI $conclusion" \ + --message "$message" \ + --host stream \ + --json 2>&1)" + emit_status="$?" + set -e + if test "$emit_status" -eq 0; then + printf '%s\n' "$emitted" + if test "${STREAM_KEEP_ALIVE:-1}" = 1; then exec sleep "${STREAM_KEEP_ALIVE_SECONDS:-300}"; fi + exit 0 + fi + if test "$emit_attempt" -lt "$emit_attempts"; then + sleep "$emit_backoff" + emit_backoff="$(awk -v delay="$emit_backoff" 'BEGIN { delay *= 2; if (delay > 8) delay = 8; print delay }')" + fi + done + printf '%s\n' "$emitted" >&2 + echo "failed to publish terminal GitHub PR CI result after $emit_attempts attempts" >&2 + exit 75 + fi + + if test "$attempt" -lt "$max_attempts"; then sleep "$poll_seconds"; fi +done + +echo "GitHub PR $repo#$number checks did not reach a terminal state after $max_attempts attempts" >&2 +exit 75 diff --git a/cells/stream-github-ci-waiter/stream-github-ci-waiter.kdl b/cells/stream-github-ci-waiter/stream-github-ci-waiter.kdl new file mode 100644 index 0000000..724fe5c --- /dev/null +++ b/cells/stream-github-ci-waiter/stream-github-ci-waiter.kdl @@ -0,0 +1,27 @@ +// Realistic GitHub PR CI waiter: deterministic controls by default, authenticated network smoke when opted in. +eval { + copy "./fixture" + max-timeout "25m" + + run "contract" { + command "bash ./contract.sh" + } + + judges { + judge "RETRY AND DEDUP - pending checks are polled to terminal success and replay emits once" { + exec "grep -Fqx GH-CI-RETRY-DEDUP-GREEN-b9e4 $RUNS_DIR/contract.out" + } + judge "FAILURE SUPERSESSION - terminal failure replaces the prior keyed CI result" { + exec "grep -Fqx GH-CI-FAILURE-SUPERSESSION-GREEN-b9e4 $RUNS_DIR/contract.out" + } + judge "DELIVERY RETRY - transient ingress failures back off and eventually publish exactly once" { + exec "grep -Fqx GH-CI-DELIVERY-RETRY-GREEN-b9e4 $RUNS_DIR/contract.out" + } + judge "TIMEOUT - permanently pending checks exhaust the bound without emitting" { + exec "grep -Fqx GH-CI-TIMEOUT-NO-EVENT-GREEN-b9e4 $RUNS_DIR/contract.out" + } + judge "CLEANUP - the waiter leaves no eval-owned process" { + exec "grep -Fqx GH-CI-CLEANUP-GREEN-b9e4 $RUNS_DIR/contract.out" + } + } +} diff --git a/cells/stream-nix-build-waiter/README.md b/cells/stream-nix-build-waiter/README.md new file mode 100644 index 0000000..06a7bcb --- /dev/null +++ b/cells/stream-nix-build-waiter/README.md @@ -0,0 +1,22 @@ +# stream-nix-build-waiter + +Model-free use-case E2E for waiting on real Nix builds through native st2 stream adapters. + +Two adapters are lowered from Agent Spec and supervised by st2. Each creates a uniquely named local +`runCommand` derivation so the build cannot be satisfied by an earlier eval. The builders remain active for +two seconds, making the pre-terminal waiting state observable. One build succeeds and one exits with status +23; both outcomes are converted into truthful terminal events through the public `st2 event emit` CLI. + +Each waiter repeats the identical terminal publication to model retry after an uncertain acknowledgement. +Before that succeeds, a fixture wrapper injects two transient failures at the real `st2 event emit` process +boundary. The adapter retries the exact immutable arguments with a bounded exponential schedule and remains +supervised rather than flapping. The wrapper then forwards to the explicitly captured candidate st2 binary; +Nix itself is never mocked. + +The cell then kills and reconciles both adapters, proving a fresh process derives the same event identity from +the stable build request and is also deduplicated. It retains the original filename and leaves exactly one +inbox message per build. Finally it shuts the agent down and proves no waiter task or eval-owned PTY remains. + +The fixture is hermetic with respect to external services: it uses the host's configured `` but no +network, credentials, substituter fetch, or model seat. The expected-failure derivation is an intentional +negative control, not an infrastructure error. diff --git a/cells/stream-nix-build-waiter/fixture/contract.sh b/cells/stream-nix-build-waiter/fixture/contract.sh new file mode 100755 index 0000000..728e479 --- /dev/null +++ b/cells/stream-nix-build-waiter/fixture/contract.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +set -euo pipefail + +root="${CATALOG:?CATALOG must be set}" +net="$root/net" +export CATALOG="$net" +export ST_ROOT="$net" +export PTY_ROOT="$net/pty" +export XDG_STATE_HOME="$root/state" +export ST2_REAL_BIN="$(command -v st2)" +export ST2_EVENT_BIN="$root/inject-event-failures.sh" +printf '%s\n' "$(date +%s%N)-$$" >"$root/run-id" + +cleanup() { + st2 down --catalog "$net" --host stream >/dev/null 2>&1 || true + PTY_ROOT="$PTY_ROOT" pty rm stream.nix-watcher >/dev/null 2>&1 || true +} +trap cleanup EXIT + +st2 validate --catalog "$net" --host stream --strict >/dev/null +st2 up --once --catalog "$net" --host stream >"$root/up.out" +grep -Fq 'stream.nix-watcher.stream-nix-success' "$root/up.out" +grep -Fq 'stream.nix-watcher.stream-nix-failure' "$root/up.out" + +for mode in success failure; do + for _ in $(seq 1 200); do + test -s "$root/nix-$mode.started" && break + sleep 0.05 + done + test -s "$root/nix-$mode.started" + test ! -e "$root/nix-$mode.events.jsonl" +done +st2 tasks --catalog "$net" --host stream --json >"$root/tasks-waiting.json" +jq -e ' + [.tasks[] | select(.task == "stream-nix-success" or .task == "stream-nix-failure")] + | length == 2 and all(.runtime.state == "running") +' "$root/tasks-waiting.json" >/dev/null +echo "STREAM-NIX-WAITING-GREEN-4d91" + +for mode in success failure; do + for _ in $(seq 1 1200); do + test -s "$root/nix-$mode.events.jsonl" && break + sleep 0.05 + done + test -s "$root/nix-$mode.events.jsonl" + started="$(cat "$root/nix-$mode.started")" + finished="$(cat "$root/nix-$mode.finished")" + test $((finished - started)) -ge 1800000000 +done +jq -e '.terminal == "success" and .buildStatus == 0 and .first.status == "created"' \ + "$root/nix-success.events.jsonl" >/dev/null +jq -e '.terminal == "failure" and .buildStatus == 1 and .first.status == "created"' \ + "$root/nix-failure.events.jsonl" >/dev/null +grep -Fq 'intentional-nix-waiter-failure' "$root/nix-failure.build.log" +echo "STREAM-NIX-TERMINALS-GREEN-4d91" + +for mode in success failure; do + test "$(sed -n '1,3p' "$root/injection-nix-$mode.log" | wc -l)" -eq 3 + test "$(sed -n '1,3p' "$root/injection-nix-$mode.log" | awk '{print $4}' | sort -u | wc -l)" -eq 1 + grep -Eq '^1 injected ' "$root/injection-nix-$mode.log" + grep -Eq '^2 injected ' "$root/injection-nix-$mode.log" + grep -Eq '^3 forwarded ' "$root/injection-nix-$mode.log" + jq -e ' + .retry.status == "deduplicated" and + .retry.filename == .first.filename and + .retry.recipient == "stream.nix-watcher" + ' "$root/nix-$mode.events.jsonl" >/dev/null +done +st2 tasks --catalog "$net" --host stream --json >"$root/tasks-after-retry.json" +jq -e ' + [.tasks[] | select(.task == "stream-nix-success" or .task == "stream-nix-failure")] + | length == 2 and all(.runtime.state == "running") +' "$root/tasks-after-retry.json" >/dev/null +inbox="$net/agents/stream/nix-watcher/resources/inbox" +test "$(find "$inbox" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 2 + +# Model a supervisor restart after an uncertain acknowledgement. The adapter +# derives its event identity from the stable build request, not its process ID. +st2 down --catalog "$net" --host stream >/dev/null +st2 up --once --catalog "$net" --host stream >"$root/restart-up.out" +for mode in success failure; do + for _ in $(seq 1 1200); do + test "$(wc -l <"$root/nix-$mode.events.jsonl")" -ge 2 && break + sleep 0.05 + done + test "$(wc -l <"$root/nix-$mode.events.jsonl")" -eq 2 + jq -se ' + .[1].first.status == "deduplicated" and + .[1].first.filename == .[0].first.filename and + .[1].retry.status == "deduplicated" + ' "$root/nix-$mode.events.jsonl" >/dev/null +done +test "$(find "$inbox" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 2 +echo "STREAM-NIX-RETRY-GREEN-4d91" + +st2 down --catalog "$net" --host stream >/dev/null +PTY_ROOT="$PTY_ROOT" pty rm stream.nix-watcher >/dev/null 2>&1 || true +trap - EXIT +test "$(PTY_ROOT="$PTY_ROOT" pty list --json | jq 'length')" -eq 0 +st2 tasks --catalog "$net" --host stream --json >"$root/tasks-clean.json" +jq -e '.tasks | all(.runtime.state != "running")' "$root/tasks-clean.json" >/dev/null +echo "STREAM-NIX-CLEANUP-GREEN-4d91" diff --git a/cells/stream-nix-build-waiter/fixture/inject-event-failures.sh b/cells/stream-nix-build-waiter/fixture/inject-event-failures.sh new file mode 100755 index 0000000..8ed16c0 --- /dev/null +++ b/cells/stream-nix-build-waiter/fixture/inject-event-failures.sh @@ -0,0 +1,33 @@ +#!/usr/bin/env bash +set -euo pipefail + +real="${ST2_REAL_BIN:?exact candidate st2 path required}" +root="${CATALOG:?CATALOG required}/.." +stream='' +event_id='' +previous='' +for argument in "$@"; do + case "$previous" in + --stream) stream="$argument" ;; + --event-id) event_id="$argument" ;; + esac + previous="$argument" +done +test -n "$stream" +test -n "$event_id" + +state="$root/injection-$stream.count" +count=0 +test ! -e "$state" || count="$(cat "$state")" +count=$((count + 1)) +printf '%s\n' "$count" >"$state" +payload_hash="$(printf '%s\0' "$@" | sha256sum | cut -d ' ' -f 1)" + +if test "$count" -le 2; then + printf '%s injected %s %s\n' "$count" "$event_id" "$payload_hash" >>"$root/injection-$stream.log" + echo "injected transient event delivery failure $count" >&2 + exit 75 +fi + +printf '%s forwarded %s %s\n' "$count" "$event_id" "$payload_hash" >>"$root/injection-$stream.log" +exec "$real" "$@" diff --git a/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl b/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl new file mode 100644 index 0000000..c341292 --- /dev/null +++ b/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl @@ -0,0 +1,6 @@ +agent "nix-watcher" { + host "stream" + command "sleep 300" + stream "nix-success" { argv "bash" "$CATALOG/../nix-build-waiter.sh" "success" } + stream "nix-failure" { argv "bash" "$CATALOG/../nix-build-waiter.sh" "failure" } +} diff --git a/cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh b/cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh new file mode 100755 index 0000000..3864d23 --- /dev/null +++ b/cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash +set -euo pipefail + +mode="${1:?success or failure required}" +case "$mode" in + success|failure) ;; + *) echo "unsupported waiter mode: $mode" >&2; exit 64 ;; +esac + +root="${CATALOG:?CATALOG required}/.." +started="$root/nix-$mode.started" +finished="$root/nix-$mode.finished" +result="$root/nix-$mode.events.jsonl" +build_log="$root/nix-$mode.build.log" +nonce="$(cat "$root/run-id")" +printf '%s\n' "$(date +%s%N)" >"$started" + +if test "$mode" = success; then + build_body='sleep 2; printf success > $out' + expected_status=0 + terminal=success +else + build_body='sleep 2; echo intentional-nix-waiter-failure >&2; exit 23' + expected_status=1 + terminal=failure +fi + +set +e +nix-build --no-out-link --option substitute false --expr \ + "with import {}; runCommand \"st2-stream-e2e-$mode-$nonce\" {} ''$build_body''" \ + >"$build_log" 2>&1 +build_status="$?" +set -e + +if test "$build_status" -ne "$expected_status"; then + printf 'unexpected nix-build status: got %s, expected %s\n' "$build_status" "$expected_status" >&2 + cat "$build_log" >&2 + exit 1 +fi + +event_id="nix-$mode-terminal-$nonce" +subject="Nix build $terminal" +message="real Nix build reached terminal status $terminal" +event_bin="${ST2_EVENT_BIN:-st2}" +emit_terminal() { + "$event_bin" event emit "${ST_AGENT:?ST_AGENT required}" \ + --stream "nix-$mode" \ + --event-id "$event_id" \ + --key "$mode" \ + --subject "$subject" \ + --message "$message" \ + --host stream \ + --json +} + +first='' +delay=0.05 +for attempt in 1 2 3 4 5; do + set +e + first="$(emit_terminal 2>>"$root/nix-$mode.emit.err")" + emit_status="$?" + set -e + test "$emit_status" -eq 0 && break + if test "$attempt" -eq 5; then + echo "terminal event delivery exhausted after $attempt attempts" >&2 + exit "$emit_status" + fi + sleep "$delay" + delay="$(awk -v delay="$delay" 'BEGIN { printf "%.2f", delay * 2 }')" +done + +retry="$("$event_bin" event emit "$ST_AGENT" \ + --stream "nix-$mode" \ + --event-id "$event_id" \ + --key "$mode" \ + --subject "$subject" \ + --message "$message" \ + --host stream \ + --json)" +jq -cn --argjson first "$first" --argjson retry "$retry" \ + --arg terminal "$terminal" --argjson buildStatus "$build_status" \ + '{first:$first,retry:$retry,terminal:$terminal,buildStatus:$buildStatus}' >>"$result" +printf '%s\n' "$(date +%s%N)" >"$finished" + +# Keep the adapter supervised after delivering its terminal edge; reconciliation +# must not turn a completed one-shot waiter into an emission loop. +exec sleep 300 diff --git a/cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl b/cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl new file mode 100644 index 0000000..6f1a9ad --- /dev/null +++ b/cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl @@ -0,0 +1,24 @@ +// Real, model-free Nix build waiters supervised as native st2 stream adapters. +eval { + copy "./fixture" + max-timeout "120s" + + run "contract" { + command "bash ./contract.sh" + } + + judges { + judge "WAITING - real uncached Nix derivations remain observable before terminal ingress" { + exec "grep -Fqx STREAM-NIX-WAITING-GREEN-4d91 $RUNS_DIR/contract.out" + } + judge "TERMINALS - real successful and failed Nix builds emit truthful terminal events" { + exec "grep -Fqx STREAM-NIX-TERMINALS-GREEN-4d91 $RUNS_DIR/contract.out" + } + judge "RETRY - repeated terminal publication deduplicates to one inbox message" { + exec "grep -Fqx STREAM-NIX-RETRY-GREEN-4d91 $RUNS_DIR/contract.out" + } + judge "CLEANUP - shutdown leaves neither waiter task nor eval-owned PTY running" { + exec "grep -Fqx STREAM-NIX-CLEANUP-GREEN-4d91 $RUNS_DIR/contract.out" + } + } +} diff --git a/evidence/harness-exclusions.tsv b/evidence/harness-exclusions.tsv index 6cd005c..24d5086 100644 --- a/evidence/harness-exclusions.tsv +++ b/evidence/harness-exclusions.tsv @@ -18,6 +18,8 @@ render-target-safety cell deterministic native render-policy probe; no harness s shared-workspace-render-ownership cell deterministic native shared-render admission probe; no harness seat st2-doctor-structure cell deterministic catalog-health probe; no harness seat st2-network cell deterministic catalog/message round-trip probe; no harness seat +stream-github-ci-waiter cell deterministic GitHub CI waiter controls with an opt-in authenticated read-only smoke; no harness seat +stream-nix-build-waiter cell deterministic real Nix build waiter and retry probe; no harness seat strict-validation-json cell deterministic native validation-diagnostics probe; no harness seat targeted-reconcile-isolation cell deterministic selected reconcile isolation probe; no bus-connected agent two-networks-coexist cell deterministic catalog-isolation probe; no harness seat From 93ed7ebc0f7320f7a612c796f35a30f4447a05ad Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 13:03:31 +0200 Subject: [PATCH 06/17] test(stream): prove DING delivery for Nix waiters --- CATALOG.md | 2 +- cells/stream-nix-build-waiter/README.md | 10 +++- .../fixture/contract.sh | 47 ++++++++++++++++-- .../net/agents/stream/nix-watcher/agent.kdl | 3 +- .../fixture/net/fake-tui.pl | 48 +++++++++++++++++++ .../stream-nix-build-waiter.kdl | 5 +- 6 files changed, 107 insertions(+), 8 deletions(-) create mode 100644 cells/stream-nix-build-waiter/fixture/net/fake-tui.pl diff --git a/CATALOG.md b/CATALOG.md index a9ffb39..597e168 100644 --- a/CATALOG.md +++ b/CATALOG.md @@ -65,7 +65,7 @@ harness-native loader plus canonical hook file. | `st2-doctor-structure` | model-free | — | 0 | none | `60s` | 4 | — | **NO STRUCTURED RUN** | | `st2-network` | model-free | — | 0 | none | `60s` | 2 | — | **NO STRUCTURED RUN** | | `stream-github-ci-waiter` | model-free | — | 0 | none | `25m` | 5 | — | **NO STRUCTURED RUN** | -| `stream-nix-build-waiter` | model-free | — | 0 | none | `120s` | 4 | — | **NO STRUCTURED RUN** | +| `stream-nix-build-waiter` | model-free | — | 0 | none | `120s` | 5 | — | **NO STRUCTURED RUN** | | `strict-validation-json` | model-free | — | 0 | none | `60s` | 4 | — | **NO STRUCTURED RUN** | | `targeted-reconcile-isolation` | model-free | — | 0 | none | `60s` | 7 | — | **NO STRUCTURED RUN** | | `test-writing` | Claude | `claude-sonnet-5` / medium | 2 | medium | `1200s` | 3 | — | **NO STRUCTURED RUN** | diff --git a/cells/stream-nix-build-waiter/README.md b/cells/stream-nix-build-waiter/README.md index 06a7bcb..c9310bd 100644 --- a/cells/stream-nix-build-waiter/README.md +++ b/cells/stream-nix-build-waiter/README.md @@ -13,9 +13,15 @@ boundary. The adapter retries the exact immutable arguments with a bounded expon supervised rather than flapping. The wrapper then forwards to the explicitly captured candidate st2 binary; Nix itself is never mocked. +The owning agent is a deterministic maintained-composer PTY with a real generated DING sidecar. Both terminal +events visibly reach that live session exactly once. The fixture reads their structured stream identity through +the public message CLI, archives them, reads the archived records again byte-for-byte, and proves the inbox is +drained. + The cell then kills and reconciles both adapters, proving a fresh process derives the same event identity from -the stable build request and is also deduplicated. It retains the original filename and leaves exactly one -inbox message per build. Finally it shuts the agent down and proves no waiter task or eval-owned PTY remains. +the stable build request and is also deduplicated. It retains the original archived filenames without recreating +an inbox record or redelivering either DING. Finally it shuts the agent down and proves no waiter task, +generated DING, or eval-owned PTY remains. The fixture is hermetic with respect to external services: it uses the host's configured `` but no network, credentials, substituter fetch, or model seat. The expected-failure derivation is an intentional diff --git a/cells/stream-nix-build-waiter/fixture/contract.sh b/cells/stream-nix-build-waiter/fixture/contract.sh index 728e479..a2630a0 100755 --- a/cells/stream-nix-build-waiter/fixture/contract.sh +++ b/cells/stream-nix-build-waiter/fixture/contract.sh @@ -17,6 +17,17 @@ cleanup() { } trap cleanup EXIT +wait_for_text() { + marker="$1" + file="$2" + for _ in $(seq 1 400); do + grep -Fq "$marker" "$file" 2>/dev/null && return 0 + sleep 0.05 + done + printf 'text %s did not reach %s\n' "$marker" "$file" >&2 + return 1 +} + st2 validate --catalog "$net" --host stream --strict >/dev/null st2 up --once --catalog "$net" --host stream >"$root/up.out" grep -Fq 'stream.nix-watcher.stream-nix-success' "$root/up.out" @@ -54,6 +65,33 @@ jq -e '.terminal == "failure" and .buildStatus == 1 and .first.status == "create grep -Fq 'intentional-nix-waiter-failure' "$root/nix-failure.build.log" echo "STREAM-NIX-TERMINALS-GREEN-4d91" +deliveries="$net/deliveries.log" +wait_for_text 'Nix build success' "$deliveries" +wait_for_text 'Nix build failure' "$deliveries" +test "$(wc -l <"$deliveries")" -eq 2 + +for mode in success failure; do + filename="$(jq -r '.first.filename' "$root/nix-$mode.events.jsonl")" + st2 message read stream.nix-watcher "$filename" \ + --catalog "$net" --host stream --json >"$root/nix-$mode.message.json" + jq -e --arg stream "nix-$mode" --arg key "$mode" ' + .stream == $stream and + .eventKey == $key and + .eventId != null and + (.body | contains("real Nix build reached terminal status")) + ' "$root/nix-$mode.message.json" >/dev/null + st2 message archive stream.nix-watcher "$filename" \ + --catalog "$net" --host stream >/dev/null + st2 message read stream.nix-watcher "$filename" --archive \ + --catalog "$net" --host stream --json >"$root/nix-$mode.archived.json" + cmp "$root/nix-$mode.message.json" "$root/nix-$mode.archived.json" +done +inbox="$net/agents/stream/nix-watcher/resources/inbox" +archive="$net/agents/stream/nix-watcher/resources/archive" +test "$(find "$inbox" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 0 +test "$(find "$archive" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 2 +echo "STREAM-NIX-DING-GREEN-4d91" + for mode in success failure; do test "$(sed -n '1,3p' "$root/injection-nix-$mode.log" | wc -l)" -eq 3 test "$(sed -n '1,3p' "$root/injection-nix-$mode.log" | awk '{print $4}' | sort -u | wc -l)" -eq 1 @@ -71,8 +109,8 @@ jq -e ' [.tasks[] | select(.task == "stream-nix-success" or .task == "stream-nix-failure")] | length == 2 and all(.runtime.state == "running") ' "$root/tasks-after-retry.json" >/dev/null -inbox="$net/agents/stream/nix-watcher/resources/inbox" -test "$(find "$inbox" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 2 +test "$(find "$inbox" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 0 +test "$(find "$archive" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 2 # Model a supervisor restart after an uncertain acknowledgement. The adapter # derives its event identity from the stable build request, not its process ID. @@ -90,7 +128,10 @@ for mode in success failure; do .[1].retry.status == "deduplicated" ' "$root/nix-$mode.events.jsonl" >/dev/null done -test "$(find "$inbox" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 2 +sleep 1 +test "$(wc -l <"$deliveries")" -eq 2 +test "$(find "$inbox" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 0 +test "$(find "$archive" -maxdepth 1 -type f -name '*.md' | wc -l)" -eq 2 echo "STREAM-NIX-RETRY-GREEN-4d91" st2 down --catalog "$net" --host stream >/dev/null diff --git a/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl b/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl index c341292..3de3ea3 100644 --- a/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl +++ b/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl @@ -1,6 +1,7 @@ agent "nix-watcher" { host "stream" - command "sleep 300" + command "exec perl \"$CATALOG/fake-tui.pl\"" + ding stream "nix-success" { argv "bash" "$CATALOG/../nix-build-waiter.sh" "success" } stream "nix-failure" { argv "bash" "$CATALOG/../nix-build-waiter.sh" "failure" } } diff --git a/cells/stream-nix-build-waiter/fixture/net/fake-tui.pl b/cells/stream-nix-build-waiter/fixture/net/fake-tui.pl new file mode 100644 index 0000000..039fedd --- /dev/null +++ b/cells/stream-nix-build-waiter/fixture/net/fake-tui.pl @@ -0,0 +1,48 @@ +#!/usr/bin/env perl +use strict; +use warnings; + +$| = 1; +system("stty raw -echo"); +open(my $log, ">>", "$ENV{CATALOG}/deliveries.log") or die "open deliveries: $!"; +select((select($log), $| = 1)[0]); + +sub idle { + print "\e[2J\e[H\e[1m›\e[1C\e[22;2mFind and fix a bug in \@filename\r\n\r\n"; + print " \e[0mgpt-5.6-sol xhigh · /workspace"; +} + +sub staged { + my ($text) = @_; + print "\e[2J\e[H\e[1m›\e[1C\e[0m$text\r\n\r\n"; + print " \e[0mgpt-5.6-sol xhigh · /workspace"; +} + +sub accepted { + my ($text) = @_; + print "\e[2J\e[H\e[1m›\e[1C\e[0m$text\r\n\r\n"; + print "\e[1m›\e[1C\e[22;2mFind and fix a bug in \@filename\r\n\r\n"; + print " \e[0mgpt-5.6-sol xhigh · /workspace"; +} + +idle(); + +my $buffer = ""; +my $payload; +while (1) { + my $chunk = ""; + my $read = sysread(STDIN, $chunk, 4096); + last if !defined($read) || $read == 0; + $buffer .= $chunk; + + if (!defined($payload) && $buffer =~ s/.*?\e\[200~(.*?)\e\[201~//s) { + $payload = $1; + staged($payload); + } + if (defined($payload) && $buffer =~ s/^[^\r\n]*[\r\n]//s) { + $payload =~ s/[\r\n]+/ /g; + print {$log} "$payload\n"; + accepted($payload); + undef($payload); + } +} diff --git a/cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl b/cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl index 6f1a9ad..d6e5a90 100644 --- a/cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl +++ b/cells/stream-nix-build-waiter/stream-nix-build-waiter.kdl @@ -14,7 +14,10 @@ eval { judge "TERMINALS - real successful and failed Nix builds emit truthful terminal events" { exec "grep -Fqx STREAM-NIX-TERMINALS-GREEN-4d91 $RUNS_DIR/contract.out" } - judge "RETRY - repeated terminal publication deduplicates to one inbox message" { + judge "DING - terminal events wake the live owner PTY, remain readable, and archive cleanly" { + exec "grep -Fqx STREAM-NIX-DING-GREEN-4d91 $RUNS_DIR/contract.out" + } + judge "RETRY - repeated and restarted publication does not recreate or redeliver archived events" { exec "grep -Fqx STREAM-NIX-RETRY-GREEN-4d91 $RUNS_DIR/contract.out" } judge "CLEANUP - shutdown leaves neither waiter task nor eval-owned PTY running" { From 572c0859e9017986425c5b51c03b76fefd8fc8a0 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 13:41:36 +0200 Subject: [PATCH 07/17] test(stream): close waiter review gaps --- cells/agent-spec-streams/README.md | 4 +-- .../agent-spec-streams/agent-spec-streams.kdl | 2 +- cells/agent-spec-streams/fixture/adapter.sh | 5 +++ cells/agent-spec-streams/fixture/contract.sh | 32 +++++++++++++++++++ .../net/agents/stream/worker/agent.kdl | 4 ++- .../fixture/dependency-mutations.sh | 4 +-- cells/stream-github-ci-waiter/fixture/fake-gh | 2 ++ .../fixture/wait-gh-pr-ci.sh | 7 ++++ .../net/agents/stream/nix-watcher/agent.kdl | 6 ++-- 9 files changed, 58 insertions(+), 8 deletions(-) diff --git a/cells/agent-spec-streams/README.md b/cells/agent-spec-streams/README.md index 0648a52..4f0dfd9 100644 --- a/cells/agent-spec-streams/README.md +++ b/cells/agent-spec-streams/README.md @@ -7,5 +7,5 @@ The cell crosses the public parser, reconciliation, process, event-ingress, auth inventory boundaries. Its negative controls prove the contract is fail-closed for invalid names, unsupported intervals, ambiguous launch shapes, task collisions, undeclared streams, suspended recipients, and conflicting event identity reuse. It also proves keyed-head to keyless supersession, strict discovery for ingress and -authoring, and retained no-follow state/inbox/temp-file capabilities that prevent symlink redirection outside -the agent directory. No model seat or paid provider is started. +authoring, exact argv byte boundaries, and retained no-follow state/inbox/archive/temp-file capabilities that +prevent symlink redirection outside the agent directory. No model seat or paid provider is started. diff --git a/cells/agent-spec-streams/agent-spec-streams.kdl b/cells/agent-spec-streams/agent-spec-streams.kdl index 34c4589..e35d17f 100644 --- a/cells/agent-spec-streams/agent-spec-streams.kdl +++ b/cells/agent-spec-streams/agent-spec-streams.kdl @@ -20,7 +20,7 @@ eval { judge "INGRESS - external events deduplicate stably and conflicting identity reuse fails closed" { exec "grep -Fqx STREAM-INGRESS-GREEN-83a7 $RUNS_DIR/contract.out" } - judge "CAPABILITIES - strict discovery and no-follow state/inbox operations prevent redirected writes" { + judge "CAPABILITIES - strict discovery and no-follow state/inbox/archive operations prevent redirected writes" { exec "grep -Fqx STREAM-CAPABILITIES-GREEN-83a7 $RUNS_DIR/contract.out" } judge "LIFECYCLE - suspend removes owner plus adapters and refuses ingress; resume relaunches them" { diff --git a/cells/agent-spec-streams/fixture/adapter.sh b/cells/agent-spec-streams/fixture/adapter.sh index 02ee3d6..9603d25 100755 --- a/cells/agent-spec-streams/fixture/adapter.sh +++ b/cells/agent-spec-streams/fixture/adapter.sh @@ -2,6 +2,11 @@ set -euo pipefail stream="${1:?stream name required}" +if test "$stream" = argv; then + test "$#" -eq 3 + test "$2" = 'value with spaces' + test "$3" = 'literal;$(not-expanded)' +fi marker="${CATALOG:?CATALOG required}/../adapter-$stream.json" st2 event emit "${ST_AGENT:?ST_AGENT required}" \ --stream "$stream" \ diff --git a/cells/agent-spec-streams/fixture/contract.sh b/cells/agent-spec-streams/fixture/contract.sh index e018f61..f34b79c 100755 --- a/cells/agent-spec-streams/fixture/contract.sh +++ b/cells/agent-spec-streams/fixture/contract.sh @@ -129,6 +129,38 @@ nofollow_case() { nofollow_case state streams nofollow_case inbox inbox +archive_catalog="$root/no-follow-archive" +archive_agent="$archive_catalog/agents/stream/worker" +archive_outside="$root/outside-archive" +mkdir -p "$archive_agent/resources" "$archive_outside" +cp "$original" "$archive_agent/agent.kdl" +archive_first="$(st2 event emit stream.worker \ + --catalog "$archive_catalog" \ + --stream external \ + --event-id no-follow-archive-first \ + --key archive \ + --message first \ + --host stream \ + --json)" +archive_first_filename="$(jq -r .filename <<<"$archive_first")" +rmdir "$archive_agent/resources/archive" +ln -s "$archive_outside" "$archive_agent/resources/archive" +set +e +st2 event emit stream.worker \ + --catalog "$archive_catalog" \ + --stream external \ + --event-id no-follow-archive-second \ + --key archive \ + --supersede \ + --message second \ + --host stream \ + --json >"$root/no-follow-archive.out" 2>&1 +archive_status="$?" +set -e +test "$archive_status" -ne 0 +test -e "$archive_agent/resources/inbox/$archive_first_filename" +test -z "$(find "$archive_outside" -mindepth 1 -print -quit)" + temporary_catalog="$root/no-follow-temporary" temporary_agent="$temporary_catalog/agents/stream/worker" temporary_state="$temporary_agent/resources/streams/external" diff --git a/cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl b/cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl index e082acb..7087db6 100644 --- a/cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl +++ b/cells/agent-spec-streams/fixture/net/agents/stream/worker/agent.kdl @@ -1,7 +1,9 @@ agent "worker" { host "stream" + workspace "$CATALOG/.." + env { ST_AGENT "stream.worker" } command "exec sleep 300" stream "external" {} stream "shell" { command #"exec bash "$CATALOG/../adapter.sh" shell"# } - stream "argv" { argv "bash" "$CATALOG/../adapter.sh" "argv" } + stream "argv" { argv "bash" "adapter.sh" "argv" "value with spaces" "literal;$(not-expanded)" } } diff --git a/cells/managed-agent-color-env/fixture/dependency-mutations.sh b/cells/managed-agent-color-env/fixture/dependency-mutations.sh index 8db257b..815a032 100755 --- a/cells/managed-agent-color-env/fixture/dependency-mutations.sh +++ b/cells/managed-agent-color-env/fixture/dependency-mutations.sh @@ -10,13 +10,13 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" test "$(sha256sum "$current_st2" | awk '{ print $1 }')" = \ - "e193141523bb35a397f90170f6fc182e663d205eef2d95af27a94dcae806cc48" + "25f35c420a305badc4e13eb62555a371eadd01b21d02ad448f3e4d57c4233dd4" test "$(sha256sum "$current_pty" | awk '{ print $1 }')" = \ "1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" test "$(sha256sum "$old_st2" | awk '{ print $1 }')" = \ "d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" case "$("$current_st2" --version)" in - *"(8ca0f36,"*) ;; + *"(493bc5c,"*) ;; *) printf 'unexpected current st2 identity: %s\n' "$("$current_st2" --version)" >&2; exit 1 ;; esac case "$("$old_st2" --version)" in diff --git a/cells/stream-github-ci-waiter/fixture/fake-gh b/cells/stream-github-ci-waiter/fixture/fake-gh index 90c0aa6..21fbf6e 100755 --- a/cells/stream-github-ci-waiter/fixture/fake-gh +++ b/cells/stream-github-ci-waiter/fixture/fake-gh @@ -13,8 +13,10 @@ case "${1:-} ${2:-}" in printf '%s' "$count" >"$count_file" if test "${FAKE_GH_MODE:-success}" = pending; then printf '%s\n' '[{"bucket":"pending","link":"https://example.invalid/run","name":"check","state":"IN_PROGRESS","workflow":"Nix"}]' + exit 8 elif test "$count" -lt 3; then printf '%s\n' '[{"bucket":"pending","link":"https://example.invalid/run","name":"check","state":"IN_PROGRESS","workflow":"Nix"}]' + exit 8 elif test "${FAKE_GH_MODE:-success}" = failure; then printf '%s\n' '[{"bucket":"fail","link":"https://example.invalid/run","name":"check","state":"FAILURE","workflow":"Nix"}]' else diff --git a/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh index 2991f75..0b3501a 100755 --- a/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh +++ b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh @@ -20,7 +20,14 @@ url="$(jq -er '.url' <<<"$pr_json")" head="$(jq -er '.headRefOid' <<<"$pr_json")" for attempt in $(seq 1 "$max_attempts"); do + set +e checks="$("$gh_bin" pr checks "$pr" --repo "$repo" --json name,state,bucket,workflow,link)" + checks_status="$?" + set -e + if test "$checks_status" -ne 0 && test "$checks_status" -ne 8; then + printf 'gh pr checks failed with status %s\n' "$checks_status" >&2 + exit "$checks_status" + fi summary="$(jq -cer 'sort_by(.workflow, .name) | map({workflow,name,state,bucket,link})' <<<"$checks")" count="$(jq 'length' <<<"$summary")" pending="$(jq '[.[] | select(.bucket == "pending")] | length' <<<"$summary")" diff --git a/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl b/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl index 3de3ea3..bc53a6d 100644 --- a/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl +++ b/cells/stream-nix-build-waiter/fixture/net/agents/stream/nix-watcher/agent.kdl @@ -1,7 +1,9 @@ agent "nix-watcher" { host "stream" + workspace "$CATALOG/.." + env { ST_AGENT "stream.nix-watcher" } command "exec perl \"$CATALOG/fake-tui.pl\"" ding - stream "nix-success" { argv "bash" "$CATALOG/../nix-build-waiter.sh" "success" } - stream "nix-failure" { argv "bash" "$CATALOG/../nix-build-waiter.sh" "failure" } + stream "nix-success" { argv "bash" "nix-build-waiter.sh" "success" } + stream "nix-failure" { argv "bash" "nix-build-waiter.sh" "failure" } } From f857869de8596d548fc5356137588dd3ac908e0e Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 13:56:05 +0200 Subject: [PATCH 08/17] test(stream): repin final reviewed st2 candidate --- AGENT-SPEC.md | 8 ++++---- README.md | 2 +- bin/check-corpus.sh | 6 +++--- .../fixture/dependency-mutations.sh | 4 ++-- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index b4dff17..5cc9f2a 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`493bc5cb70c680650a7a49de0d3b33cccd5b6810`](https://github.com/compoundingtech/st2/commit/493bc5cb70c680650a7a49de0d3b33cccd5b6810) -(`0.1.0`, source `493bc5c`). The pin identifies the implementation and version the corpus currently proves; it +[`474e6c0bd143d713486da7d8b875fcb62c6832a6`](https://github.com/compoundingtech/st2/commit/474e6c0bd143d713486da7d8b875fcb62c6832a6) +(`0.1.0`, source `474e6c0`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -600,8 +600,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned release-mode executable for source `493bc5c` has SHA256 -`25f35c420a305badc4e13eb62555a371eadd01b21d02ad448f3e4d57c4233dd4`. It is built from the exact source +The latest independently pinned release-mode executable for source `474e6c0` has SHA256 +`bc8cd6b10ff2ba98bd847f070fa45be44fb869d0daefd45367aa0d9c9adb82fc`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index 19532d2..1febaed 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ VERDICT: PASS ``` Requirements are `st2 0.1.0` from source -[`493bc5c`](https://github.com/compoundingtech/st2/commit/493bc5cb70c680650a7a49de0d3b33cccd5b6810), +[`474e6c0`](https://github.com/compoundingtech/st2/commit/474e6c0bd143d713486da7d8b875fcb62c6832a6), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index e415708..a547e71 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="493bc5c" -expected_binary_sha256="25f35c420a305badc4e13eb62555a371eadd01b21d02ad448f3e4d57c4233dd4" -expected_version_regex='^st2 0\.1\.0 — running from local source \(493bc5c, .+ ago\)$' +expected_source="474e6c0" +expected_binary_sha256="bc8cd6b10ff2ba98bd847f070fa45be44fb869d0daefd45367aa0d9c9adb82fc" +expected_version_regex='^st2 0\.1\.0 — running from local source \(474e6c0, .+ ago\)$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/managed-agent-color-env/fixture/dependency-mutations.sh b/cells/managed-agent-color-env/fixture/dependency-mutations.sh index 815a032..1ac10be 100755 --- a/cells/managed-agent-color-env/fixture/dependency-mutations.sh +++ b/cells/managed-agent-color-env/fixture/dependency-mutations.sh @@ -10,13 +10,13 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" test "$(sha256sum "$current_st2" | awk '{ print $1 }')" = \ - "25f35c420a305badc4e13eb62555a371eadd01b21d02ad448f3e4d57c4233dd4" + "bc8cd6b10ff2ba98bd847f070fa45be44fb869d0daefd45367aa0d9c9adb82fc" test "$(sha256sum "$current_pty" | awk '{ print $1 }')" = \ "1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" test "$(sha256sum "$old_st2" | awk '{ print $1 }')" = \ "d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" case "$("$current_st2" --version)" in - *"(493bc5c,"*) ;; + *"(474e6c0,"*) ;; *) printf 'unexpected current st2 identity: %s\n' "$("$current_st2" --version)" >&2; exit 1 ;; esac case "$("$old_st2" --version)" in From 2101e2d151e24f4ad128c0aa188b9aea75e596ce Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:01:01 +0200 Subject: [PATCH 09/17] fix(eval): accept failed GitHub check results --- AGENT-SPEC.md | 8 ++++---- README.md | 2 +- bin/check-corpus.sh | 6 +++--- cells/managed-agent-color-env/README.md | 9 ++++----- .../fixture/dependency-mutations.sh | 4 ++-- cells/managed-agent-color-env/fixture/matrix.sh | 4 ++-- cells/stream-github-ci-waiter/fixture/fake-gh | 1 + cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh | 4 +++- 8 files changed, 20 insertions(+), 18 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index 5cc9f2a..71f228f 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`474e6c0bd143d713486da7d8b875fcb62c6832a6`](https://github.com/compoundingtech/st2/commit/474e6c0bd143d713486da7d8b875fcb62c6832a6) -(`0.1.0`, source `474e6c0`). The pin identifies the implementation and version the corpus currently proves; it +[`47c4aed618dc88413001f6149f2c2bf86a549851`](https://github.com/compoundingtech/st2/commit/47c4aed618dc88413001f6149f2c2bf86a549851) +(`0.1.0`, source `47c4aed`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -600,8 +600,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned release-mode executable for source `474e6c0` has SHA256 -`bc8cd6b10ff2ba98bd847f070fa45be44fb869d0daefd45367aa0d9c9adb82fc`. It is built from the exact source +The latest independently pinned release-mode executable for source `47c4aed` has SHA256 +`d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index 1febaed..469c7a4 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ VERDICT: PASS ``` Requirements are `st2 0.1.0` from source -[`474e6c0`](https://github.com/compoundingtech/st2/commit/474e6c0bd143d713486da7d8b875fcb62c6832a6), +[`47c4aed`](https://github.com/compoundingtech/st2/commit/47c4aed618dc88413001f6149f2c2bf86a549851), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index a547e71..34e5f29 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="474e6c0" -expected_binary_sha256="bc8cd6b10ff2ba98bd847f070fa45be44fb869d0daefd45367aa0d9c9adb82fc" -expected_version_regex='^st2 0\.1\.0 — running from local source \(474e6c0, .+ ago\)$' +expected_source="47c4aed" +expected_binary_sha256="d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08" +expected_version_regex='^st2 0\.1\.0 — running from local source \(47c4aed, .+ ago\)$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/managed-agent-color-env/README.md b/cells/managed-agent-color-env/README.md index 8867a86..b8fd4af 100644 --- a/cells/managed-agent-color-env/README.md +++ b/cells/managed-agent-color-env/README.md @@ -50,14 +50,13 @@ checks run before either run step starts. The control is the Linux executable fr The composed runtime uses: - PTY `d5fabc3917407aeb937a012bd97679c303e18033` (merged); -- st2 `8ca0f3653aae6ca6ebcb880f82e9296016df6e08`, the merge identity of - [st2 #132](https://github.com/compoundingtech/st2/pull/132). +- st2 `47c4aed618dc88413001f6149f2c2bf86a549851`, the exact stream candidate proven by this corpus. The executable SHA-256 identities are: - PTY: `1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3`; -- st2 merged source: `e193141523bb35a397f90170f6fc182e663d205eef2d95af27a94dcae806cc48`; +- st2 candidate source: `d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08`; - st2 `0fed14b` control: `d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed`. -The merged st2 identity is the accepted composition boundary for this cell. The repository-wide corpus runner -remains a separate published-version pin until its normal release update. +The exact st2 candidate identity is the accepted composition boundary for this cell and the repository-wide +corpus runner. diff --git a/cells/managed-agent-color-env/fixture/dependency-mutations.sh b/cells/managed-agent-color-env/fixture/dependency-mutations.sh index 1ac10be..562d938 100755 --- a/cells/managed-agent-color-env/fixture/dependency-mutations.sh +++ b/cells/managed-agent-color-env/fixture/dependency-mutations.sh @@ -10,13 +10,13 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" test "$(sha256sum "$current_st2" | awk '{ print $1 }')" = \ - "bc8cd6b10ff2ba98bd847f070fa45be44fb869d0daefd45367aa0d9c9adb82fc" + "d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08" test "$(sha256sum "$current_pty" | awk '{ print $1 }')" = \ "1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" test "$(sha256sum "$old_st2" | awk '{ print $1 }')" = \ "d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" case "$("$current_st2" --version)" in - *"(474e6c0,"*) ;; + *"(47c4aed,"*) ;; *) printf 'unexpected current st2 identity: %s\n' "$("$current_st2" --version)" >&2; exit 1 ;; esac case "$("$old_st2" --version)" in diff --git a/cells/managed-agent-color-env/fixture/matrix.sh b/cells/managed-agent-color-env/fixture/matrix.sh index 4da1cf4..920fb17 100755 --- a/cells/managed-agent-color-env/fixture/matrix.sh +++ b/cells/managed-agent-color-env/fixture/matrix.sh @@ -10,8 +10,8 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" systemd-run --user --scope --quiet true -expected_st2_commit="8ca0f36" -expected_st2_sha256="e193141523bb35a397f90170f6fc182e663d205eef2d95af27a94dcae806cc48" +expected_st2_commit="47c4aed" +expected_st2_sha256="d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08" expected_pty_sha256="1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" st2_path="$(command -v st2)" pty_path="$(command -v pty)" diff --git a/cells/stream-github-ci-waiter/fixture/fake-gh b/cells/stream-github-ci-waiter/fixture/fake-gh index 21fbf6e..509143b 100755 --- a/cells/stream-github-ci-waiter/fixture/fake-gh +++ b/cells/stream-github-ci-waiter/fixture/fake-gh @@ -19,6 +19,7 @@ case "${1:-} ${2:-}" in exit 8 elif test "${FAKE_GH_MODE:-success}" = failure; then printf '%s\n' '[{"bucket":"fail","link":"https://example.invalid/run","name":"check","state":"FAILURE","workflow":"Nix"}]' + exit 1 else printf '[{"bucket":"pass","link":"https://example.invalid/run%s","name":"check","state":"SUCCESS","workflow":"Nix"}]\n' "${FAKE_GH_LINK_SUFFIX:-}" fi diff --git a/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh index 0b3501a..60db651 100755 --- a/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh +++ b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh @@ -24,7 +24,9 @@ for attempt in $(seq 1 "$max_attempts"); do checks="$("$gh_bin" pr checks "$pr" --repo "$repo" --json name,state,bucket,workflow,link)" checks_status="$?" set -e - if test "$checks_status" -ne 0 && test "$checks_status" -ne 8; then + # `gh pr checks` returns 1 for a terminal failed/cancelled check while still + # writing the requested JSON, and 8 while checks remain pending. + if test "$checks_status" -ne 0 && test "$checks_status" -ne 1 && test "$checks_status" -ne 8; then printf 'gh pr checks failed with status %s\n' "$checks_status" >&2 exit "$checks_status" fi From 0b5ae32d14193be475776154f0f73622dcb36fa7 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:03:41 +0200 Subject: [PATCH 10/17] fix(eval): drain coalesced stream deliveries --- cells/stream-github-ci-waiter/fixture/contract.sh | 2 +- .../stream-nix-build-waiter/fixture/net/fake-tui.pl | 13 ++++++++----- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/cells/stream-github-ci-waiter/fixture/contract.sh b/cells/stream-github-ci-waiter/fixture/contract.sh index afe688d..d483519 100755 --- a/cells/stream-github-ci-waiter/fixture/contract.sh +++ b/cells/stream-github-ci-waiter/fixture/contract.sh @@ -9,7 +9,6 @@ export PTY_ROOT="$net/pty" export XDG_STATE_HOME="$root/state" export ST_AGENT=stream.worker export STREAM_KEEP_ALIVE=0 -real_gh="$(command -v gh)" real_st2="$(command -v st2)" cleanup() { @@ -72,6 +71,7 @@ test "$(find "$net/agents/stream/worker/resources/inbox" -type f | wc -l)" -eq 1 echo "GH-CI-TIMEOUT-NO-EVENT-GREEN-b9e4" if test "${STREAM_GH_LIVE:-0}" = 1; then + real_gh="$(command -v gh)" "$real_gh" auth status >/dev/null rm -f "$root/live.count" live="$(STREAM_GH_BIN="$real_gh" STREAM_GH_TRACE_FILE="$root/live-trace.jsonl" STREAM_GH_MAX_ATTEMPTS=700 STREAM_GH_POLL_SECONDS=2 "$root/wait-gh-pr-ci.sh")" diff --git a/cells/stream-nix-build-waiter/fixture/net/fake-tui.pl b/cells/stream-nix-build-waiter/fixture/net/fake-tui.pl index 039fedd..a3aa591 100644 --- a/cells/stream-nix-build-waiter/fixture/net/fake-tui.pl +++ b/cells/stream-nix-build-waiter/fixture/net/fake-tui.pl @@ -35,11 +35,14 @@ sub accepted { last if !defined($read) || $read == 0; $buffer .= $chunk; - if (!defined($payload) && $buffer =~ s/.*?\e\[200~(.*?)\e\[201~//s) { - $payload = $1; - staged($payload); - } - if (defined($payload) && $buffer =~ s/^[^\r\n]*[\r\n]//s) { + while (1) { + if (!defined($payload)) { + last unless $buffer =~ s/.*?\e\[200~(.*?)\e\[201~//s; + $payload = $1; + staged($payload); + next; + } + last unless $buffer =~ s/^[^\r\n]*[\r\n]//s; $payload =~ s/[\r\n]+/ /g; print {$log} "$payload\n"; accepted($payload); From 807b3be63caf75b75726df56dd3191a294b1f925 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:11:41 +0200 Subject: [PATCH 11/17] test(stream): repin authenticated ingress candidate --- AGENT-SPEC.md | 8 ++++---- README.md | 2 +- bin/check-corpus.sh | 6 +++--- cells/managed-agent-color-env/README.md | 4 ++-- .../fixture/dependency-mutations.sh | 4 ++-- cells/managed-agent-color-env/fixture/matrix.sh | 4 ++-- 6 files changed, 14 insertions(+), 14 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index 71f228f..dee109d 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`47c4aed618dc88413001f6149f2c2bf86a549851`](https://github.com/compoundingtech/st2/commit/47c4aed618dc88413001f6149f2c2bf86a549851) -(`0.1.0`, source `47c4aed`). The pin identifies the implementation and version the corpus currently proves; it +[`15c63a9c4821a1fad68da4f65f58606917d0760a`](https://github.com/compoundingtech/st2/commit/15c63a9c4821a1fad68da4f65f58606917d0760a) +(`0.1.0`, source `15c63a9`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -600,8 +600,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned release-mode executable for source `47c4aed` has SHA256 -`d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08`. It is built from the exact source +The latest independently pinned release-mode executable for source `15c63a9` has SHA256 +`6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index 469c7a4..00a1f8d 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ VERDICT: PASS ``` Requirements are `st2 0.1.0` from source -[`47c4aed`](https://github.com/compoundingtech/st2/commit/47c4aed618dc88413001f6149f2c2bf86a549851), +[`15c63a9`](https://github.com/compoundingtech/st2/commit/15c63a9c4821a1fad68da4f65f58606917d0760a), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index 34e5f29..87201f9 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="47c4aed" -expected_binary_sha256="d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08" -expected_version_regex='^st2 0\.1\.0 — running from local source \(47c4aed, .+ ago\)$' +expected_source="15c63a9" +expected_binary_sha256="6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc" +expected_version_regex='^st2 0\.1\.0 — running from local source \(15c63a9, .+ ago\)$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/managed-agent-color-env/README.md b/cells/managed-agent-color-env/README.md index b8fd4af..7740f8e 100644 --- a/cells/managed-agent-color-env/README.md +++ b/cells/managed-agent-color-env/README.md @@ -50,12 +50,12 @@ checks run before either run step starts. The control is the Linux executable fr The composed runtime uses: - PTY `d5fabc3917407aeb937a012bd97679c303e18033` (merged); -- st2 `47c4aed618dc88413001f6149f2c2bf86a549851`, the exact stream candidate proven by this corpus. +- st2 `15c63a9c4821a1fad68da4f65f58606917d0760a`, the exact stream candidate proven by this corpus. The executable SHA-256 identities are: - PTY: `1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3`; -- st2 candidate source: `d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08`; +- st2 candidate source: `6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc`; - st2 `0fed14b` control: `d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed`. The exact st2 candidate identity is the accepted composition boundary for this cell and the repository-wide diff --git a/cells/managed-agent-color-env/fixture/dependency-mutations.sh b/cells/managed-agent-color-env/fixture/dependency-mutations.sh index 562d938..d75f0b2 100755 --- a/cells/managed-agent-color-env/fixture/dependency-mutations.sh +++ b/cells/managed-agent-color-env/fixture/dependency-mutations.sh @@ -10,13 +10,13 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" test "$(sha256sum "$current_st2" | awk '{ print $1 }')" = \ - "d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08" + "6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc" test "$(sha256sum "$current_pty" | awk '{ print $1 }')" = \ "1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" test "$(sha256sum "$old_st2" | awk '{ print $1 }')" = \ "d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" case "$("$current_st2" --version)" in - *"(47c4aed,"*) ;; + *"(15c63a9,"*) ;; *) printf 'unexpected current st2 identity: %s\n' "$("$current_st2" --version)" >&2; exit 1 ;; esac case "$("$old_st2" --version)" in diff --git a/cells/managed-agent-color-env/fixture/matrix.sh b/cells/managed-agent-color-env/fixture/matrix.sh index 920fb17..7de9764 100755 --- a/cells/managed-agent-color-env/fixture/matrix.sh +++ b/cells/managed-agent-color-env/fixture/matrix.sh @@ -10,8 +10,8 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" systemd-run --user --scope --quiet true -expected_st2_commit="47c4aed" -expected_st2_sha256="d2d692bde797c162071e30ee4a6cf9889bdd8cadb38227cd64d1f29af47cba08" +expected_st2_commit="15c63a9" +expected_st2_sha256="6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc" expected_pty_sha256="1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" st2_path="$(command -v st2)" pty_path="$(command -v pty)" From cdc530a0881cd7a2188c7a0282df13c56d042452 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:14:07 +0200 Subject: [PATCH 12/17] fix(eval): accept Nix build-failure status 100 --- cells/stream-nix-build-waiter/fixture/contract.sh | 2 +- .../fixture/nix-build-waiter.sh | 11 +++++++++-- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/cells/stream-nix-build-waiter/fixture/contract.sh b/cells/stream-nix-build-waiter/fixture/contract.sh index a2630a0..96d7f73 100755 --- a/cells/stream-nix-build-waiter/fixture/contract.sh +++ b/cells/stream-nix-build-waiter/fixture/contract.sh @@ -60,7 +60,7 @@ for mode in success failure; do done jq -e '.terminal == "success" and .buildStatus == 0 and .first.status == "created"' \ "$root/nix-success.events.jsonl" >/dev/null -jq -e '.terminal == "failure" and .buildStatus == 1 and .first.status == "created"' \ +jq -e '.terminal == "failure" and (.buildStatus == 1 or .buildStatus == 100) and .first.status == "created"' \ "$root/nix-failure.events.jsonl" >/dev/null grep -Fq 'intentional-nix-waiter-failure' "$root/nix-failure.build.log" echo "STREAM-NIX-TERMINALS-GREEN-4d91" diff --git a/cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh b/cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh index 3864d23..317b03e 100755 --- a/cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh +++ b/cells/stream-nix-build-waiter/fixture/nix-build-waiter.sh @@ -21,7 +21,7 @@ if test "$mode" = success; then terminal=success else build_body='sleep 2; echo intentional-nix-waiter-failure >&2; exit 23' - expected_status=1 + expected_status='1 or 100' terminal=failure fi @@ -32,7 +32,14 @@ nix-build --no-out-link --option substitute false --expr \ build_status="$?" set -e -if test "$build_status" -ne "$expected_status"; then +if test "$mode" = success; then + status_matches=$((build_status == 0)) +else + # Legacy nix-build releases use 1 for a failed derivation; newer releases + # reserve 100 for this same terminal build-failure outcome. + status_matches=$((build_status == 1 || build_status == 100)) +fi +if test "$status_matches" -ne 1; then printf 'unexpected nix-build status: got %s, expected %s\n' "$build_status" "$expected_status" >&2 cat "$build_log" >&2 exit 1 From 9a22edfa21bfe0356916efb6fca203d82ffd045e Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:21:51 +0200 Subject: [PATCH 13/17] test(stream): repin final reviewed ingress candidate --- AGENT-SPEC.md | 8 ++++---- README.md | 2 +- bin/check-corpus.sh | 6 +++--- cells/managed-agent-color-env/README.md | 4 ++-- .../fixture/dependency-mutations.sh | 4 ++-- cells/managed-agent-color-env/fixture/matrix.sh | 4 ++-- 6 files changed, 14 insertions(+), 14 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index dee109d..027cac2 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`15c63a9c4821a1fad68da4f65f58606917d0760a`](https://github.com/compoundingtech/st2/commit/15c63a9c4821a1fad68da4f65f58606917d0760a) -(`0.1.0`, source `15c63a9`). The pin identifies the implementation and version the corpus currently proves; it +[`6e06b55100dca79ca6976eac52a3e4af1dd0816f`](https://github.com/compoundingtech/st2/commit/6e06b55100dca79ca6976eac52a3e4af1dd0816f) +(`0.1.0`, source `6e06b55`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -600,8 +600,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned release-mode executable for source `15c63a9` has SHA256 -`6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc`. It is built from the exact source +The latest independently pinned release-mode executable for source `6e06b55` has SHA256 +`b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index 00a1f8d..a937a52 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ VERDICT: PASS ``` Requirements are `st2 0.1.0` from source -[`15c63a9`](https://github.com/compoundingtech/st2/commit/15c63a9c4821a1fad68da4f65f58606917d0760a), +[`6e06b55`](https://github.com/compoundingtech/st2/commit/6e06b55100dca79ca6976eac52a3e4af1dd0816f), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index 87201f9..3a7a25a 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="15c63a9" -expected_binary_sha256="6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc" -expected_version_regex='^st2 0\.1\.0 — running from local source \(15c63a9, .+ ago\)$' +expected_source="6e06b55" +expected_binary_sha256="b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f" +expected_version_regex='^st2 0\.1\.0 — running from local source \(6e06b55, .+ ago\)$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/managed-agent-color-env/README.md b/cells/managed-agent-color-env/README.md index 7740f8e..5e24737 100644 --- a/cells/managed-agent-color-env/README.md +++ b/cells/managed-agent-color-env/README.md @@ -50,12 +50,12 @@ checks run before either run step starts. The control is the Linux executable fr The composed runtime uses: - PTY `d5fabc3917407aeb937a012bd97679c303e18033` (merged); -- st2 `15c63a9c4821a1fad68da4f65f58606917d0760a`, the exact stream candidate proven by this corpus. +- st2 `6e06b55100dca79ca6976eac52a3e4af1dd0816f`, the exact stream candidate proven by this corpus. The executable SHA-256 identities are: - PTY: `1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3`; -- st2 candidate source: `6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc`; +- st2 candidate source: `b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f`; - st2 `0fed14b` control: `d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed`. The exact st2 candidate identity is the accepted composition boundary for this cell and the repository-wide diff --git a/cells/managed-agent-color-env/fixture/dependency-mutations.sh b/cells/managed-agent-color-env/fixture/dependency-mutations.sh index d75f0b2..16857eb 100755 --- a/cells/managed-agent-color-env/fixture/dependency-mutations.sh +++ b/cells/managed-agent-color-env/fixture/dependency-mutations.sh @@ -10,13 +10,13 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" test "$(sha256sum "$current_st2" | awk '{ print $1 }')" = \ - "6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc" + "b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f" test "$(sha256sum "$current_pty" | awk '{ print $1 }')" = \ "1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" test "$(sha256sum "$old_st2" | awk '{ print $1 }')" = \ "d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" case "$("$current_st2" --version)" in - *"(15c63a9,"*) ;; + *"(6e06b55,"*) ;; *) printf 'unexpected current st2 identity: %s\n' "$("$current_st2" --version)" >&2; exit 1 ;; esac case "$("$old_st2" --version)" in diff --git a/cells/managed-agent-color-env/fixture/matrix.sh b/cells/managed-agent-color-env/fixture/matrix.sh index 7de9764..1e7863b 100755 --- a/cells/managed-agent-color-env/fixture/matrix.sh +++ b/cells/managed-agent-color-env/fixture/matrix.sh @@ -10,8 +10,8 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" systemd-run --user --scope --quiet true -expected_st2_commit="15c63a9" -expected_st2_sha256="6f5e76942f56b4ea1d2121d3f47efd216f27c91686333e95a9cfe5877e3f36fc" +expected_st2_commit="6e06b55" +expected_st2_sha256="b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f" expected_pty_sha256="1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" st2_path="$(command -v st2)" pty_path="$(command -v pty)" From 1bcccd83aff2b6d11f4cf4dd07e8f16fcb0560fc Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:23:56 +0200 Subject: [PATCH 14/17] fix(eval): fence GitHub waiter to one PR head --- cells/stream-github-ci-waiter/fixture/contract.sh | 8 ++++++++ cells/stream-github-ci-waiter/fixture/fake-gh | 11 +++++++++-- .../stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh | 6 ++++++ 3 files changed, 23 insertions(+), 2 deletions(-) diff --git a/cells/stream-github-ci-waiter/fixture/contract.sh b/cells/stream-github-ci-waiter/fixture/contract.sh index d483519..37a9a50 100755 --- a/cells/stream-github-ci-waiter/fixture/contract.sh +++ b/cells/stream-github-ci-waiter/fixture/contract.sh @@ -70,6 +70,14 @@ grep -Fq 'did not reach a terminal state after 3 attempts' "$root/pending.out" test "$(find "$net/agents/stream/worker/resources/inbox" -type f | wc -l)" -eq 1 echo "GH-CI-TIMEOUT-NO-EVENT-GREEN-b9e4" +set +e +STREAM_GH_MAX_ATTEMPTS=3 run_fake head-change "$root/head-change.count" >"$root/head-change.out" 2>&1 +head_change_status="$?" +set -e +test "$head_change_status" -eq 75 +grep -Fq 'GitHub PR head changed while polling' "$root/head-change.out" +test "$(find "$net/agents/stream/worker/resources/inbox" -type f | wc -l)" -eq 1 + if test "${STREAM_GH_LIVE:-0}" = 1; then real_gh="$(command -v gh)" "$real_gh" auth status >/dev/null diff --git a/cells/stream-github-ci-waiter/fixture/fake-gh b/cells/stream-github-ci-waiter/fixture/fake-gh index 509143b..9f55b15 100755 --- a/cells/stream-github-ci-waiter/fixture/fake-gh +++ b/cells/stream-github-ci-waiter/fixture/fake-gh @@ -3,7 +3,11 @@ set -euo pipefail case "${1:-} ${2:-}" in 'pr view') - printf '%s\n' '{"number":285,"url":"https://github.com/compoundingtech/st2/pull/285","headRefOid":"1ca43e397f804277c553b08b4571408e82831bb4","state":"MERGED"}' + head='1ca43e397f804277c553b08b4571408e82831bb4' + if test "${FAKE_GH_MODE:-success}" = head-change && test -e "${FAKE_GH_COUNT_FILE:?}.head-changed"; then + head='aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' + fi + printf '{"number":285,"url":"https://github.com/compoundingtech/st2/pull/285","headRefOid":"%s","state":"MERGED"}\n' "$head" ;; 'pr checks') count_file="${FAKE_GH_COUNT_FILE:?FAKE_GH_COUNT_FILE is required}" @@ -11,7 +15,10 @@ case "${1:-} ${2:-}" in test ! -f "$count_file" || count="$(cat "$count_file")" count=$((count + 1)) printf '%s' "$count" >"$count_file" - if test "${FAKE_GH_MODE:-success}" = pending; then + if test "${FAKE_GH_MODE:-success}" = head-change; then + : >"$count_file.head-changed" + printf '%s\n' '[{"bucket":"pass","link":"https://example.invalid/run","name":"check","state":"SUCCESS","workflow":"Nix"}]' + elif test "${FAKE_GH_MODE:-success}" = pending; then printf '%s\n' '[{"bucket":"pending","link":"https://example.invalid/run","name":"check","state":"IN_PROGRESS","workflow":"Nix"}]' exit 8 elif test "$count" -lt 3; then diff --git a/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh index 60db651..ed66d49 100755 --- a/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh +++ b/cells/stream-github-ci-waiter/fixture/wait-gh-pr-ci.sh @@ -30,6 +30,12 @@ for attempt in $(seq 1 "$max_attempts"); do printf 'gh pr checks failed with status %s\n' "$checks_status" >&2 exit "$checks_status" fi + current_head_json="$("$gh_bin" pr view "$pr" --repo "$repo" --json headRefOid)" + current_head="$(jq -er '.headRefOid' <<<"$current_head_json")" + if test "$current_head" != "$head"; then + printf 'GitHub PR head changed while polling: %s -> %s\n' "$head" "$current_head" >&2 + exit 75 + fi summary="$(jq -cer 'sort_by(.workflow, .name) | map({workflow,name,state,bucket,link})' <<<"$checks")" count="$(jq 'length' <<<"$summary")" pending="$(jq '[.[] | select(.bucket == "pending")] | length' <<<"$summary")" From c08defc1baa4a11bb36c654b3d3dc278adf1cca1 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:34:45 +0200 Subject: [PATCH 15/17] test(stream): repin final lifecycle candidate --- AGENT-SPEC.md | 8 ++++---- README.md | 2 +- bin/check-corpus.sh | 6 +++--- cells/managed-agent-color-env/README.md | 4 ++-- .../fixture/dependency-mutations.sh | 4 ++-- cells/managed-agent-color-env/fixture/matrix.sh | 4 ++-- 6 files changed, 14 insertions(+), 14 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index 027cac2..48eeee4 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`6e06b55100dca79ca6976eac52a3e4af1dd0816f`](https://github.com/compoundingtech/st2/commit/6e06b55100dca79ca6976eac52a3e4af1dd0816f) -(`0.1.0`, source `6e06b55`). The pin identifies the implementation and version the corpus currently proves; it +[`389eeb8038ad8e20736bb762dd509ea6a6fa46f1`](https://github.com/compoundingtech/st2/commit/389eeb8038ad8e20736bb762dd509ea6a6fa46f1) +(`0.1.0`, source `389eeb8`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -600,8 +600,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned release-mode executable for source `6e06b55` has SHA256 -`b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f`. It is built from the exact source +The latest independently pinned release-mode executable for source `389eeb8` has SHA256 +`f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index a937a52..bf668a9 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ VERDICT: PASS ``` Requirements are `st2 0.1.0` from source -[`6e06b55`](https://github.com/compoundingtech/st2/commit/6e06b55100dca79ca6976eac52a3e4af1dd0816f), +[`389eeb8`](https://github.com/compoundingtech/st2/commit/389eeb8038ad8e20736bb762dd509ea6a6fa46f1), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell also needs every harness named by its dry-run row. diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index 3a7a25a..572e92a 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="6e06b55" -expected_binary_sha256="b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f" -expected_version_regex='^st2 0\.1\.0 — running from local source \(6e06b55, .+ ago\)$' +expected_source="389eeb8" +expected_binary_sha256="f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9" +expected_version_regex='^st2 0\.1\.0 — running from local source \(389eeb8, .+ ago\)$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/managed-agent-color-env/README.md b/cells/managed-agent-color-env/README.md index 5e24737..e6e56aa 100644 --- a/cells/managed-agent-color-env/README.md +++ b/cells/managed-agent-color-env/README.md @@ -50,12 +50,12 @@ checks run before either run step starts. The control is the Linux executable fr The composed runtime uses: - PTY `d5fabc3917407aeb937a012bd97679c303e18033` (merged); -- st2 `6e06b55100dca79ca6976eac52a3e4af1dd0816f`, the exact stream candidate proven by this corpus. +- st2 `389eeb8038ad8e20736bb762dd509ea6a6fa46f1`, the exact stream candidate proven by this corpus. The executable SHA-256 identities are: - PTY: `1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3`; -- st2 candidate source: `b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f`; +- st2 candidate source: `f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9`; - st2 `0fed14b` control: `d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed`. The exact st2 candidate identity is the accepted composition boundary for this cell and the repository-wide diff --git a/cells/managed-agent-color-env/fixture/dependency-mutations.sh b/cells/managed-agent-color-env/fixture/dependency-mutations.sh index 16857eb..3e0fa64 100755 --- a/cells/managed-agent-color-env/fixture/dependency-mutations.sh +++ b/cells/managed-agent-color-env/fixture/dependency-mutations.sh @@ -10,13 +10,13 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" test "$(sha256sum "$current_st2" | awk '{ print $1 }')" = \ - "b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f" + "f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9" test "$(sha256sum "$current_pty" | awk '{ print $1 }')" = \ "1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" test "$(sha256sum "$old_st2" | awk '{ print $1 }')" = \ "d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" case "$("$current_st2" --version)" in - *"(6e06b55,"*) ;; + *"(389eeb8,"*) ;; *) printf 'unexpected current st2 identity: %s\n' "$("$current_st2" --version)" >&2; exit 1 ;; esac case "$("$old_st2" --version)" in diff --git a/cells/managed-agent-color-env/fixture/matrix.sh b/cells/managed-agent-color-env/fixture/matrix.sh index 1e7863b..db8721a 100755 --- a/cells/managed-agent-color-env/fixture/matrix.sh +++ b/cells/managed-agent-color-env/fixture/matrix.sh @@ -10,8 +10,8 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" systemd-run --user --scope --quiet true -expected_st2_commit="6e06b55" -expected_st2_sha256="b5a2a408830039e35ebfbfd4ab29ba66c629b810c8e7efa4f1d061faba3d405f" +expected_st2_commit="389eeb8" +expected_st2_sha256="f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9" expected_pty_sha256="1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" st2_path="$(command -v st2)" pty_path="$(command -v pty)" From 9e072ad26eedff9bb7422d70e187bff02b3759cf Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:36:38 +0200 Subject: [PATCH 16/17] fix(eval): preflight Nix waiter dependencies --- README.md | 6 ++++-- bin/overnight.sh | 19 ++++++++++++++++++- 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index bf668a9..18acccc 100644 --- a/README.md +++ b/README.md @@ -98,8 +98,10 @@ VERDICT: PASS Requirements are `st2 0.1.0` from source [`389eeb8`](https://github.com/compoundingtech/st2/commit/389eeb8038ad8e20736bb762dd509ea6a6fa46f1), -`pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell -also needs every harness named by its dry-run row. +`pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. The +`stream-nix-build-waiter` cell additionally requires `nix-build`, `nix-instantiate`, and a configured +`` lookup; the overnight runner checks these before starting the selected corpus. A paid cell also +needs every harness named by its dry-run row. ## Cell layout diff --git a/bin/overnight.sh b/bin/overnight.sh index 5640810..5147398 100755 --- a/bin/overnight.sh +++ b/bin/overnight.sh @@ -128,9 +128,11 @@ fi requires_claude=0 requires_codex=0 -while IFS=$'\t' read -r _cell _harness models _effort _seats _cost _timeout _judges; do +requires_nix=0 +while IFS=$'\t' read -r cell _harness models _effort _seats _cost _timeout _judges; do [[ "$models" != *claude-sonnet-5* ]] || requires_claude=1 [[ "$models" != *gpt-5.6-sol* ]] || requires_codex=1 + [ "$cell" != stream-nix-build-waiter ] || requires_nix=1 done < "$inventory" printf '%-30s %-10s %-35s %-7s %-5s %-8s %s\n' \ @@ -182,6 +184,21 @@ echo echo "== free preflight (no model seats) ==" bin/check-corpus.sh +if [ "$requires_nix" -eq 1 ]; then + command -v nix-build >/dev/null || { + echo "FAIL: selected cells require Nix, but nix-build is not on PATH" >&2 + exit 1 + } + command -v nix-instantiate >/dev/null || { + echo "FAIL: selected cells require Nix, but nix-instantiate is not on PATH" >&2 + exit 1 + } + nix-instantiate --find-file nixpkgs >/dev/null 2>&1 || { + echo "FAIL: selected cells require Nix, but is not configured" >&2 + exit 1 + } +fi + if [ "$requires_claude" -eq 1 ]; then command -v claude >/dev/null || { echo "FAIL: selected cells require Claude, but claude is not on PATH" >&2 From 5095efcd25aee36dd6b6e84ac8382f0298e76405 Mon Sep 17 00:00:00 2001 From: Johannes Schickling Date: Thu, 20 Aug 2026 14:39:12 +0200 Subject: [PATCH 17/17] test(stream): repin final authenticated candidate --- AGENT-SPEC.md | 8 ++++---- README.md | 2 +- bin/check-corpus.sh | 6 +++--- cells/managed-agent-color-env/README.md | 4 ++-- .../fixture/dependency-mutations.sh | 4 ++-- cells/managed-agent-color-env/fixture/matrix.sh | 4 ++-- 6 files changed, 14 insertions(+), 14 deletions(-) diff --git a/AGENT-SPEC.md b/AGENT-SPEC.md index 48eeee4..49d5719 100644 --- a/AGENT-SPEC.md +++ b/AGENT-SPEC.md @@ -5,8 +5,8 @@ by evals. st2 is the current implementation, not the owner of the contract; a fu implementation can target the same contract and proofs. The current corpus proof is pinned to st2 -[`389eeb8038ad8e20736bb762dd509ea6a6fa46f1`](https://github.com/compoundingtech/st2/commit/389eeb8038ad8e20736bb762dd509ea6a6fa46f1) -(`0.1.0`, source `389eeb8`). The pin identifies the implementation and version the corpus currently proves; it +[`ffdb83c9541978a96ff8ce4c466628e15918cbc1`](https://github.com/compoundingtech/st2/commit/ffdb83c9541978a96ff8ce4c466628e15918cbc1) +(`0.1.0`, source `ffdb83c`). The pin identifies the implementation and version the corpus currently proves; it does not transfer ownership of the specification to st2. A proposed behavior change must update this contract and its maintained proof cells before an implementation claims conformance. Do not infer additional fields or commands from older corpus fixtures. @@ -600,8 +600,8 @@ Inspect the declaration, every referenced template, and every workspace destinat materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting the network is a separate, explicitly authorized action. -The latest independently pinned release-mode executable for source `389eeb8` has SHA256 -`f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9`. It is built from the exact source +The latest independently pinned release-mode executable for source `ffdb83c` has SHA256 +`adbd2099db237c17df3dac29052cb387f4ed99888e7477910c33e518c377a3e8`. It is built from the exact source above and exercised by the maintained stream cell. It is not yet an immutable published release artifact. For the older source `0fed14b`, the accepted published Linux executable has SHA256 diff --git a/README.md b/README.md index 18acccc..12908eb 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ VERDICT: PASS ``` Requirements are `st2 0.1.0` from source -[`389eeb8`](https://github.com/compoundingtech/st2/commit/389eeb8038ad8e20736bb762dd509ea6a6fa46f1), +[`ffdb83c`](https://github.com/compoundingtech/st2/commit/ffdb83c9541978a96ff8ce4c466628e15918cbc1), `pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. The `stream-nix-build-waiter` cell additionally requires `nix-build`, `nix-instantiate`, and a configured `` lookup; the overnight runner checks these before starting the selected corpus. A paid cell also diff --git a/bin/check-corpus.sh b/bin/check-corpus.sh index 572e92a..db4ba15 100755 --- a/bin/check-corpus.sh +++ b/bin/check-corpus.sh @@ -5,9 +5,9 @@ set -euo pipefail repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$repo_root" -expected_source="389eeb8" -expected_binary_sha256="f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9" -expected_version_regex='^st2 0\.1\.0 — running from local source \(389eeb8, .+ ago\)$' +expected_source="ffdb83c" +expected_binary_sha256="adbd2099db237c17df3dac29052cb387f4ed99888e7477910c33e518c377a3e8" +expected_version_regex='^st2 0\.1\.0 — running from local source \(ffdb83c, .+ ago\)$' st2_path="$(command -v st2)" actual_version="$(st2 --version)" [[ "$actual_version" =~ $expected_version_regex ]] || { diff --git a/cells/managed-agent-color-env/README.md b/cells/managed-agent-color-env/README.md index e6e56aa..23193a7 100644 --- a/cells/managed-agent-color-env/README.md +++ b/cells/managed-agent-color-env/README.md @@ -50,12 +50,12 @@ checks run before either run step starts. The control is the Linux executable fr The composed runtime uses: - PTY `d5fabc3917407aeb937a012bd97679c303e18033` (merged); -- st2 `389eeb8038ad8e20736bb762dd509ea6a6fa46f1`, the exact stream candidate proven by this corpus. +- st2 `ffdb83c9541978a96ff8ce4c466628e15918cbc1`, the exact stream candidate proven by this corpus. The executable SHA-256 identities are: - PTY: `1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3`; -- st2 candidate source: `f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9`; +- st2 candidate source: `adbd2099db237c17df3dac29052cb387f4ed99888e7477910c33e518c377a3e8`; - st2 `0fed14b` control: `d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed`. The exact st2 candidate identity is the accepted composition boundary for this cell and the repository-wide diff --git a/cells/managed-agent-color-env/fixture/dependency-mutations.sh b/cells/managed-agent-color-env/fixture/dependency-mutations.sh index 3e0fa64..5333199 100755 --- a/cells/managed-agent-color-env/fixture/dependency-mutations.sh +++ b/cells/managed-agent-color-env/fixture/dependency-mutations.sh @@ -10,13 +10,13 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" test "$(sha256sum "$current_st2" | awk '{ print $1 }')" = \ - "f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9" + "adbd2099db237c17df3dac29052cb387f4ed99888e7477910c33e518c377a3e8" test "$(sha256sum "$current_pty" | awk '{ print $1 }')" = \ "1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" test "$(sha256sum "$old_st2" | awk '{ print $1 }')" = \ "d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed" case "$("$current_st2" --version)" in - *"(389eeb8,"*) ;; + *"(ffdb83c,"*) ;; *) printf 'unexpected current st2 identity: %s\n' "$("$current_st2" --version)" >&2; exit 1 ;; esac case "$("$old_st2" --version)" in diff --git a/cells/managed-agent-color-env/fixture/matrix.sh b/cells/managed-agent-color-env/fixture/matrix.sh index db8721a..3e9f937 100755 --- a/cells/managed-agent-color-env/fixture/matrix.sh +++ b/cells/managed-agent-color-env/fixture/matrix.sh @@ -10,8 +10,8 @@ test -S "$runtime_dir/bus" export XDG_RUNTIME_DIR="$runtime_dir" systemd-run --user --scope --quiet true -expected_st2_commit="389eeb8" -expected_st2_sha256="f3dbd6901f59decdea999de08c6ddc72683a36d79ad4cf5cb219b554c63c8ba9" +expected_st2_commit="ffdb83c" +expected_st2_sha256="adbd2099db237c17df3dac29052cb387f4ed99888e7477910c33e518c377a3e8" expected_pty_sha256="1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3" st2_path="$(command -v st2)" pty_path="$(command -v pty)"