Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 56 additions & 7 deletions AGENT-SPEC.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Canonical st2 agent specification

This is the sole agent-authoring specification for this repository. It is pinned to st2
[`9887b2842222def0838c2cd82e6c24c218f7efa6`](https://github.com/compoundingtech/st2/commit/9887b2842222def0838c2cd82e6c24c218f7efa6)
(`0.1.0`, source `9887b28`). It documents the hand-authored KDL accepted at that commit. Do not infer
[`acb00164d8e1d0b08e70c8cc7fb932aee214f555`](https://github.com/compoundingtech/st2/commit/acb00164d8e1d0b08e70c8cc7fb932aee214f555)
(`0.1.0`, source `acb0016`). It documents the hand-authored KDL accepted at that commit. Do not infer
additional fields or commands from older corpus fixtures.

st2 runs long-lived `service` agents made of interactive `pty` tasks and terminal-free `exec` tasks.
Expand Down Expand Up @@ -38,6 +38,8 @@ owns product work. Shipped declarations must not contain a developer's absolute
```kdl
agent "<identity>" {
identity "<identity>"
name "<human-facing name>"
description "<enduring responsibility>"
host "<host>"
role "worker"
type "service"
Expand Down Expand Up @@ -75,6 +77,8 @@ Supported agent children are:
| Node | Meaning |
|---|---|
| `identity "…"` | Overrides the positional/path-derived identity. |
| `name "…"` | Optional, non-unique human-facing name. It never routes or selects work. |
| `description "…"` | Optional enduring responsibility boundary. It never grants authority. |
| `host "…"` | Execution host. The canonical folder path and content should agree. |
| `role "…"` | Optional metadata with no execution behavior. |
| `type "service"` | Optional; `service` is the only accepted value and the default. |
Expand All @@ -94,6 +98,51 @@ Canonical declarations normally omit `type`. Unknown non-render children may be
extension syntax, and required behavior must never depend on them. `schedule` is explicitly reserved and
rejected. Unknown render directives are errors.

## Stable identity and presentation

The positional or child `identity` remains the stable automation ID. It alone
owns bus routing, supervisor edges, task IDs, durable state paths, resources,
authorization, and lifecycle reconciliation. Existing positional and child
identity grammar remains valid; roster JSON continues to expose the stable bus
ID as `identity`.

`name` and `description` are optional presentation metadata. Explicit values
must be non-empty, trimmed, single-line, and free of control characters. Limits
are 160 Unicode scalars for `name` and 1,000 for `description`; omission means
absent. Names need not be unique and are never st2 message, status, resource,
authorization, or lifecycle aliases. The retired sibling `name` file is not a
fallback source.

The roster exposes separate nullable `name` and `description` fields. Mutable
canonical KDL can be edited source-preservingly with:

```console
st2 rename <stable-id> <name>
st2 rename <stable-id> --clear
st2 describe <stable-id> <description>
st2 describe <stable-id> --clear
```

An agent may edit itself; a declared supervisor ancestor may edit a descendant;
an operator with no `ST_AGENT` may edit a selected mutable catalog. Peer edits
fail. JSON, TOML, and declarations marked `meta { managed-by "nix" }` are
readable but refuse these live authoring commands. Concurrent commands serialize
through the catalog lock and preserve both accepted field edits.

Every managed PTY receives the owned tags
`agent.presentation.schema=1`, `agent.actor.path=<host>.<identity>`, and the
optional `agent.presentation.description`. Only the primary `agent` PTY maps
Agent Spec `name` to native `displayName`; secondary PTYs retain their existing
task-specific display behavior. st2 reconciles this envelope through exact PTY
IDs with one atomic metadata patch, preserves unrelated tags, clears removed
owned fields, emits one coherent `metadata_change` event per real transition,
and emits no event for a no-op. Presentation reconciliation never replaces the
PTY process or its durable Agent Spec state.

This contract is proved model-free by
[`agent-presentation-contract`](./cells/agent-presentation-contract/) and
[`agent-presentation-continuity`](./cells/agent-presentation-continuity/).

The restart defaults are 3 attempts per 60 seconds, no delay, and `mode "delay"`. Durations accept bare
seconds or `ms`, `s|sec|secs`, `m|min|mins`, `h|hr|hrs`, and `d|day|days`. `mode "delay"` keeps retrying with
the window reset; `mode "fail"` parks the task after attempts are exhausted and sends one best-effort
Expand Down Expand Up @@ -432,8 +481,8 @@ Inspect the declaration, every referenced template, and every workspace destinat
materialization command. Materialization is byte-idempotent and does not imply hook installation. Starting
the network is a separate, explicitly authorized action.

For source `9887b28`, the accepted Linux executable has SHA256
`d49d44fd4f3f6f655455c212353a469fefa956082bedf22163deb767d8a36a0d`; its published archive has SHA256
`32ee103bd17ccb3e155ac63d816a3906c2470a3c98e3cc04b56e5a67138b9927`. `bin/check-corpus.sh` verifies
the variable-age version contract, exact installed binary, embedded full source commit, strict semantic
validation, fixture resets, and the rest of the model-free corpus gate before an eval may run.
For source `acb00164d8e1d0b08e70c8cc7fb932aee214f555`, the accepted Linux executable has SHA256
`bd933332784b1d87ba3d539c5570e4e3fd0572504ee2ec1ee52acbb5e8cdbbf4`; its published archive has SHA256
`c09a743c5a757998edcb3ff2963b80e459a9050d8c452d5eb3b241d34806ac86`. `bin/check-corpus.sh` verifies
the variable-age version contract with the Nix-stamped short revision, exact installed binary, strict
semantic validation, fixture resets, and the rest of the model-free corpus gate before an eval may run.
8 changes: 6 additions & 2 deletions CATALOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<!-- Generated by bin/generate-catalog.sh from cell KDL and evidence/*.tsv. Do not edit. -->
# Current eval catalog and evidence

This is the sole current corpus surface: **47 included cells** (16 model-free,
This is the sole current corpus surface: **49 included cells** (18 model-free,
31 model-backed) and **4 retired exclusions**. Inventory fields are derived from the active
KDL and executable judge scripts. Latest accepted PASS evidence remains distinct from the append-only run
history: a failed last run is visible without being advertised as accepted evidence, and cells with no
Expand All @@ -17,6 +17,8 @@ harness-native loader plus canonical hook file.
| Cell | Harness | Model(s) / effort | Model seats | Cost | Timeout | Held-out judges | Latest accepted PASS | Last recorded run |
|---|---|---|---:|---|---|---:|---|---|
| `adopt-only-migration` | model-free | — | 0 | none | `90s` | 6 | — | **NO STRUCTURED RUN** |
| `agent-presentation-continuity` | model-free | — | 0 | none | `90s` | 5 | — | **NO STRUCTURED RUN** |
| `agent-presentation-contract` | model-free | — | 0 | none | `90s` | 8 | — | **NO STRUCTURED RUN** |
| `context-resource-continuity` | model-free | — | 0 | none | `90s` | 4 | — | **NO STRUCTURED RUN** |
| `crash-ding` | mixed | `claude-sonnet-5+gpt-5.6-sol` / medium | 4 | high | `180s` | 5 | — | **NO STRUCTURED RUN** |
| `ding-mode` | Claude | `claude-sonnet-5` / medium | 2 | medium | `1200s` | 4 | — | **NO STRUCTURED RUN** |
Expand Down Expand Up @@ -84,6 +86,8 @@ while the last-run column makes a recorded failure distinct from a cell with no
## Harness hook exclusions

- `adopt-only-migration` / `cell` — deterministic native adoption/replacement lifecycle probe; no harness seat
- `agent-presentation-continuity` / `cell` — deterministic native presentation continuity probe; no harness seat
- `agent-presentation-contract` / `cell` — deterministic native Agent Spec and PTY presentation probe; no harness seat
- `context-resource-continuity` / `cell` — deterministic native context/resource restart probe; no harness seat
- `docs` / `judge:cold-reader` — one-shot offline Claude print grader; no bus identity, DING, or hook surface
- `exec-lifecycle-logging` / `cell` — deterministic native exec lifecycle probe; no harness seat
Expand All @@ -101,7 +105,7 @@ while the last-run column makes a recorded failure distinct from a cell with no
- `targeted-reconcile-isolation` / `cell` — deterministic selected reconcile isolation probe; no bus-connected agent
- `two-networks-coexist` / `cell` — deterministic catalog-isolation probe; no harness seat

These **17** rows are generated from structured exclusions. A cell-level row is required for
These **19** rows are generated from structured exclusions. A cell-level row is required for
every derived model-free cell. The one-shot docs cold reader is a model-backed grader, but not a bus-connected
agent: it has no st2 identity, DING sidecar, or lifecycle hooks.

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ VERDICT: PASS
```

Requirements are `st2 0.1.0` from source
[`9887b28`](https://github.com/compoundingtech/st2/commit/9887b2842222def0838c2cd82e6c24c218f7efa6),
[`acb0016`](https://github.com/compoundingtech/st2/commit/acb00164d8e1d0b08e70c8cc7fb932aee214f555),
`pty`, Bash, Git, `jq`, Rust/Cargo for the pinned KDL parser gate, and Node for JavaScript fixtures. A paid cell
also needs every harness named by its dry-run row.

Expand Down
19 changes: 10 additions & 9 deletions bin/check-corpus.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,15 @@ set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repo_root"

expected_source="9887b28"
expected_source_full="9887b2842222def0838c2cd82e6c24c218f7efa6"
expected_binary_sha256="d49d44fd4f3f6f655455c212353a469fefa956082bedf22163deb767d8a36a0d"
expected_version_regex='^st2 0\.1\.0 — running from local source \(9887b28, .+ ago\)$'
expected_source="acb0016"
expected_source_full="acb00164d8e1d0b08e70c8cc7fb932aee214f555"
expected_archive_sha256="c09a743c5a757998edcb3ff2963b80e459a9050d8c452d5eb3b241d34806ac86"
expected_binary_sha256="bd933332784b1d87ba3d539c5570e4e3fd0572504ee2ec1ee52acbb5e8cdbbf4"
expected_version_regex='^st2 0\.1\.0\+acb0016 — committed .+ ago$'
[[ "$expected_source_full" == "$expected_source"* ]] || {
echo "FAIL: full pinned source $expected_source_full does not begin with short source $expected_source" >&2
exit 1
}
st2_path="$(command -v st2)"
actual_version="$(st2 --version)"
[[ "$actual_version" =~ $expected_version_regex ]] || {
Expand All @@ -20,11 +25,7 @@ actual_binary_sha256="$(sha256sum "$st2_path" | awk '{ print $1 }')"
echo "FAIL: expected st2 binary sha256 $expected_binary_sha256, found $actual_binary_sha256 at $st2_path" >&2
exit 1
}
LC_ALL=C grep -aFq "$expected_source_full" "$st2_path" || {
echo "FAIL: st2 binary at $st2_path does not embed full pinned source $expected_source_full" >&2
exit 1
}
echo "PASS: pinned runner source $expected_source ($actual_version; sha256 $actual_binary_sha256)"
echo "PASS: pinned runner source $expected_source_full (archive sha256 $expected_archive_sha256; $actual_version; binary sha256 $actual_binary_sha256)"

mapfile -d '' shell_files < <(
find bin cells -type f -name '*.sh' -not -path '*/_git/*' -print0 | sort -z
Expand Down
20 changes: 20 additions & 0 deletions cells/agent-presentation-continuity/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# agent-presentation-continuity

Model-free acceptance coverage for adding, changing, and clearing Agent Spec
presentation metadata without changing stable identity or the live process
generation.

The cell begins with the existing positional stable ID and a conflicting
legacy sibling `name` file. It proves that Agent Spec is authoritative: absent
presentation does not fall back to the sibling file, and declared `name` and
`description` appear in the roster after an in-place catalog
edit. Across add, repeat, change, and clear reconciliations it requires the
same PTY ID, PID, creation timestamp, and single `session_start` event.

The same transitions must preserve unread and archived messages, context,
decisions, resources, presence, and terminal transcript. Cleanup uses the
existing explicit retirement lifecycle and leaves no PTY state.

This cell does not retire the existing `identity` grammar. Schema versioning
and breaking grammar changes belong to compoundingtech/st2#127; this cell is
the additive evidence for compoundingtech/st2#128.
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// Additive Agent Spec presentation changes preserve stable identity, runtime generation, and durable state.
eval {
copy "./fixture"
max-timeout "90s"

run "continuity" {
command "bash ./continuity.sh"
}

judges {
judge "AUTHORITY - Agent Spec presentation ignores the retired sibling name file" {
exec "grep -Fqx PRESENTATION-AUTHORITY-GREEN-a128 $RUNS_DIR/continuity.out"
}
judge "ROSTER - add, change, repeat, and clear project exact presentation fields" {
exec "grep -Fqx PRESENTATION-ROSTER-GREEN-a128 $RUNS_DIR/continuity.out"
}
judge "GENERATION - every presentation-only reconcile preserves PTY id, pid, createdAt, and one session start" {
exec "grep -Fqx PRESENTATION-GENERATION-GREEN-a128 $RUNS_DIR/continuity.out"
}
judge "DURABILITY - inbox, archive, context, decisions, resources, presence, and transcript survive" {
exec "grep -Fqx PRESENTATION-DURABILITY-GREEN-a128 $RUNS_DIR/continuity.out"
}
judge "CLEANUP - explicit retirement leaves no retained PTY state" {
exec "grep -Fqx PRESENTATION-CLEANUP-GREEN-a128 $RUNS_DIR/continuity.out"
}
}
}
160 changes: 160 additions & 0 deletions cells/agent-presentation-continuity/fixture/continuity.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
#!/usr/bin/env bash
set -euo pipefail

root="${CATALOG:?CATALOG must be set}"
net="$root/net"
spec="$net/agents/pc/worker/agent.kdl"
original="$root/worker.original.kdl"
pty_id="pc.worker"
export PTY_ROOT="$net/pty"
export XDG_STATE_HOME="$root/state"

cp "$spec" "$original"

pty_at() {
env -u PTY_SESSION PTY_ROOT="$PTY_ROOT" pty "$@"
}

session() {
pty_at list --json | jq -cer --arg id "$pty_id" '.[] | select(.name == $id)'
}

generation() {
session | jq -c '{name,pid,createdAt}'
}

start_count() {
jq -s '[.[] | select(.type == "session_start")] | length' "$PTY_ROOT/$pty_id.events.jsonl"
}

wait_running() {
for _ in $(seq 1 100); do
test "$(session | jq -r '.status')" = running && return 0
sleep 0.05
done
echo "presentation fixture did not become ready" >&2
return 1
}

retire() {
grep -Fq 'retired #true' "$spec" || sed -i '/role "worker"/a\\ retired #true' "$spec"
}

cleanup() {
if test -f "$spec"; then
retire 2>/dev/null || true
st2 up --once --catalog "$net" --host pc >/dev/null 2>&1 || true
fi
pty_at kill "$pty_id" >/dev/null 2>&1 || true
pty_at rm "$pty_id" >/dev/null 2>&1 || true
cp "$original" "$spec" 2>/dev/null || true
}
trap cleanup EXIT

st2 validate --catalog "$net" --host pc --strict >/dev/null
st2 agents --catalog "$net" --host pc --json >"$root/roster-absent.json"
jq -e '
length == 2 and
(.[] | select(.identity == "pc.worker") | .name == null and .description == null)
' "$root/roster-absent.json" >/dev/null
echo "PRESENTATION-AUTHORITY-GREEN-a128"

st2 up --once --catalog "$net" --host pc >"$root/launch.out"
grep -Fq 'launched (1): pc.worker' "$root/launch.out"
wait_running
before="$(generation)"
test "$(start_count)" -eq 1

st2 status "$pty_id" --set busy --catalog "$net" --host pc --as "$pty_id" >/dev/null
printf '%s\n' CONTEXT-NOW-a128 | st2 context write "$pty_id" --catalog "$net" --as "$pty_id"
st2 context append "$pty_id" --catalog "$net" --as "$pty_id" \
--decision DECISION-a128 --why DECISION-WHY-a128
st2 resource add https://example.invalid/presentation-a128 \
--catalog "$net" --as "$pty_id" --title PRESENTATION-RESOURCE-a128 \
--tag presentation,continuity --relation output >"$root/resource-ref"

st2 message send "$pty_id" --catalog "$net" --host pc --as pc.sender \
--subject ARCHIVED-a128 >/dev/null <<'MSG'
ARCHIVED-BODY-a128
MSG
archived_path=("$net/agents/pc/worker/resources/inbox/"*.md)
test "${#archived_path[@]}" -eq 1
archived_name="$(basename "${archived_path[0]}")"
st2 message archive "$pty_id" "$archived_name" --catalog "$net" --host pc \
--as "$pty_id" >/dev/null
st2 message send "$pty_id" --catalog "$net" --host pc --as pc.sender \
--subject UNREAD-a128 >/dev/null <<'MSG'
UNREAD-BODY-a128
MSG

for _ in $(seq 1 100); do
pty_at peek --plain "$pty_id" 2>/dev/null | grep -Fq PRESENTATION-TRANSCRIPT-a128 && break
sleep 0.05
done
pty_at peek --plain "$pty_id" | grep -Fq PRESENTATION-TRANSCRIPT-a128

add_fields() {
sed -i '/role "worker"/a\\ description "Owns durable identity acceptance"\n name "Evidence Worker"' "$spec"
}

assert_generation() {
test "$(generation)" = "$before"
test "$(start_count)" -eq 1
}

add_fields
st2 validate --catalog "$net" --host pc --strict >/dev/null
st2 up --once --catalog "$net" --host pc >"$root/add.out"
grep -Fq 'adopted (1): worker' "$root/add.out"
test -z "$(sed -n '/launched (/p;/torn down (/p' "$root/add.out")"
assert_generation
st2 agents --catalog "$net" --host pc --json >"$root/roster-added.json"
jq -e '
.[] | select(.identity == "pc.worker") |
.name == "Evidence Worker" and
.description == "Owns durable identity acceptance"
' "$root/roster-added.json" >/dev/null

st2 up --once --catalog "$net" --host pc >"$root/repeat.out"
grep -Fq 'adopted (1): worker' "$root/repeat.out"
assert_generation

sed -i 's/name "Evidence Worker"/name "Shared Presentation"/' "$spec"
sed -i 's/description "Owns durable identity acceptance"/description "Owns changed identity acceptance"/' "$spec"
st2 up --once --catalog "$net" --host pc >"$root/change.out"
assert_generation
st2 agents --catalog "$net" --host pc --json >"$root/roster-changed.json"
jq -e '
.[] | select(.identity == "pc.worker") |
.name == "Shared Presentation" and
.description == "Owns changed identity acceptance"
' "$root/roster-changed.json" >/dev/null

sed -i '/^ name "/d;/^ description "/d' "$spec"
st2 up --once --catalog "$net" --host pc >"$root/clear.out"
assert_generation
st2 agents --catalog "$net" --host pc --json >"$root/roster-cleared.json"
jq -e '
.[] | select(.identity == "pc.worker") |
.name == null and .description == null
' "$root/roster-cleared.json" >/dev/null
echo "PRESENTATION-ROSTER-GREEN-a128"
echo "PRESENTATION-GENERATION-GREEN-a128"

test "$(st2 status "$pty_id" --catalog "$net" --host pc --as "$pty_id")" = busy
st2 context read "$pty_id" --catalog "$net" --as "$pty_id" | grep -Fq CONTEXT-NOW-a128
st2 context read "$pty_id" --catalog "$net" --as "$pty_id" --decisions | grep -Fq DECISION-WHY-a128
st2 resource ls --catalog "$net" --as "$pty_id" | grep -Fq https://example.invalid/presentation-a128
test -f "$net/agents/pc/worker/resources/archive/$archived_name"
grep -Fq ARCHIVED-BODY-a128 "$net/agents/pc/worker/resources/archive/$archived_name"
grep -Fq UNREAD-BODY-a128 "$net/agents/pc/worker/resources/inbox/"*.md
pty_at peek --plain "$pty_id" | grep -Fq PRESENTATION-TRANSCRIPT-a128
echo "PRESENTATION-DURABILITY-GREEN-a128"

retire
st2 up --once --catalog "$net" --host pc >/dev/null
pty_at rm "$pty_id" >/dev/null 2>&1 || true
cp "$original" "$spec"
trap - EXIT
test "$(pty_at list --json | jq 'length')" -eq 0
echo "PRESENTATION-CLEANUP-GREEN-a128"
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
agent "sender" {
identity "sender"
host "pc"
role "fixture"
retired #true
workspace "$CATALOG/workspace"
command "true"
}
Loading