diff --git a/.github/workflows/verify.yml b/.github/workflows/verify.yml index afad4a80b..7e7e6b347 100644 --- a/.github/workflows/verify.yml +++ b/.github/workflows/verify.yml @@ -70,3 +70,14 @@ jobs: # its corpus exits 2 and fails the job — an unmeasured guard is never folded into a pass. - name: Run every guard run: npm run check:all + + # Delivery cost is the one property no guard measures, because it is a property of a WALK + # rather than of a file: the gate walks work-package end to end and compares the characters + # delivered against scripts/fixtures/token-benchmark-baseline.json. The corpus above is the + # pinned commit, so a delta is attributable to what this tree changed. + # + # A definition change that adds delivery fails here at 1%. That is the gate working: confirm + # the increase is wanted, re-record the fixture in the same commit, and say what the corpus + # gained. Left unrun, delivery rose 31.3% in 32 days with nothing reporting it. + - name: Gate delivery cost against the recorded baseline + run: npm run --silent bench:token -- --label=ci --context-mode=fresh --gate > /dev/null diff --git a/docs/development.md b/docs/development.md index f71928fac..693ab6605 100644 --- a/docs/development.md +++ b/docs/development.md @@ -213,12 +213,30 @@ Stdout is one JSON object with per-activity fresh/resume characters and the aggr [`scripts/run-token-benchmark.ts`](../scripts/run-token-benchmark.ts) measures payload-char and history/ledger cost for a fixed headless walk (`work-package` / e2e `skip-optional`), comparing `context_mode: fresh` vs `persistent` and resource reference delivery. It reuses the e2e harness/walker and probes `get_resource` for linked + hot templates (the robot walker does not call `get_resource` on its own). -By default each run compares against the frozen pre-optimisation reference -[`scripts/fixtures/token-benchmark-a0-reference.json`](../scripts/fixtures/token-benchmark-a0-reference.json) -(A0: fresh, recorded 2026-07-16 against `workflows@a1409d5b`). Stderr prints a +By default each run compares against the committed baseline +[`scripts/fixtures/token-benchmark-baseline.json`](../scripts/fixtures/token-benchmark-baseline.json) +(fresh mode, recorded 2026-08-17 against `workflows@34cd5429`). Stderr prints a compact scorecard; stdout JSON includes `vsReference` with absolute/percent deltas -and a **deliveryCostIndex** (A0 = 100, lower is better — sum of activity + workflow -+ resource + technique chars). +and a **deliveryCostIndex** (baseline = 100, lower is better — sum of activity + +workflow + resource + technique chars). + +#### The gate runs on every pull request + +The [Verify](../.github/workflows/verify.yml) workflow runs `--gate` at the 1% default +against the pinned corpus. No guard can measure this, because delivery cost is a +property of a walk rather than of a file — so until the job existed, delivery rose +31.3% in 32 days with nothing reporting it. + +**A definition change that adds delivery fails the gate, and that is the gate working.** +Pricing corpus growth at merge is the point. When the increase is wanted: + +1. Confirm it — a new activity or a widened contract legitimately costs characters. +2. Re-record the fixture from a `--no-compare` run on the same corpus commit, in the + same commit as the change. +3. Say in the fixture's `description` what the corpus gained for the characters. + +A fixture recorded against a different corpus makes ordinary authoring read as a +regression, which is how a gate stops being run at all. #### A persistent-only comparison is not a valid ship gate @@ -240,8 +258,8 @@ measurement of the reference-delivery win, never the gate. WORKFLOWS_DIR=/path/to/workflows npm run --silent bench:token -- \ --label=AFTER --context-mode=fresh --gate --max-regression-pct=1 -# Baseline (full redelivery) — expect deliveryCostIndex ≈ 100 -npm run --silent bench:token -- --label=A0 --context-mode=fresh +# Re-record the baseline (same corpus commit as the change that moved it) +npm run --silent bench:token -- --label=baseline --context-mode=fresh --no-compare # Supplementary: the reference-delivery win. Banner-warned as cross-mode, not a gate. npm run --silent bench:token -- --label=opt --context-mode=persistent @@ -375,6 +393,27 @@ npm run test:ci -- -u # re-baseline the walk npm run baseline:stamp # record the corpus commit it was baselined against ``` +The binding-fidelity triage carries the same coupling: its 69 verdicts are judgements about +definitions as they stood at `corpusSha`. The guard prints how far the corpus has moved since, +without failing on it — a verdict usually survives edits elsewhere, and an entry whose finding no +longer occurs is already reported by name as stale. + +### Sessions in flight + +A definition edit reaches the runs already walking that workflow. Their variable bags were seeded +from the declarations on disk when they opened, so a declaration added since is absent until they +resume: on resume the server seeds what the bag lacks and re-stamps the recorded version. What that +does *not* cover is a run part-way through an activity whose steps changed under it. + +Count them before landing: + +```bash +npm run sessions:census -- --workflow work-package --status running --list +``` + +Zero means the edit reaches nothing in flight. A non-zero count is the set of runs that will pick it +up, and the `--list` output names each one's folder, recorded version and current activity. + ## Branch Structure | Branch | Content | Purpose | diff --git a/docs/dispatch_model.md b/docs/dispatch_model.md index d8ab60385..a11ee0b43 100644 --- a/docs/dispatch_model.md +++ b/docs/dispatch_model.md @@ -88,21 +88,19 @@ The run pauses at every activity boundary, because the orchestrator owns the com The character budget carries a headroom fraction of its own because `BUNDLE_HEADROOM_FRACTION` answers a different question — how much of one activity's window may go to inlined step techniques — and at `0.80` the arithmetic admits thirteen of the main workflow's fifteen activities into one context. The activity cap covers what a character count is blind to: the establishment the server never delivers, the code the worker reads, the artifacts it drafts, and degradation across a long walk. -**Which limit binds depends on the workflow, and both cases are wanted.** The two rest on different evidence. `npm run bench:batch` measures activity payloads only — it never fetches a technique or resource lazily — so its 161,027 characters for the three-activity analysis run is the *eager floor*, not what a batch really accumulates. Read off 112 worker contexts in the sealed session records, one activity costs a median 74,109 characters once its lazy fetches are counted, with a 90th percentile of 182,642 and a maximum of 261,827. The lazy half is usually the larger one. +**Which limit binds depends on the workflow, and both cases are wanted.** The two rest on different evidence. `npm run bench:batch` measures activity payloads only — it never fetches a technique or resource lazily — so its figure for the three-activity analysis run is the *eager floor*, not what a batch really accumulates. The lazy half is usually the larger one. -At a 200,000-token window, giving a 280,000-character budget: +At a 200,000-token window, giving a 280,000-character budget, **the cap binds first on measured content**. The benchmark's three activities cost 159,093 characters batched — 78,128, then 58,588, then 22,377 — which is 57% of budget, because a batch's second and later activities collapse the invariant blocks and the ancestor contract their techniques share (see [Reference Delivery](resource_resolution_model.md#11-reference-delivery)). Standalone, the same three cost 232,954. Reaching the budget takes roughly seven activities of that weight, and a worker declaring a smaller window is bounded proportionally: the budget takes over below roughly 114,000 declared tokens on this workload. -- **On the main workflow the budget binds first** — two real runs reach it after two activities. That is the mechanism working: three heavy activities would put over half the declared window into workflow content before a line of code is read. -- **On the setup sequence the cap binds first**, its activities costing 33,000 to 154,000. That sequence is batching's first user, and a character budget alone would admit more of it than a context should hold. -- **A smaller declared window is bounded proportionally**, and where the third activity is refused depends on what the first two cost. On the median activity the budget binds before the cap below roughly 106,000 declared tokens; on the 90th percentile, below roughly 261,000 — so on heavy content the budget is the binding limit at any window worth declaring. The lighter run the benchmark walks puts the crossover near 99,000. +Admission is checked *before* a delivery rather than after, so the admitted activity can carry a batch past the budget by up to one heavy activity. Refusing after composing would pay the composition and still not un-deliver it. -Admission is checked *before* a delivery rather than after, so the admitted activity can carry a batch past the budget — by up to one heavy activity, 261,827 characters on measured content. Refusing after composing would pay the composition and still not un-deliver it. +**Revising either value needs evidence a byte count cannot supply.** The cap covers the context establishment the server never delivers, the code the worker reads and the artifacts it drafts — so `batch_refused` counts and per-activity usage rows over real runs are what a revision rests on, not a benchmark that only sees payloads. Both limits count each delivery once. An `activity_dispatched` size is the whole `get_activity` response, so the techniques and resources it bundled eagerly are already inside it and their own observability events are not added again; what counts on top is only what the worker went back for lazily. Counting the bundled entries twice inflated one activity of the main workflow by 48% and a run of three by 70%, which made a nominal 280,000-character budget bind at 164,540. `get_activity` reports where a context stands in `_meta.batch` (`activities`, `max_activities`, `delivered_chars`, `budget_chars`, `may_continue`), so the ordinary end of a batch is the worker stopping. Asking past the bound is refused with the payload undelivered and a `batch_refused` history event naming the limit — recorded once per scope, activity and limit, so the tally counts how often a limit bound rather than how often a worker retried. That tally is what the starting settings are revised from. -`may_continue` is answered as of that delivery, and the worker then fetches techniques and resources lazily while it runs the activity, drawing down the same budget. So a batch reported as having room can still be refused at the next boundary — the delivered and budget counts on the same response are what a reader compares to see how close it was. The refusal is an expected outcome rather than an error, and the orchestrator handles it by releasing the identity and dispatching a replacement — which must carry a **new** `agent_id`, since the bound is keyed on the identity and a fresh context under a used one would receive markers for content it does not hold. +`may_continue` on `get_activity` is answered as of that delivery, and the worker then fetches techniques and resources lazily while it runs the activity, drawing down the same budget. So a batch reported as having room can still be refused at the next boundary. `next_activity` answers the same question at the boundary instead — pass the exiting worker's `agent_id` and `context_tokens` and its `_meta.batch` counts those lazy fetches, which is the reading a continue-or-respawn decision wants. The refusal is an expected outcome rather than an error, and the orchestrator handles it by releasing the identity and dispatching a replacement — which must carry a **new** `agent_id`, since the bound is keyed on the identity and a fresh context under a used one would receive markers for content it does not hold. Three carve-outs keep the bound aimed at what it is for: diff --git a/docs/resource_resolution_model.md b/docs/resource_resolution_model.md index 4733216da..d2ecfea3f 100644 --- a/docs/resource_resolution_model.md +++ b/docs/resource_resolution_model.md @@ -164,6 +164,13 @@ Reference delivery belongs to the **agent context**, not to the session: A marker is only ever valid for the context that received the bytes it stands for. +Two things establish that a context holds a payload, and they are independent: + +- **Its own ledger.** A scope the server has already delivered an activity to is that same context arriving again — the orchestrator mints one `agent_id` per dispatch and reuses it verbatim for as long as that worker carries its batch. So the **invariant blocks** (the worker technique bundle, its `rules`, and the inherited `activity_rules`) collapse for a returning identity in *every* delivery mode, not only under `persistent`. A replacement worker arrives under a new `agent_id`, reads an empty ledger, and takes them in full. +- **The response itself.** A marker may instead point at a byte-identical copy earlier in the *same* response. That needs no ledger and no mode: the copy travels in the payload the marker travels in, so any recipient that can read the marker can read the copy. See [Blocks inside a technique](#blocks-inside-a-technique). + +Resuming with `context_mode: "fresh"` drops that scope's ledger entries, because the caller is stating this identity retains nothing it was sent. The next delivery to it is therefore full. `bundle: "full"` does the same for one call without touching the ledger. + ### Turning it on | How | Applies to | @@ -184,7 +191,7 @@ The orchestrator mints an `agent_id` per dispatch and reuses it verbatim for as ### What collapses, call by call -- **`get_activity`** — the response carries `bundle_mode: reference` and a `bundle_note`. Any bundled technique whose composed content is byte-identical to an earlier delivery collapses to a marker, as do the `rules` and `activity_rules` blocks. Techniques new to the activity, or whose content changed, arrive in full. The activity body itself is always delivered. +- **`get_activity`** — under reference delivery the response carries `bundle_mode: reference`. Any bundled technique whose composed content is byte-identical to an earlier delivery collapses to a marker, as do the `rules` and `activity_rules` blocks. Techniques new to the activity, or whose content changed, arrive in full. The activity body itself is always delivered. In the default mode the invariant blocks still collapse for a returning identity, and shared blocks still collapse within the response — so a `bundle_note` accompanies any response that carries a marker, in either mode, naming which referents that response can produce. - **`get_technique`** — a byte-identical refetch returns `delivery: unchanged` and a `content_hash` instead of the composed technique. Step-bound provenance annotations (`source:` / `destination:`) are part of that content. They are fixed for a given corpus and step, so refetching the same step collapses; fetching the same operation from a *different* step re-delivers in full rather than handing back a stale reference. - **`get_resource`** — a byte-identical refetch of the same `resource_id` returns `delivery: unchanged` and a `content_hash` instead of the body. The key is the caller's exact `resource_id`, anchor included, so `pr-description` and `pr-description#templates` occupy independent slots. - **`get_workflow`** — under `context_mode: "persistent"` the orchestrator ops bundle (everything above the `---` separator) is keyed under `workflow_bundle:`. On a resume where the agent already holds it, the whole bundle collapses to a single marker, while the workflow summary below the separator stays full. @@ -197,6 +204,10 @@ Collapsing can go finer than a whole technique. Techniques sharing a workflow co So when a technique is new to the context but one of its shared blocks already arrived with a sibling technique, that block becomes a marker in place while the technique-specific core arrives in full. This happens both on the `get_technique` full-delivery path and inside each eagerly inlined `get_activity` `step_techniques` entry. +Inside one `get_activity` response this pass runs **in every delivery mode**, because composition merges each ancestor group's rules into every technique that group covers: a response bundling ten techniques of one group would otherwise carry that group's rules ten times. The marker points at the sibling entry in the same payload, so it is readable by a worker holding nothing from before. Widening it to the ledger — collapsing against what arrived on an *earlier call* — is what reference delivery adds. + +Two steps bound to the same technique in one activity collapse the same way: the second entry is a marker naming the first. + Hashing the content is what keeps this from going stale: a block annotated with binding-seam provenance hashes differently, so it correctly arrives in full. ### Forcing full delivery @@ -211,11 +222,11 @@ Hashing the content is what keeps this from going stale: a block annotated with **Sizes are summable.** `technique_fetched`, `technique_bundled` and `resource_fetched` each carry `chars` — always the full payload size, on both paths — and `delivery: "full" | "unchanged"`. Characters delivered and characters saved are therefore both totals you can add up from the ledger. -**Benchmarks.** `npm run bench:token` compares delivery cost per session mode over a fixed `work-package` walk, against the frozen pre-optimisation A0 reference ([`scripts/run-token-benchmark.ts`](../scripts/run-token-benchmark.ts); `vsReference.deliveryCostIndex` reports A0 = 100, lower is better). `npm run bench:dispatch` measures the other axis — a fresh worker dispatch against the same worker resumed ([`scripts/run-dispatch-benchmark.ts`](../scripts/run-dispatch-benchmark.ts)). See [development.md](development.md#token-delivery-benchmark). +**Benchmarks.** `npm run bench:token` compares delivery cost per session mode over a fixed `work-package` walk, against the committed baseline ([`scripts/run-token-benchmark.ts`](../scripts/run-token-benchmark.ts); `vsReference.deliveryCostIndex` reports baseline = 100, lower is better). The Verify workflow gates on it at 1%, so a definition change that adds delivery is priced at merge — see [development.md](development.md#the-gate-runs-on-every-pull-request). `npm run bench:dispatch` measures the other axis — a fresh worker dispatch against the same worker resumed ([`scripts/run-dispatch-benchmark.ts`](../scripts/run-dispatch-benchmark.ts)). See [development.md](development.md#token-delivery-benchmark). ## 12. Hybrid Technique Bundling -`get_activity` inlines the composed content of an activity's small, ungated step techniques under a `step_techniques` map, so those steps run without a fetch round-trip. This is automatic and corpus-wide — there is no per-activity opt-in. What sizes the bundle is the worker's REQUIRED `context_tokens`. +`get_activity` inlines the composed content of an activity's small step techniques under a `step_techniques` map, so those steps run without a fetch round-trip. This is automatic and corpus-wide — there is no per-activity opt-in. What sizes the bundle is the worker's REQUIRED `context_tokens`. ### The budget @@ -227,15 +238,31 @@ context_tokens × headroomFraction × charsPerToken `headroomFraction` (default 0.80) and `charsPerToken` (default 4) are server config, overridable with `BUNDLE_HEADROOM_FRACTION` and `BUNDLE_CHARS_PER_TOKEN`. -That budget governs everything inlined eagerly. Technique bodies draw on it first, in document order; eagerly bundled resource bodies then draw on the same counter. Each loop stops at the first entry that would overflow what remains, and the rest stay lazy. Unchanged-reference markers cost almost nothing and never draw it down. +That budget governs everything inlined eagerly, so the worker technique bundle opens the tally at what it costs this response. Step technique bodies draw on the remainder, in document order; eagerly bundled resource bodies then draw on the same counter. Each loop stops at the first entry that would overflow what remains, and the rest stay lazy. Unchanged-reference markers cost almost nothing and never draw it down. + +`spent_chars` on the delivery cost line is that whole tally, against `eager_budget_chars`; `worker_bundle_chars` reports the invariant part on its own. This is how `context_tokens` comes to bound the eager bundle, which is the budget policy's stated purpose in [`src/config.ts`](../src/config.ts). ### Which steps get inlined -Each ungated technique-kind step, in document order, until the budget runs out. +Each technique-kind step whose gate answers **true**, in document order, until the budget runs out. A step with no gate answers true. + +The server holds the variable bag and the reference gate evaluators, so it can take that answer itself. A gate has an answer for the whole activity when every variable it compares is already bound *and* no step of this activity produces one of them. Otherwise it is **unanswered**, and the step stays lazy: + +| Gate reads | Answer | Delivery | +|---|---|---| +| variables bound before the activity opened, none of them written inside it | `true` | inlined — the worker certainly reaches this step | +| the same, evaluating false | `false` | lazy, and nothing is shipped for a step the run will not execute | +| a variable this activity produces | unanswered | lazy | +| a variable absent from the bag | unanswered | lazy — both evaluators return false for an unbound read, which is not the same as a negative one | +| an expression that does not parse | unanswered | lazy; the malformed expression is the corpus guards' business | + +An enclosing loop's gate narrows its body, so a step is inlined only where every gate above it also answers true. A body inlined under a gated loop is the protocol for **every** iteration — engage it once per pass from the copy already held, rather than re-fetching it each time. + +Whatever the executing agent evaluates when it reaches the step is still what decides execution. This answer decides only how the content travels. -A `when` or `condition` — on the step itself, or on a loop enclosing it — keeps that step lazy, so bundling never ships content for a step that might not run. +`lazy_gate_unanswered` and `lazy_gate_false` on the delivery cost line count the steps each reading left behind. An activity may also set `bundleTechniques: { maxChars: }`, a per-technique size cap layered on the budget: any single technique larger than `maxChars` stays lazy. `maxChars: 0` opts the activity out of eager bundling altogether. Anything not inlined stays a `get_technique { step_id }` fetch. diff --git a/package.json b/package.json index 72057c505..a35e1f447 100644 --- a/package.json +++ b/package.json @@ -47,6 +47,7 @@ "check:delta": "tsx scripts/check-delta.ts", "worktree:provision": "bash scripts/provision-worktree.sh", "baseline:stamp": "tsx scripts/stamp-corpus-baseline.ts", + "sessions:census": "tsx scripts/count-workflow-sessions.ts", "bench:token": "tsx scripts/run-token-benchmark.ts", "bench:dispatch": "tsx scripts/run-dispatch-benchmark.ts", "bench:batch": "tsx scripts/run-batch-benchmark.ts", diff --git a/schemas/activity.schema.json b/schemas/activity.schema.json index c0ea2697e..c838c5605 100644 --- a/schemas/activity.schema.json +++ b/schemas/activity.schema.json @@ -44,7 +44,7 @@ "maxChars" ], "additionalProperties": false, - "description": "Opt-in hybrid bundling: get_activity inlines each ungated step technique whose composed wire form is at most maxChars; larger and gated ones remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check." + "description": "Opt-in hybrid bundling: get_activity inlines each step technique whose composed wire form is at most maxChars and whose gate answers true at activity open; larger ones, and those whose gate has no answer yet, remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check." }, "steps": { "type": "array", diff --git a/schemas/workflow.schema.json b/schemas/workflow.schema.json index 2af677d26..4aa480f24 100644 --- a/schemas/workflow.schema.json +++ b/schemas/workflow.schema.json @@ -461,7 +461,7 @@ "maxChars" ], "additionalProperties": false, - "description": "Opt-in hybrid bundling: get_activity inlines each ungated step technique whose composed wire form is at most maxChars; larger and gated ones remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check." + "description": "Opt-in hybrid bundling: get_activity inlines each step technique whose composed wire form is at most maxChars and whose gate answers true at activity open; larger ones, and those whose gate has no answer yet, remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check." }, "steps": { "type": "array", diff --git a/scripts/binding-fidelity-triage.json b/scripts/binding-fidelity-triage.json index 75dc4e806..c94c03e5d 100644 --- a/scripts/binding-fidelity-triage.json +++ b/scripts/binding-fidelity-triage.json @@ -324,6 +324,13 @@ "verdict": "harmless", "rationale": "shared-op-caller-argument" }, + { + "check": "orphan-input", + "site": "work-package :: meta::workflow-engine::handle-sub-workflow", + "detail": "own input 'parent_session_index' has no producer in workflow 'work-package' (no step-binding entry, workflow variable, step output, or default)", + "verdict": "harmless", + "rationale": "shared-op-caller-argument" + }, { "check": "orphan-input", "site": "substrate-node-security-audit :: execute-sub-agent", diff --git a/scripts/check-binding-fidelity.ts b/scripts/check-binding-fidelity.ts index ce421b0a9..4bceacd38 100644 --- a/scripts/check-binding-fidelity.ts +++ b/scripts/check-binding-fidelity.ts @@ -60,6 +60,7 @@ import { injectCheckpointFragmentBodies, resolveCheckpointFragment } from '../sr import { fragmentsLookupSync } from './fragments-index.js'; import { assertScanned } from './workflows-root.js'; import { findingKey, report, requireRootOrExit, wantsJson, type Finding } from './guard-protocol.js'; +import { spawnSync } from 'node:child_process'; // Resolve paths from this file's own URL (reliable under both tsx CLI and the vitest runner, // where import.meta.dirname is not populated). @@ -747,6 +748,25 @@ export function loadTriage(): TriageFile { return JSON.parse(readFileSync(TRIAGE, 'utf-8')) as TriageFile; } +/** + * How far the corpus has moved since these verdicts were made, or null where that cannot be + * established. Report-only: see docs/development.md § Corpus-coupled baselines. + */ +export function triageStampNote(corpusSha: string, root: string = ROOT): string | null { + if (!corpusSha) return null; + const head = spawnSync('git', ['-C', root, 'rev-parse', 'HEAD'], { encoding: 'utf-8' }); + if (head.status !== 0) return null; + const current = head.stdout.trim(); + if (!current || current === corpusSha) return null; + const behind = spawnSync( + 'git', ['-C', root, 'rev-list', '--count', `${corpusSha}..${current}`], { encoding: 'utf-8' }, + ); + const commits = behind.status === 0 ? behind.stdout.trim() : ''; + const distance = commits && commits !== '0' ? ` — ${commits} corpus commit(s) since` : ''; + return `triage verdicts were made against corpus ${corpusSha.slice(0, 12)}, ` + + `the checkout is at ${current.slice(0, 12)}${distance}`; +} + export interface TriagedResult { findings: Finding[]; counts: Record; @@ -816,6 +836,8 @@ if (isMain) { process.stdout.write(`binding-fidelity: ${total} violation(s) — ${counts.harmless} harmless, ` + `${counts['fix-later']} fix-later, ${counts['live-bug']} live bug(s), ${counts.untriaged} untriaged` + `${counts.stale ? `, ${counts.stale} stale triage entr(ies)` : ''}\n`); + const stamp = triageStampNote(loadTriage().corpusSha); + if (stamp) process.stdout.write(`binding-fidelity: ${stamp}\n`); } report('binding-fidelity', findings, { okMessage: `no live or untriaged binding defects (${counts.harmless + counts['fix-later']} triaged as accepted debt)`, diff --git a/scripts/count-workflow-sessions.ts b/scripts/count-workflow-sessions.ts index 1373f4bc0..c77431280 100644 --- a/scripts/count-workflow-sessions.ts +++ b/scripts/count-workflow-sessions.ts @@ -12,8 +12,11 @@ * * Counts every state in the tree whose `workflowId` matches, optionally filtered by `status`. * + * Also the pre-landing count of runs a definition change reaches mid-flight — see + * docs/development.md § Sessions in flight. + * * Run: - * npx tsx scripts/count-workflow-sessions.ts --workflow workflow-design --status running --list + * npm run sessions:census -- --workflow work-package --status running --list * * Options: * --workflow workflow id to count (required) diff --git a/scripts/fixtures/token-benchmark-a0-reference.json b/scripts/fixtures/token-benchmark-a0-reference.json deleted file mode 100644 index c4241f8ce..000000000 --- a/scripts/fixtures/token-benchmark-a0-reference.json +++ /dev/null @@ -1,52 +0,0 @@ -{ - "label": "A0", - "description": "Pre-optimisation reference: work-package / skip-optional / robot + hot-resource probe, context_mode fresh. Recorded 2026-07-16 (server@7aaf7e2b, workflows@a1409d5b). See .engineering/artifacts/planning/2026-07-16-token-usage-reductions/20-token-benchmark.md.", - "contextMode": "fresh", - "workflowsRev": "a1409d5b", - "agentId": "bench-solo", - "path": [ - "start-work-package", - "design-philosophy", - "codebase-comprehension", - "plan-prepare", - "assumptions-review", - "implement", - "lean-coding-audit", - "post-impl-review", - "validate", - "strategic-review", - "submit-for-review", - "complete" - ], - "finalStatus": "completed", - "toolCalls": { - "start_session": 1, - "get_workflow": 1, - "next_activity": 12, - "get_activity": 12, - "get_technique": 26, - "get_resource": 128, - "yield_checkpoint": 12, - "respond_checkpoint": 12, - "resume_checkpoint": 12 - }, - "chars": { - "get_activity": 687936, - "get_workflow": 59455, - "get_resource": 448084, - "get_technique": 160057 - }, - "history": { - "technique_bundled": 62, - "technique_fetched": 26, - "resource_fetched": 95 - }, - "deliveredContentKeys": 73, - "resourceLedgerKeys": 0, - "unchangedResourceAnswers": 0, - "unchangedTechniqueAnswers": 0, - "getActivityChars": 687936, - "getWorkflowChars": 59455, - "getResourceChars": 448084, - "getTechniqueChars": 160057 -} diff --git a/scripts/fixtures/token-benchmark-baseline.json b/scripts/fixtures/token-benchmark-baseline.json new file mode 100644 index 000000000..1f93c89aa --- /dev/null +++ b/scripts/fixtures/token-benchmark-baseline.json @@ -0,0 +1,52 @@ +{ + "label": "baseline", + "description": "Gate baseline: work-package / skip-optional / robot + hot-resource probe, context_mode fresh. Recorded 2026-08-17 against workflows@cf4d0774. The July A0 reference recorded 1,355,532 delivery characters at workflows@a1409d5b; the same walk measured 1,780,292 on 2026-08-17 before the delivery corrections in .engineering/artifacts/planning/2026-08-17-meta-and-work-package-workflow-optimisation/EVALUATION-REPORT.md.", + "contextMode": "fresh", + "workflowsRev": "cf4d0774", + "agentId": "bench-solo", + "path": [ + "start-work-package", + "design-philosophy", + "codebase-comprehension", + "plan-prepare", + "assumptions-review", + "implement", + "lean-coding-audit", + "post-impl-review", + "validate", + "strategic-review", + "submit-for-review", + "complete" + ], + "finalStatus": "completed", + "toolCalls": { + "start_session": 1, + "get_workflow": 1, + "next_activity": 12, + "get_activity": 12, + "get_technique": 23, + "get_resource": 162, + "yield_checkpoint": 11, + "respond_checkpoint": 11, + "resume_checkpoint": 11 + }, + "chars": { + "get_activity": 518679, + "get_workflow": 108280, + "get_resource": 527683, + "get_technique": 142118 + }, + "history": { + "technique_bundled": 66, + "technique_fetched": 23, + "resource_fetched": 146 + }, + "deliveredContentKeys": 265, + "resourceLedgerKeys": 77, + "unchangedResourceAnswers": 0, + "unchangedTechniqueAnswers": 0, + "getActivityChars": 518679, + "getWorkflowChars": 108280, + "getResourceChars": 527683, + "getTechniqueChars": 142118 +} diff --git a/scripts/run-token-benchmark.ts b/scripts/run-token-benchmark.ts index 305800303..63aed9a83 100644 --- a/scripts/run-token-benchmark.ts +++ b/scripts/run-token-benchmark.ts @@ -6,16 +6,19 @@ * technique-linked resources plus a fixed hot-template set on every `get_activity` * (cross-activity resource repeat tax). Prints one JSON metrics object to stdout. * - * By default, stdout also includes `vsReference`: relative deltas against the frozen - * pre-optimisation A0 fixture (`scripts/fixtures/token-benchmark-a0-reference.json`). - * A compact scorecard is written to stderr. + * By default, stdout also includes `vsReference`: relative deltas against the committed + * baseline fixture (`scripts/fixtures/token-benchmark-baseline.json`). A compact + * scorecard is written to stderr. + * + * `--gate` is what the Verify workflow runs, at the 1% default. Re-recording it, and why the + * fixture must name the corpus commit under review: docs/development.md § Token delivery benchmark. * * Usage (from a server checkout with `node_modules` and a `workflows/` worktree): * - * npm run bench:token -- --label=A0 --context-mode=fresh + * npm run bench:token -- --label=check --context-mode=fresh --gate * npm run bench:token -- --label=opt --context-mode=persistent * WORKFLOWS_DIR=/path/to/workflows npm run bench:token -- \ - * --label=A3 --context-mode=persistent --server-root=$PWD + * --label=rerecord --context-mode=fresh --no-compare --server-root=$PWD * * Flags: * --workflow= Workflow to walk (default: work-package). Recorded in the output; a @@ -24,7 +27,7 @@ * --context-mode=fresh|persistent Forced on start_session (default: fresh) * --agent-id= Forced agent_id / ledger key (default: bench-solo) * --server-root= Server checkout root (default: cwd) - * --reference= A0 fixture path (default: /scripts/fixtures/…) + * --reference= Baseline fixture path (default: /scripts/fixtures/…) * --no-compare Skip vs-reference scorecard * --gate Fail (exit 3) on a delivery-char regression beyond the threshold * --max-regression-pct= Gate threshold in percent (default: 1) @@ -164,7 +167,7 @@ const HOT_RESOURCES = [ 'review-mode#review-type-selection', ] as const; -const DEFAULT_REFERENCE = 'scripts/fixtures/token-benchmark-a0-reference.json'; +const DEFAULT_REFERENCE = 'scripts/fixtures/token-benchmark-baseline.json'; /** Default `--gate` threshold: total delivery chars may not regress by more than this percent. */ const DEFAULT_MAX_REGRESSION_PCT = 1; @@ -291,7 +294,7 @@ function formatPct(deltaPct: number | null): string { function writeScorecard(vs: VsReference, contextMode: ContextMode, corpusNote?: string): void { const lines: string[] = [ '', - `vs ${vs.referenceLabel} (pre-optimisation reference) · ${contextMode}-mode · deliveryCostIndex ${vs.deliveryCostIndex.relative} (A0 = 100, lower is better)`, + `vs ${vs.referenceLabel} · ${contextMode}-mode · deliveryCostIndex ${vs.deliveryCostIndex.relative} (baseline = 100, lower is better)`, '─'.repeat(72), ]; if (!vs.modeMatched) lines.push(` ⚠ NOT A VALID GATE — ${vs.caveat}`, ''); diff --git a/site/api/schemas.html b/site/api/schemas.html index 0f6591df4..9e8087e93 100644 --- a/site/api/schemas.html +++ b/site/api/schemas.html @@ -86,7 +86,7 @@

activity.schema.json

namestringyesHuman-readable activity name descriptionstringnoDetailed description of the activity techniquesstring[]noActivity-wide technique references (:: paths); bundled into get_activity. - bundleTechniquesobjectnoOpt-in hybrid bundling: get_activity inlines each ungated step technique whose composed wire form is at most maxChars; larger and gated ones remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check. + bundleTechniquesobjectnoOpt-in hybrid bundling: get_activity inlines each step technique whose composed wire form is at most maxChars and whose gate answers true at activity open; larger ones, and those whose gate has no answer yet, remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check. steps(technique | action | checkpoint | loop)[]noOrdered, kind-tagged execution steps for this activity decisionsobject[]noConditional branching points; branch conditions are evaluated by the orchestrator, not the server. decisions[].idstringyes- @@ -289,7 +289,7 @@

workflow.schema.json

activities[].namestringyesHuman-readable activity name activities[].descriptionstringnoDetailed description of the activity activities[].techniquesstring[]noActivity-wide technique references (:: paths); bundled into get_activity. - activities[].bundleTechniquesobjectnoOpt-in hybrid bundling: get_activity inlines each ungated step technique whose composed wire form is at most maxChars; larger and gated ones remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check. + activities[].bundleTechniquesobjectnoOpt-in hybrid bundling: get_activity inlines each step technique whose composed wire form is at most maxChars and whose gate answers true at activity open; larger ones, and those whose gate has no answer yet, remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check. activities[].steps(technique | action | checkpoint | loop)[]noOrdered, kind-tagged execution steps for this activity activities[].decisionsobject[]noConditional branching points; branch conditions are evaluated by the orchestrator, not the server. activities[].decisions[].idstringyes- diff --git a/site/api/tools.html b/site/api/tools.html index cf53227e2..932465c94 100644 --- a/site/api/tools.html +++ b/site/api/tools.html @@ -252,6 +252,7 @@

next_activity

variables_changedobjectnoVariable assignments the completing activity produced — relay the worker's activity_complete variables_changed map verbatim. The server writes them into the session variable bag and records one variable_set history event per name, so the bag a later get_workflow_status / inspect_session returns reflects worker outputs and survives a lost agent context. Declared types are validated warn-only: a mismatch is stored as written and surfaced in _meta.validation. Omit when the activity changed nothing. artifacts_producedobject[]noArtifacts the completing activity produced: [{id, name, path?}]. Merged by id into the session declared-artifact accumulation; planning-folder reconciliation joins on id (warn-only). agent_idstringnoLabel for this agent in the session trace. + context_tokensintegernoYour worker context window in tokens. Required so the server can size inline technique bundling. @@ -394,6 +395,7 @@

get_technique

session_indexstringyesSix-character token from start_session. Use the same value for every call in this session. agent_idstringnoLabel for this agent in the session trace. step_idstringnoStep within the current activity. Omit to get the first technique for the activity or workflow. + activity_idstringnoActivity to move to. First call: use initialActivity from get_workflow. Later: use an id from transitions. bundle"reference" | "full"noreference: return unchanged markers for content already delivered. full: always return complete text. fullbooleannoForce full content even when persistent mode would return an unchanged marker (get_technique or get_resource). diff --git a/src/config.ts b/src/config.ts index 62ae98129..5b9dfaa07 100644 --- a/src/config.ts +++ b/src/config.ts @@ -156,46 +156,10 @@ export const DEFAULT_BUNDLE_HEADROOM_FRACTION = 0.8; export const DEFAULT_BUNDLE_CHARS_PER_TOKEN = 4; /** - * Batch bound policy (#407). One dispatched worker context walks a run of - * activities, and the run is bounded twice: by cumulative delivered characters, - * `context_tokens × batchHeadroomFraction × charsPerToken`, and by a hard cap on - * distinct activities. - * - * WHICH LIMIT BINDS DEPENDS ON THE WORKFLOW, and both cases are wanted. - * - * The two rest on different evidence. `npm run bench:batch` measures activity - * payloads only — it never fetches a technique or a resource lazily — so its - * 161,027 characters for the three-activity analysis run is the EAGER floor, not - * what a batch really accumulates. Read off 112 worker contexts in the sealed - * session records, one activity costs a median 74,109 characters once its lazy - * fetches are counted, with a 90th percentile of 182,642 and a maximum of 261,827. - * The lazy half is usually the larger one. - * - * So at a 200,000-token window, giving a 280,000-character budget: - * - * - On the main workflow, whose activities are heavy, the BUDGET binds first — two - * real runs reach it after two activities. That is the mechanism working: three - * heavy activities would put over half the declared window into workflow content - * before a line of code is read. - * - On the setup sequence, whose activities cost 33,000 to 154,000, the CAP binds - * first. That sequence is batching's first user, and a character budget alone - * would admit more of it than a context should hold. - * - A worker declaring a smaller window is bounded proportionally, and where the - * third activity is refused depends on what the first two cost. On the median - * activity the budget binds before the cap below roughly 106,000 declared tokens; - * on the 90th percentile, below roughly 261,000 — so on heavy content the budget - * is the binding limit at any window worth declaring. The lighter run the - * benchmark walks puts the crossover near 99,000. - * - * Admission is checked BEFORE a delivery rather than after, so the activity that - * is admitted can carry a batch past the budget — by up to one heavy activity, - * 261,827 characters on measured content. Refusing after composing would pay the - * composition and still not un-deliver it. - * - * The bundling fraction of 0.80 would admit thirteen of fifteen activities into - * one context. Both values are revised from `batch_refused` counts and - * per-activity usage rows over real runs, where the context establishment a byte - * count cannot see is finally visible. + * Batch bound policy (#407): a worker context is bounded by cumulative delivered + * characters, `context_tokens × batchHeadroomFraction × charsPerToken`, and by a cap + * on distinct activities. Which one binds, the measurements behind these two values, + * and what revising them needs: docs/dispatch_model.md § Batching a run of activities. */ export const DEFAULT_BATCH_HEADROOM_FRACTION = 0.35; export const DEFAULT_BATCH_MAX_ACTIVITIES = 3; diff --git a/src/loaders/technique-loader.ts b/src/loaders/technique-loader.ts index cdc9d6f41..7670b8bf7 100644 --- a/src/loaders/technique-loader.ts +++ b/src/loaders/technique-loader.ts @@ -98,6 +98,20 @@ export async function readTechnique( workflowDir: string, workflowId?: string, ): Promise> { + const found = await readTechniqueWithSource(techniqueId, workflowDir, workflowId); + return found.success ? ok(found.value.technique) : found; +} + +/** + * `readTechnique`, plus the workflow the file was found in — which differs from the `workflowId` + * asked for under a cross-workflow prefix or a meta fallback. This is the id a technique's own bare + * resource links qualify against. + */ +export async function readTechniqueWithSource( + techniqueId: string, + workflowDir: string, + workflowId?: string, +): Promise> { if (techniqueId.includes('/')) { const [targetWorkflow, actualSkillId] = techniqueId.split('/', 2); if (!targetWorkflow || !actualSkillId) { @@ -106,7 +120,7 @@ export async function readTechnique( const technique = await tryLoadSkillInWorkflow(workflowDir, targetWorkflow, actualSkillId); if (technique) { logInfo('Technique loaded (explicit prefix)', { id: techniqueId, targetWorkflow }); - return ok(technique); + return ok({ technique, sourceWorkflowId: targetWorkflow }); } return err(new TechniqueNotFoundError(techniqueId)); } @@ -128,7 +142,7 @@ export async function readTechnique( : await tryLoadNestedTechnique(getWorkflowTechniquesDir(workflowDir, targetWorkflow), rest[0]!, rest.slice(1).join('/')); if (t) { logInfo('Technique loaded (cross-workflow ::)', { id: techniqueId, targetWorkflow }); - return ok(t); + return ok({ technique: t, sourceWorkflowId: targetWorkflow }); } } catch (error) { if (!(error instanceof MarkdownTechniqueParseError)) throw error; @@ -145,7 +159,7 @@ export async function readTechnique( const t = await tryLoadNestedTechnique(getWorkflowTechniquesDir(workflowDir, wf), group, opPath); if (t) { logInfo('Technique loaded (nested)', { id: techniqueId, workflowId: wf }); - return ok(t); + return ok({ technique: t, sourceWorkflowId: wf }); } } catch (error) { if (!(error instanceof MarkdownTechniqueParseError)) throw error; @@ -159,7 +173,7 @@ export async function readTechnique( const local = await tryLoadSkillInWorkflow(workflowDir, workflowId, techniqueId); if (local) { logInfo('Technique loaded (workflow-local)', { id: techniqueId, workflowId }); - return ok(local); + return ok({ technique: local, sourceWorkflowId: workflowId }); } } @@ -168,7 +182,7 @@ export async function readTechnique( const shared = await tryLoadSkillInWorkflow(workflowDir, META_WORKFLOW_ID, techniqueId); if (shared) { logInfo('Technique loaded (meta shared layer)', { id: techniqueId, workflowId: workflowId ?? '(none)' }); - return ok(shared); + return ok({ technique: shared, sourceWorkflowId: META_WORKFLOW_ID }); } } @@ -577,7 +591,17 @@ export async function composeTechnique( workflowDir: string, workflowId: string, ): Promise> { - const base = await readTechnique(techniqueId, workflowDir, workflowId); + const composed = await composeTechniqueWithSource(techniqueId, workflowDir, workflowId); + return composed.success ? ok(composed.value.technique) : composed; +} + +/** `composeTechnique`, plus the workflow the technique file was found in (`readTechniqueWithSource`). */ +export async function composeTechniqueWithSource( + techniqueId: string, + workflowDir: string, + workflowId: string, +): Promise> { + const base = await readTechniqueWithSource(techniqueId, workflowDir, workflowId); if (!base.success) return base; // Derive path segments within the workflow's techniques directory. @@ -586,7 +610,10 @@ export async function composeTechnique( const pathSegments = rawId.split('::').filter(s => s.length > 0); const techniquesDir = getWorkflowTechniquesDir(workflowDir, workflowId); - return ok(await composeLoaded(base.value, pathSegments, techniquesDir)); + return ok({ + technique: await composeLoaded(base.value.technique, pathSegments, techniquesDir), + sourceWorkflowId: base.value.sourceWorkflowId, + }); } /** @@ -596,7 +623,8 @@ export async function composeTechnique( * and an op that shares its group's name resolves to the op, not the group base. Foreign/ * cross-group refs are written qualified and resolve as-authored; if no group named after the * activity holds the op, the bare ref falls back to as-authored. Returns the RESOLVED id - * alongside the composition — the id the delivery ledger and fidelity events are keyed by. + * alongside the composition — the id the delivery ledger and fidelity events are keyed by — and the + * workflow the technique file was found in, which its own bare resource links resolve against. * The single resolution implementation behind step-bound get_technique and get_activity's * hybrid step-technique bundling, so both deliver identical composition by construction. */ @@ -605,16 +633,16 @@ export async function composeActivityTechnique( workflowDir: string, workflowId: string, activityId?: string, -): Promise> { +): Promise> { if (!techniqueRef.includes('::') && activityId) { - const viaGroup = await composeTechnique(`${activityId}::${techniqueRef}`, workflowDir, workflowId); + const viaGroup = await composeTechniqueWithSource(`${activityId}::${techniqueRef}`, workflowDir, workflowId); if (viaGroup.success) { - return ok({ techniqueId: `${activityId}::${techniqueRef}`, technique: viaGroup.value }); + return ok({ techniqueId: `${activityId}::${techniqueRef}`, ...viaGroup.value }); } } - const composed = await composeTechnique(techniqueRef, workflowDir, workflowId); + const composed = await composeTechniqueWithSource(techniqueRef, workflowDir, workflowId); if (!composed.success) return composed; - return ok({ techniqueId: techniqueRef, technique: composed.value }); + return ok({ techniqueId: techniqueRef, ...composed.value }); } /** diff --git a/src/schema/activity.schema.ts b/src/schema/activity.schema.ts index 78059a12b..89a8e1df1 100644 --- a/src/schema/activity.schema.ts +++ b/src/schema/activity.schema.ts @@ -11,11 +11,11 @@ export type TechniquesReference = z.infer; // Hybrid technique bundling (#189 C1c) — optional per-activity override on the automatic, // context-derived eager bundling get_activity performs for EVERY activity. get_activity inlines -// the composed content of the activity's small, ungated step-bound techniques under a -// `step_techniques` map, sized to a cumulative budget derived from the caller's `context_tokens`. -// `maxChars` is an explicit per-technique size cap layered on that budget; `maxChars: 0` opts the -// activity out of eager bundling entirely. Gated steps (a `when`/`condition` on the step or an -// enclosing loop) always stay lazy via get_technique. +// the composed content of the activity's small step-bound techniques under a `step_techniques` map, +// sized to a cumulative budget derived from the caller's `context_tokens`. `maxChars` is an explicit +// per-technique size cap layered on that budget; `maxChars: 0` opts the activity out of eager +// bundling entirely. A step whose gate has no fixed answer at delivery time — it reads a variable +// this activity produces, or one absent from the bag — stays lazy via get_technique. export const BundleTechniquesSchema = z.object({ maxChars: z.number().int().nonnegative().describe('Per-technique character cap layered on the server-derived per-activity eager-delivery budget: a step technique whose composed wire form exceeds this is not inlined and is fetched with get_technique { step_id }. Set to 0 to opt this activity out of eager step-technique bundling entirely.'), }).strict(); @@ -284,7 +284,7 @@ export const ActivitySchema = z.object({ techniques: TechniquesReferenceSchema.optional(), // Opt-in hybrid bundling of step-bound techniques into get_activity (#166 B11). - bundleTechniques: BundleTechniquesSchema.optional().describe('Opt-in hybrid bundling: get_activity inlines each ungated step technique whose composed wire form is at most maxChars; larger and gated ones remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check.'), + bundleTechniques: BundleTechniquesSchema.optional().describe('Opt-in hybrid bundling: get_activity inlines each step technique whose composed wire form is at most maxChars and whose gate answers true at activity open; larger ones, and those whose gate has no answer yet, remain lazy-fetched via get_technique. Bundled deliveries are recorded as technique_bundled history events and satisfy the manifest fidelity check.'), // Execution — the single ordered list of kind-tagged steps (technique | action | checkpoint | loop). // Checkpoints are inline kind:checkpoint steps and loops are compound kind:loop steps: there are no diff --git a/src/tools/resource-tools.ts b/src/tools/resource-tools.ts index d0252e7e5..3b2ad368e 100644 --- a/src/tools/resource-tools.ts +++ b/src/tools/resource-tools.ts @@ -274,14 +274,43 @@ export function registerResourceTools(server: McpServer, config: ServerConfig): // A resume carries a fresh request from the user — rebind it so the bag // describes why the session is running now, not why it opened. const requestDrift = user_request !== undefined && state.variables?.['user_request'] !== user_request; + // Definition drift: declarations added since this session opened are absent from its bag, so + // seed the ones it lacks. A value already present is a decision, not a stale default. + const versionDrift = effectiveWorkflowVersion !== '' + && state.workflowVersion !== effectiveWorkflowVersion; + const lateSeed = versionDrift && wfPreLoad.success + ? Object.fromEntries( + Object.entries(seedDefaults(wfPreLoad.value.variables)) + .filter(([name]) => state.variables?.[name] === undefined), + ) + : {}; + const lateSeedNames = Object.keys(lateSeed); + // A fresh context retains nothing it was sent, so its ledger no longer describes it. + const disownsPriorDeliveries = modeDrift && context_mode === 'fresh'; let nextState = state; - if (pathDrift || agentDrift || modeDrift || requestDrift) { + if (pathDrift || agentDrift || modeDrift || requestDrift || versionDrift) { nextState = { ...nextState, ...(agentDrift ? { agentId: agent_id } : {}), ...(pathDrift ? { planningFolderPath: canonicalFolder } : {}), ...(modeDrift ? { contextMode: context_mode } : {}), - ...(requestDrift ? { variables: { ...nextState.variables, user_request } } : {}), + ...(disownsPriorDeliveries && state.deliveredContent + ? { + deliveredContent: Object.fromEntries( + Object.entries(state.deliveredContent).filter(([s]) => s !== agent_id), + ), + } + : {}), + ...(versionDrift ? { workflowVersion: effectiveWorkflowVersion } : {}), + ...(requestDrift || lateSeedNames.length + ? { + variables: { + ...lateSeed, + ...nextState.variables, + ...(requestDrift ? { user_request } : {}), + }, + } + : {}), }; } const repoBindRaw = repo?.trim() || sessionRoot.repo; @@ -608,10 +637,11 @@ export function registerResourceTools(server: McpServer, config: ServerConfig): ...sessionIndexParam, ...agentIdParam, step_id: z.string().optional().describe('Optional. Step id whose bound technique to load; omit for the activity/workflow first technique.'), + activity_id: z.string().optional().describe('Optional. The activity you were dispatched for. A step id resolves against the session\'s CURRENT activity, so passing this turns a pointer that has moved on into an error instead of a technique from the wrong activity.'), bundle: z.enum(['reference', 'full']).optional().describe('Optional. "reference" collapses a refetch already delivered to THIS agent_id scope. "full" forces full delivery. Defaults from context_mode.'), full: z.boolean().optional().describe('Optional. Force full content when reference delivery would return an unchanged-reference (e.g. after summarization). Overrides bundle.'), }, - withAuditLog('get_technique', withSessionStoreErrors(async ({ session_index, agent_id, step_id, bundle, full }) => { + withAuditLog('get_technique', withSessionStoreErrors(async ({ session_index, agent_id, step_id, activity_id, bundle, full }) => { const loadOpts = await sessionLoadOpts(); const loaded = await loadSessionForTool(planningRootDir, session_index, loadOpts); const { state } = loaded; @@ -620,6 +650,16 @@ export function registerResourceTools(server: McpServer, config: ServerConfig): assertNoActiveCheckpoint(state); + // A step id resolves against the session pointer, which any context in the session can move. + if (activity_id !== undefined && state.currentActivity !== activity_id) { + throw new Error( + `get_technique: this session's current activity is '${state.currentActivity ?? '(none)'}', ` + + `not the '${activity_id}' you were dispatched for. A step id resolves against the current ` + + 'activity, so fetching now would return another activity\'s technique. Report the mismatch ' + + 'to your orchestrator rather than retrying without activity_id.', + ); + } + const wfDiag = await loadWorkflowWithDiagnostics(config.workflowDir, workflow_id); if (!wfDiag.success) throw wfDiag.error; const wfResult = { success: true as const, value: wfDiag.value.workflow }; diff --git a/src/tools/workflow-tools.ts b/src/tools/workflow-tools.ts index 9a0a2e58b..7bd2bf95e 100644 --- a/src/tools/workflow-tools.ts +++ b/src/tools/workflow-tools.ts @@ -15,6 +15,7 @@ import { CORE_ORCHESTRATOR_TECHNIQUES, CORE_WORKER_TECHNIQUES } from '../loaders import { readResourceRaw } from '../loaders/resource-loader.js'; import { injectResolvedStepIds, techniqueName, flattenActivitySteps, type Activity, type Step } from '../schema/activity.schema.js'; import { buildProducerIndex, provenanceContextFor, decorateTechniqueProvenance } from '../utils/binding-provenance.js'; +import { bothGates, gateAnswer, variablesWrittenIn } from '../utils/gate-liveness.js'; import { withAuditLog, logInfo, logWarn } from '../logging.js'; import { applyVariableWrites } from '../utils/variable-seed.js'; import { stringifyForResponse } from '../utils/serialization.js'; @@ -525,8 +526,11 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): agent_id: z.string().min(1).optional().describe( 'Optional. Worker context that completed the exiting activity — scopes technique-fetch fidelity and step_completed attribution.', ), + context_tokens: z.number().int().positive().optional().describe( + 'Optional, with agent_id. That context\'s declared window, so `_meta.batch` reports whether it may take this activity — a reading that counts the lazy fetches the exiting activity made. Continue the worker on `may_continue: true`; spawn a fresh agent_id otherwise.', + ), }, - withAuditLog('next_activity', withSessionStoreErrors(async ({ session_index, activity_id, transition_condition, step_manifest, activity_manifest, variables_changed, artifacts_produced, agent_id }) => { + withAuditLog('next_activity', withSessionStoreErrors(async ({ session_index, activity_id, transition_condition, step_manifest, activity_manifest, variables_changed, artifacts_produced, agent_id, context_tokens }) => { const loadOpts = await sessionLoadOpts(); const loaded = await loadSessionForTool(planningRootDir, session_index, loadOpts); const { state } = loaded; @@ -735,6 +739,24 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): const meta: Record = { session_index, validation }; + // Where the exiting worker stands, read at the boundary so the lazy fetches of the activity it + // just finished are counted (docs/dispatch_model.md § Batching a run of activities). + if (agent_id && context_tokens !== undefined && !isTerminal) { + const bound = batchBound(context_tokens, { + headroomFraction: config.batchHeadroomFraction ?? DEFAULT_BATCH_HEADROOM_FRACTION, + maxActivities: config.batchMaxActivities ?? DEFAULT_BATCH_MAX_ACTIVITIES, + charsPerToken: config.bundleCharsPerToken ?? DEFAULT_BUNDLE_CHARS_PER_TOKEN, + }); + const stand = batchState(state, agent_id, bound); + meta['batch'] = { + activities: stand.activities.length, + max_activities: bound.maxActivities, + delivered_chars: stand.chars, + budget_chars: bound.budgetChars, + may_continue: batchRefusal(state, agent_id, activity_id, bound) === undefined, + }; + } + if (config.traceStore) { const segment = config.traceStore.getSegmentAndAdvanceCursor(state.sessionIndex); if (segment.events.length > 0) { @@ -850,6 +872,10 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): // content that was delivered under the default full mode. const newDeliveries: Record = {}; + // Whether this scope's ledger describes what it is holding. Governs the blocks identical on + // every activity — see docs/resource_resolution_model.md § Reference Delivery. + const mayReferBack = bundle !== 'full' && (referenceMode || hasDispatch(state, scope)); + // Bundle the techniques the activity references (delivered as full protocols), deduped with // the workflow-level techniques inherited by every activity (`techniques.activity`, injected // here so a common technique is declared once on the workflow) and the core worker techniques. @@ -868,7 +894,7 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): for (const [key, body] of Object.entries(bundleTechniques)) { const hash = contentHash(stringifyForResponse(body)); const ledgerKey = `bundle:${key}`; - if (referenceMode && deliveredHash(state, ledgerKey, scope) === hash) { + if (mayReferBack && deliveredHash(state, ledgerKey, scope) === hash) { bundleTechniques[key] = unchangedMarker(hash); } else { newDeliveries[ledgerKey] = hash; @@ -882,13 +908,16 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): if (bundleData['rules'] !== undefined) { const rulesHash = contentHash(stringifyForResponse(bundleData['rules'])); const rulesKey = `bundle:rules:${rulesHash}`; - if (referenceMode && deliveredHash(state, rulesKey, scope) === rulesHash) { + if (mayReferBack && deliveredHash(state, rulesKey, scope) === rulesHash) { bundleData['rules'] = unchangedMarker(rulesHash); } else { newDeliveries[rulesKey] = rulesHash; } } + // What the worker bundle costs this response, markers included: it opens the eager tally below. + const workerBundleChars = stringifyForResponse(bundleData).length; + // Automatic, per-agent context-derived step-technique bundling (#189 C1c): every activity // eagerly inlines its small, ungated step-bound techniques — no per-activity opt-in. The // eager-delivery budget is a CUMULATIVE per-activity character budget derived from the @@ -904,6 +933,8 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): // identical by construction. Bundled entries share the `technique:` delivery- // ledger key with get_technique, so persistent-context refetches of bundled content collapse // to unchanged-references in either direction. + // A gated step joins the bundle when its gate answers true for the whole activity; a false or + // unanswered gate stays lazy (src/utils/gate-liveness.ts). const bundledStepTechniques: Record = {}; // `chars` is the FULL composed size on both paths and `delivery` says which path ran, so the // technique_bundled / resource_fetched events below report delivered and saved characters @@ -933,25 +964,46 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): // Running total of full-content characters committed to the eager bundle. An unchanged-reference // marker costs effectively nothing, so it never draws down the budget; only full-content // entries do. Held here so the delivery's cost line can report it against the budget. - let spentChars = 0; + let spentChars = workerBundleChars; + /** Technique steps left for get_technique, by the answer their gate gave. */ + let lazyUnansweredGates = 0; + let lazyFalseGates = 0; if (!optedOut && result.success && activity) { - const eligible: Array = []; - const collectUngated = (steps: Step[] | undefined): void => { - for (const s of steps ?? []) { - if (s.when !== undefined || s.condition !== undefined) continue; - if (s.kind === 'loop') { collectUngated(s.steps as Step[]); continue; } - if (s.kind === 'technique' && s.id) eligible.push(s); - } - }; - collectUngated((activity as Activity).steps); - - if (eligible.length > 0) { + // One pass serves both the gate reading (which bag entries this activity produces) and the + // provenance decoration further down. + const bindsTechnique = flattenActivitySteps(activity as Activity) + .some((s) => s.kind === 'technique' && s.id !== undefined); + if (bindsTechnique) { producerIndex = await buildProducerIndex({ workflow: result.value, workflowDir: config.workflowDir, activitySourceWorkflow, }); } + const writtenInActivity = producerIndex + ? variablesWrittenIn(producerIndex.producers, activity_id) + : new Set(); + const bagAtOpen = state.variables ?? {}; + + const eligible: Array = []; + // An enclosing loop's gate narrows its body, so a step joins only where every gate above it + // also answers true. + const collect = (steps: Step[] | undefined, outer: boolean | undefined): void => { + for (const s of steps ?? []) { + const answer = bothGates(outer, gateAnswer({ + when: s.when, + condition: s.condition, + variables: bagAtOpen, + writtenInActivity, + })); + if (s.kind === 'loop') { collect(s.steps as Step[], answer); continue; } + if (s.kind !== 'technique' || s.id === undefined) continue; + if (answer === true) eligible.push(s); + else if (answer === false) lazyFalseGates++; + else lazyUnansweredGates++; + } + }; + collect((activity as Activity).steps, true); for (const step of eligible) { const ref = techniqueName(step.technique); @@ -962,7 +1014,7 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): // An unresolvable ref is the binding guard's business; delivery skips it (the step's // own get_technique fetch will surface the error to the worker). if (!composedStep.success) continue; - const { techniqueId } = composedStep.value; + const { techniqueId, sourceWorkflowId: techniqueWorkflowId } = composedStep.value; let technique = composedStep.value.technique; let provenanceWarnings: string[] = []; const ctx = producerIndex @@ -982,7 +1034,11 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): if (text.length > perTechniqueCap) continue; const ledgerKey = `technique:${techniqueId}`; const hash = contentHash(text); - const alreadyDelivered = referenceMode && (deliveredHash(state, ledgerKey, scope) === hash || newDeliveries[ledgerKey] === hash); + // The ledger half needs a context that retains prior payloads; the response-local half is + // readable by any recipient, the copy being in this same response. + const heldByContext = referenceMode && deliveredHash(state, ledgerKey, scope) === hash; + const earlierInResponse = newDeliveries[ledgerKey] === hash; + const alreadyDelivered = heldByContext || earlierInResponse; // ▼ STEP arrival marker: each entry is a discrete, self-describing unit that substitutes // for the intentional get_technique { step_id } call inlining removes (#189 C1c(C)1). const stepMarker = `▼ STEP ${step.id!} · technique ${techniqueId}`; @@ -1000,18 +1056,20 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): newDeliveries[ledgerKey] = hash; // The arrival marker leads the block; the composed technique fields follow at the same // level, so a bundled entry reads exactly like a get_technique fetch with a step header. - // Under reference mode, collapse any shared contract/rules block already delivered (by a - // sibling bundled step or an earlier fetch) to a marker while the core stays full. - const projected = referenceMode - ? dedupTechniqueBlocks(projectTechnique(technique), state, newDeliveries, scope) - : projectTechnique(technique); + // Shared contract/rules blocks collapse in every mode; reference mode widens the pass to + // what this context received on an earlier call. + const projected = dedupTechniqueBlocks( + projectTechnique(technique), state, newDeliveries, scope, referenceMode, + ); bundledStepTechniques[step.id!] = { marker: stepMarker, ...projected }; } // Collect linked resource ids from the full composed technique text even when this // delivery collapses to an unchanged-marker (markers omit link text). Only for steps // actually included in the bundle (after the budget break above). + // Qualified against the workflow the TECHNIQUE file was found in, not the activity's: a + // bare link in a meta technique names a resource under meta/resources/ whoever binds it. for (const rawId of extractResourceIds(text)) { - linkedResourceIds.add(qualifyResourceId(rawId, sourceWorkflowId, workflow_id)); + linkedResourceIds.add(qualifyResourceId(rawId, techniqueWorkflowId, workflow_id)); } bundledSteps.push({ stepId: step.id!, techniqueId, @@ -1124,7 +1182,7 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): bundleData['step_techniques'] = bundledStepTechniques; bundleData['step_techniques_note'] = - 'Each step_techniques entry is a discrete ▼ STEP block whose composed technique is identical to a get_technique { step_id } fetch. Engage the inlined steps strictly in step order: on reaching each step, EMIT a one-line "▶ step " begin-beat before executing it — that deliberate beat is the intentional act inlining moves off the get_technique call, and it is the stepwise observability trace for bundled steps (do NOT ping the server per bundled step; delivery-time technique_bundled events already record coverage). Resource bodies are NEVER nested inside a step_techniques entry — `resources_note` states how this response delivers the technique-linked resources. Technique steps absent from this map (gated, or past the derived eager-delivery budget / a per-activity size cap) still require get_technique { step_id } before execution.'; + 'Each step_techniques entry is a discrete ▼ STEP block whose composed technique is identical to a get_technique { step_id } fetch. Engage the inlined steps strictly in step order: on reaching each step, EMIT a one-line "▶ step " begin-beat before executing it — that deliberate beat is the intentional act inlining moves off the get_technique call, and it is the stepwise observability trace for bundled steps (do NOT ping the server per bundled step; delivery-time technique_bundled events already record coverage). Resource bodies are NEVER nested inside a step_techniques entry — `resources_note` states how this response delivers the technique-linked resources. An entry for a step inside a loop body is the protocol for EVERY iteration: engage it once per iteration from the copy you hold, and do not re-fetch it per pass. Technique steps absent from this map (a gate whose reading is not available at delivery time, a gate this activity reads as no, or past the derived eager-delivery budget / a per-activity size cap) still require get_technique { step_id } before execution.'; if (bundledResourceDeliveries.length > 0) bundleData['resources'] = bundledResources; if (resourceRefIds.length > 0) bundleData['resource_refs'] = resourceRefIds; if (linkedIds.length > 0) { @@ -1140,10 +1198,17 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): ...bundlingWarnings, ); - const opsData = referenceMode + // The note names whichever referents this response can actually produce. + const inResponseNote = 'A marker may point at a byte-identical copy EARLIER IN THIS RESPONSE — a sibling step_techniques entry, or one shared inherited_inputs/inherited_outputs/rules block several of this activity\'s techniques inherit from the same ancestor group. Find that copy by its content_hash and read it there.'; + const priorCallNote = 'A marker may point at content already in your context from an earlier call to this session — reuse it from there. Re-fetch one technique with get_technique { step_id, full: true }, or the whole payload with get_activity { bundle: "full" }.'; + const markerNotes = [ + ...(bundledSteps.length > 1 ? [inResponseNote] : []), + ...(mayReferBack ? [priorCallNote] : []), + ]; + const opsData = markerNotes.length ? { - bundle_mode: 'reference', - bundle_note: 'Entries marked { delivery: "unchanged", content_hash } are already in your context — reuse them. A marker may also replace a single inherited_inputs/inherited_outputs/rules block inside an otherwise-full step_techniques entry (that shared block came from a sibling technique). Re-fetch a technique with get_technique { step_id, full: true }, or get_activity { bundle: "full" } to re-deliver the whole bundle.', + ...(referenceMode ? { bundle_mode: 'reference' } : {}), + bundle_note: `Entries marked { delivery: "unchanged", content_hash } are content you already hold. ${markerNotes.join(' ')}`, ...bundleData, } : bundleData; @@ -1198,7 +1263,7 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): if (inheritedRules.length) { const inheritedRulesHash = contentHash(stringifyForResponse(inheritedRules)); const inheritedRulesKey = `activity_rules:${inheritedRulesHash}`; - if (referenceMode && deliveredHash(state, inheritedRulesKey, scope) === inheritedRulesHash) { + if (mayReferBack && deliveredHash(state, inheritedRulesKey, scope) === inheritedRulesHash) { activityRulesBlock = `${stringifyForResponse({ activity_rules: unchangedMarker(inheritedRulesHash) })}\n\n`; } else { newDeliveries[inheritedRulesKey] = inheritedRulesHash; @@ -1286,6 +1351,9 @@ export function registerWorkflowTools(server: McpServer, config: ServerConfig): bundled_steps: bundledSteps.length, bundled_steps_collapsed: bundledSteps.filter((b) => b.delivery === 'unchanged').length, bundled_resources: bundledResourceDeliveries.length, + worker_bundle_chars: workerBundleChars, + lazy_gate_unanswered: lazyUnansweredGates, + lazy_gate_false: lazyFalseGates, spent_chars: spentChars, eager_budget_chars: Math.floor(eagerBudgetChars), response_chars: responseText.length, diff --git a/src/utils/delivery.ts b/src/utils/delivery.ts index 5873529ad..45b405097 100644 --- a/src/utils/delivery.ts +++ b/src/utils/delivery.ts @@ -34,6 +34,9 @@ import { stringifyForResponse } from './serialization.js'; * * Full content is always recoverable: `get_activity { bundle: 'full' }`, * `get_technique { full: true }`, `get_resource { full: true }`. + * + * A response-local marker points at an earlier entry of the same response and needs no ledger; see + * docs/resource_resolution_model.md § Reference Delivery. */ /** Hash used for delivery-ledger comparison: sha256, truncated for payload brevity. */ @@ -101,6 +104,7 @@ const INHERITED_SPLIT_BLOCKS = ['inherited_inputs', 'inherited_outputs'] as cons * Content-key a field: collapse to an unchanged-marker when already delivered, * otherwise stage the hash. When `assignFull` is true, also write the full value * on first delivery (top-level blocks); nested note/items keep the spread value. + * `ledgerLookup` false restricts the comparison to this response's own entries. */ function stageField( out: Record, @@ -111,10 +115,12 @@ function stageField( scope: string, keyPrefix: string, assignFull = false, + ledgerLookup = true, ): void { const hash = contentHash(stringifyForResponse({ [field]: value })); const key = `${keyPrefix}:${hash}`; - if (deliveredHash(state, key, scope) === hash || newDeliveries[key] === hash) { + const heldByContext = ledgerLookup && deliveredHash(state, key, scope) === hash; + if (heldByContext || newDeliveries[key] === hash) { out[field] = unchangedMarker(hash); } else { newDeliveries[key] = hash; @@ -134,17 +140,20 @@ function stageField( * @param state session, for the delivery-ledger lookup. * @param newDeliveries accumulator of block-hashes to record. * @param scope delivery scope to look up (default: the session's agent). + * @param ledgerLookup whether the caller's context retains what it was sent. False keeps the pass + * response-local — a block collapses only against an earlier entry of the same response. */ export function dedupTechniqueBlocks( projected: Record, state: SessionFile, newDeliveries: Record, scope: string = state.agentId, + ledgerLookup = true, ): Record { const out = { ...projected }; if (out['provenance_note'] !== undefined) { - stageField(out, 'provenance_note', out['provenance_note'], state, newDeliveries, scope, 'technique:provenance_note', true); + stageField(out, 'provenance_note', out['provenance_note'], state, newDeliveries, scope, 'technique:provenance_note', true, ledgerLookup); } for (const block of INHERITED_SPLIT_BLOCKS) { @@ -154,28 +163,29 @@ export function dedupTechniqueBlocks( const rec = value as Record; const next: Record = { ...rec }; if (rec['note'] !== undefined) { - stageField(next, 'note', rec['note'], state, newDeliveries, scope, `technique:${block}.note`); + stageField(next, 'note', rec['note'], state, newDeliveries, scope, `technique:${block}.note`, false, ledgerLookup); } if (rec['items'] !== undefined) { - stageField(next, 'items', rec['items'], state, newDeliveries, scope, `technique:${block}.items`); + stageField(next, 'items', rec['items'], state, newDeliveries, scope, `technique:${block}.items`, false, ledgerLookup); } // Whole-block key still recorded when both halves are full (first delivery), so a // reader that only understands whole-block markers keeps working. const wholeHash = contentHash(stringifyForResponse({ [block]: value })); const wholeKey = `technique:${block}:${wholeHash}`; - if (deliveredHash(state, wholeKey, scope) === wholeHash || newDeliveries[wholeKey] === wholeHash) { + const wholeHeld = ledgerLookup && deliveredHash(state, wholeKey, scope) === wholeHash; + if (wholeHeld || newDeliveries[wholeKey] === wholeHash) { out[block] = unchangedMarker(wholeHash); } else { newDeliveries[wholeKey] = wholeHash; out[block] = next; } } else { - stageField(out, block, value, state, newDeliveries, scope, `technique:${block}`, true); + stageField(out, block, value, state, newDeliveries, scope, `technique:${block}`, true, ledgerLookup); } } if (out['rules'] !== undefined) { - stageField(out, 'rules', out['rules'], state, newDeliveries, scope, 'technique:rules', true); + stageField(out, 'rules', out['rules'], state, newDeliveries, scope, 'technique:rules', true, ledgerLookup); } return out; diff --git a/src/utils/gate-liveness.ts b/src/utils/gate-liveness.ts new file mode 100644 index 000000000..3eb09c110 --- /dev/null +++ b/src/utils/gate-liveness.ts @@ -0,0 +1,120 @@ +import { evaluateCondition, type Condition } from '../schema/condition.schema.js'; +import { evaluateWhenExpression, parseWhen, type WhenAst } from '../schema/when-expression.js'; +import type { ProducerSite } from './binding-provenance.js'; + +/** The bag entry a dotted path belongs to: writers name whole variables, gates read into them. */ +function rootOf(path: string): string { + return path.split('.')[0] ?? path; +} + +function collectWhenPaths(ast: WhenAst, out: Set): void { + switch (ast.kind) { + case 'literal': + return; + case 'truthy': + case 'cmp': + out.add(ast.path); + return; + case 'not': + collectWhenPaths(ast.expr, out); + return; + case 'and': + case 'or': + collectWhenPaths(ast.left, out); + collectWhenPaths(ast.right, out); + return; + } +} + +/** Paths a condition reads. `exists` / `notExists` answer on a missing variable; the rest need a value. */ +function collectConditionPaths( + condition: Condition, + value: Set, + presence: Set, +): void { + switch (condition.type) { + case 'simple': + if (condition.operator === 'exists' || condition.operator === 'notExists') { + presence.add(condition.variable); + } else { + value.add(condition.variable); + } + return; + case 'and': + case 'or': + for (const c of condition.conditions) collectConditionPaths(c, value, presence); + return; + case 'not': + collectConditionPaths(condition.condition, value, presence); + return; + } +} + +/** Resolve a dotted bag path, mirroring both reference evaluators' lookup. */ +function readPath(path: string, variables: Record): unknown { + let cur: unknown = variables; + for (const part of path.split('.')) { + if (cur === null || cur === undefined || typeof cur !== 'object') return undefined; + cur = (cur as Record)[part]; + } + return cur; +} + +/** + * Bag entries the producer scan places inside `activityId` — technique outputs and their remaps, + * checkpoint-effect assignments, `action: set` targets, and loop item variables. + */ +export function variablesWrittenIn( + producers: readonly ProducerSite[], + activityId: string, +): Set { + const written = new Set(); + for (const site of producers) { + if (site.activityId === activityId) written.add(rootOf(site.name)); + } + return written; +} + +/** + * What a step's gate evaluates to for the whole of the activity being delivered, or `undefined` where + * it has no answer yet. `when` and `condition` combine under and-semantics; no gate answers `true`. + * The cases and what each means for delivery: docs/resource_resolution_model.md § Which steps get inlined. + */ +export function gateAnswer(args: { + when?: string | undefined; + condition?: Condition | undefined; + variables: Record; + writtenInActivity: ReadonlySet; +}): boolean | undefined { + const { when, condition, variables, writtenInActivity } = args; + if (when === undefined && condition === undefined) return true; + + const valuePaths = new Set(); + const presencePaths = new Set(); + if (when !== undefined) { + const parsed = parseWhen(when); + if (!parsed.ok) return undefined; + collectWhenPaths(parsed.ast, valuePaths); + } + if (condition !== undefined) collectConditionPaths(condition, valuePaths, presencePaths); + + for (const path of [...valuePaths, ...presencePaths]) { + if (writtenInActivity.has(rootOf(path))) return undefined; + } + // Both evaluators return false for an unbound read and for a false one. A compared value that is + // absent is the first, so it has no answer rather than a negative one. + for (const path of valuePaths) { + if (readPath(path, variables) === undefined) return undefined; + } + + const whenSays = when === undefined ? true : evaluateWhenExpression(when, variables); + const conditionSays = condition === undefined ? true : evaluateCondition(condition, variables); + return whenSays && conditionSays; +} + +/** And-combine an enclosing gate's answer with a step's own. */ +export function bothGates(outer: boolean | undefined, own: boolean | undefined): boolean | undefined { + if (outer === false || own === false) return false; + if (outer === undefined || own === undefined) return undefined; + return true; +} diff --git a/tests/batch-loop-walk.test.ts b/tests/batch-loop-walk.test.ts index 1e2238ef6..e6b209317 100644 --- a/tests/batch-loop-walk.test.ts +++ b/tests/batch-loop-walk.test.ts @@ -226,15 +226,25 @@ describe('client activity loop walked (#407)', () => { it('carries the frame a batch of any length needs, outside the body', () => { const def = activityDef(); - // Exactly these steps, in this order. Naming positions instead would miss a step inserted between - // the prime and the loop — one that nulls the pointer keeps the loop from ever running — and a step - // appended after it that re-primes, and a second `kind: loop` the walk's own `find` cannot see. - expect(def.steps.map((s) => s.id)).toEqual([ + // Exactly these steps up to and including the loop, in this order. Naming positions instead would + // miss a step inserted between the prime and the loop — one that nulls the pointer keeps the loop + // from ever running — and a second `kind: loop` the walk's own `find` cannot see. + const ids = def.steps.map((s) => s.id); + const loopAt = ids.indexOf('client-activity-loop'); + expect(ids.slice(0, loopAt + 1)).toEqual([ 'verify-preconditions', 'prime-initial-activity', 'client-activity-loop', ]); expect(def.steps.filter((s) => s.kind === 'loop')).toHaveLength(1); + // A step after the loop reads the pointer to say how the loop ended; one that WRITES it re-primes a + // spent walk, so the activity's transition never fires and close-out is never reached. + for (const after of def.steps.slice(loopAt + 1)) { + expect( + after.actions?.some((a) => a.action === 'set' && a.target === 'current_activity'), + `step '${after.id}' sits after the loop and re-primes the pointer`, + ).toBeFalsy(); + } const [precondition, prime] = def.steps; const l = loop(); diff --git a/tests/borrowed-technique-resolution.test.ts b/tests/borrowed-technique-resolution.test.ts index 23546ce9c..aeab45876 100644 --- a/tests/borrowed-technique-resolution.test.ts +++ b/tests/borrowed-technique-resolution.test.ts @@ -1,6 +1,7 @@ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { loadWorkflowWithDiagnostics } from '../src/loaders/workflow-loader.js'; import { composeActivityTechnique } from '../src/loaders/technique-loader.js'; +import { qualifyResourceId } from '../src/utils/resource-ref.js'; import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join, resolve } from 'node:path'; @@ -55,6 +56,24 @@ describe('borrowed-activity technique resolution', () => { '## Capability', '', 'Standalone operation.', '', '## Protocol', '', '1. Operate.', ].join('\n')); + // A meta technique linking a resource of its own, bare. Whoever binds it, the link names + // meta/resources/shared-template. + const metaDir = join(fixtureDir, 'meta'); + mkdirSync(join(metaDir, 'techniques'), { recursive: true }); + mkdirSync(join(metaDir, 'resources'), { recursive: true }); + writeFileSync(join(metaDir, 'workflow.yaml'), [ + 'id: meta', + 'version: 1.0.0', + 'title: Meta Fixture', + 'initialActivity: shared-work', + ].join('\n')); + writeFileSync(join(metaDir, 'techniques', 'shared-op.md'), [ + '---', 'metadata:', ' version: 1.0.0', '---', '', + '## Capability', '', 'Shared operation with its own resource.', '', + '## Protocol', '', '1. Follow [the template](../resources/shared-template.md).', + ].join('\n')); + writeFileSync(join(metaDir, 'resources', 'shared-template.md'), '# Shared Template\n'); + // Borrower: no techniques of its own; borrows the source workflow's activity by string ref. const borrowerDir = join(fixtureDir, 'borrower-wf'); mkdirSync(join(borrowerDir, 'activities'), { recursive: true }); @@ -103,6 +122,22 @@ describe('borrowed-activity technique resolution', () => { expect(composed.success).toBe(true); }); + it('names the workflow a technique file was found in, not the one that asked for it', async () => { + // Resolved locally: the source workflow holds the file. + const local = await composeActivityTechnique('do-thing', fixtureDir, 'source-wf', 'shared-work'); + expect(local.success).toBe(true); + if (local.success) expect(local.value.sourceWorkflowId).toBe('source-wf'); + + // Resolved through the meta shared layer: source-wf asked, meta answered. A bare resource link + // in that file resolves under meta/resources/, so this is the id qualification must use. + const shared = await composeActivityTechnique('shared-op', fixtureDir, 'source-wf', 'shared-work'); + expect(shared.success).toBe(true); + if (!shared.success) return; + expect(shared.value.sourceWorkflowId).toBe('meta'); + expect(qualifyResourceId('shared-template', shared.value.sourceWorkflowId, 'source-wf')) + .toBe('meta/shared-template'); + }); + it('maps the real corpus: remediate-vuln borrows work-package activities', async () => { const result = await loadWorkflowWithDiagnostics(WORKFLOW_DIR, 'remediate-vuln'); expect(result.success).toBe(true); diff --git a/tests/e2e/__snapshots__/corpus-sha.json b/tests/e2e/__snapshots__/corpus-sha.json index 827fedb37..19efb0e2c 100644 --- a/tests/e2e/__snapshots__/corpus-sha.json +++ b/tests/e2e/__snapshots__/corpus-sha.json @@ -1,4 +1,4 @@ { - "corpusSha": "1921a6e570e3e310dbe0d6b7af95eac18c317930", + "corpusSha": "cf4d0774657e8fbc3f3c94e20130f9a497f62bb8", "note": "Corpus commit the committed walk snapshots were generated against. Update it in the same commit that bumps the workflows submodule and re-baselines the walk (npm run baseline:stamp)." } diff --git a/tests/e2e/__snapshots__/snapshot.test.ts.snap b/tests/e2e/__snapshots__/snapshot.test.ts.snap index 5c9e5e145..d83302539 100644 --- a/tests/e2e/__snapshots__/snapshot.test.ts.snap +++ b/tests/e2e/__snapshots__/snapshot.test.ts.snap @@ -115,9 +115,11 @@ exports[`work-package walk snapshots (baseline) > [default] matches committed ba "activity": "codebase-comprehension", "artifacts": [ "{codebase_area}.md", + "codebase-comprehension.md", ], "artifactsWritten": [ "15-codebase_area.md", + "15-codebase-comprehension.md", ], "checkpoints": [], "manifestStatus": "warning", @@ -130,8 +132,10 @@ exports[`work-package walk snapshots (baseline) > [default] matches committed ba "initial-deep-dive", "update-artifact-initial", "revise-initial-questions", + "record-log-initial", "analyse-challenge-pass", "update-artifact", + "record-log", ], "unresolved": [], }, @@ -554,9 +558,11 @@ exports[`work-package walk snapshots (baseline) > [elicitation-only] matches com "activity": "codebase-comprehension", "artifacts": [ "{codebase_area}.md", + "codebase-comprehension.md", ], "artifactsWritten": [ "15-codebase_area.md", + "15-codebase-comprehension.md", ], "checkpoints": [], "manifestStatus": "warning", @@ -569,8 +575,10 @@ exports[`work-package walk snapshots (baseline) > [elicitation-only] matches com "initial-deep-dive", "update-artifact-initial", "revise-initial-questions", + "record-log-initial", "analyse-challenge-pass", "update-artifact", + "record-log", ], "unresolved": [], }, @@ -1060,9 +1068,11 @@ exports[`work-package walk snapshots (baseline) > [full-workflow] matches commit "activity": "codebase-comprehension", "artifacts": [ "{codebase_area}.md", + "codebase-comprehension.md", ], "artifactsWritten": [ "15-codebase_area.md", + "15-codebase-comprehension.md", ], "checkpoints": [], "manifestStatus": "warning", @@ -1075,8 +1085,10 @@ exports[`work-package walk snapshots (baseline) > [full-workflow] matches commit "initial-deep-dive", "update-artifact-initial", "revise-initial-questions", + "record-log-initial", "analyse-challenge-pass", "update-artifact", + "record-log", ], "unresolved": [], }, @@ -1593,9 +1605,11 @@ exports[`work-package walk snapshots (baseline) > [research-only] matches commit "activity": "codebase-comprehension", "artifacts": [ "{codebase_area}.md", + "codebase-comprehension.md", ], "artifactsWritten": [ "15-codebase_area.md", + "15-codebase-comprehension.md", ], "checkpoints": [], "manifestStatus": "warning", @@ -1608,8 +1622,10 @@ exports[`work-package walk snapshots (baseline) > [research-only] matches commit "initial-deep-dive", "update-artifact-initial", "revise-initial-questions", + "record-log-initial", "analyse-challenge-pass", "update-artifact", + "record-log", ], "unresolved": [], }, @@ -2059,9 +2075,11 @@ exports[`work-package walk snapshots (baseline) > [review-mode] matches committe "activity": "codebase-comprehension", "artifacts": [ "{codebase_area}.md", + "codebase-comprehension.md", ], "artifactsWritten": [ "15-codebase_area.md", + "15-codebase-comprehension.md", ], "checkpoints": [], "manifestStatus": "warning", @@ -2074,8 +2092,10 @@ exports[`work-package walk snapshots (baseline) > [review-mode] matches committe "initial-deep-dive", "update-artifact-initial", "revise-initial-questions", + "record-log-initial", "analyse-challenge-pass", "update-artifact", + "record-log", ], "unresolved": [], }, @@ -2457,9 +2477,11 @@ exports[`work-package walk snapshots (baseline) > [skip-optional] matches commit "activity": "codebase-comprehension", "artifacts": [ "{codebase_area}.md", + "codebase-comprehension.md", ], "artifactsWritten": [ "15-codebase_area.md", + "15-codebase-comprehension.md", ], "checkpoints": [], "manifestStatus": "warning", @@ -2472,8 +2494,10 @@ exports[`work-package walk snapshots (baseline) > [skip-optional] matches commit "initial-deep-dive", "update-artifact-initial", "revise-initial-questions", + "record-log-initial", "analyse-challenge-pass", "update-artifact", + "record-log", ], "unresolved": [], }, diff --git a/tests/gate-liveness.test.ts b/tests/gate-liveness.test.ts new file mode 100644 index 000000000..8b1914320 --- /dev/null +++ b/tests/gate-liveness.test.ts @@ -0,0 +1,97 @@ +/** + * What a step gate evaluates to at the moment its activity is delivered, and when that has no answer. + * The delivery layer bundles a gated step only on a `true`, so the `undefined` cases below are the + * ones that keep a step on its `get_technique` fetch. + */ +import { describe, it, expect } from 'vitest'; +import { bothGates, gateAnswer, variablesWrittenIn } from '../src/utils/gate-liveness.js'; +import type { ProducerSite } from '../src/utils/binding-provenance.js'; +import type { Condition } from '../src/schema/condition.schema.js'; + +const NOTHING_WRITTEN: ReadonlySet = new Set(); + +const answer = ( + gate: { when?: string; condition?: Condition }, + variables: Record, + writtenInActivity: ReadonlySet = NOTHING_WRITTEN, +): boolean | undefined => gateAnswer({ ...gate, variables, writtenInActivity }); + +describe('gateAnswer', () => { + it('answers true for a step with no gate', () => { + expect(answer({}, {})).toBe(true); + }); + + it('evaluates a when expression whose variables are all bound', () => { + expect(answer({ when: 'is_review_mode == true' }, { is_review_mode: true })).toBe(true); + expect(answer({ when: 'is_review_mode == true' }, { is_review_mode: false })).toBe(false); + expect(answer({ when: 'rounds > 0' }, { rounds: 2 })).toBe(true); + expect(answer({ when: '!skip_audit' }, { skip_audit: false })).toBe(true); + }); + + it('has no answer for a variable that is not in the bag', () => { + // Both reference evaluators return false here, which conflates "unbound" with "false". A gate on + // a variable nothing has produced is unanswered, not negative. + expect(answer({ when: 'is_review_mode == true' }, {})).toBeUndefined(); + expect(answer({ when: 'a == true && b == true' }, { a: true })).toBeUndefined(); + }); + + it('has no answer for a variable the activity itself produces', () => { + expect(answer({ when: 'platform == "github"' }, { platform: 'github' }, new Set(['platform']))) + .toBeUndefined(); + // A dotted read is unanswered when the activity writes the bag entry it reads into. + expect(answer({ when: 'plan.approved == true' }, { plan: { approved: true } }, new Set(['plan']))) + .toBeUndefined(); + }); + + it('has no answer for an unparseable when expression', () => { + expect(answer({ when: 'a == && b' }, { a: true, b: true })).toBeUndefined(); + }); + + it('answers an exists condition on a variable that is absent', () => { + const exists: Condition = { type: 'simple', variable: 'issue_url', operator: 'exists' }; + expect(answer({ condition: exists }, {})).toBe(false); + expect(answer({ condition: exists }, { issue_url: 'https://example.test/1' })).toBe(true); + // Still unanswered when this activity is the thing that produces it. + expect(answer({ condition: exists }, {}, new Set(['issue_url']))).toBeUndefined(); + }); + + it('combines when and condition under and-semantics', () => { + const cond: Condition = { type: 'simple', variable: 'rounds', operator: '>', value: 0 }; + expect(answer({ when: 'a == true', condition: cond }, { a: true, rounds: 1 })).toBe(true); + expect(answer({ when: 'a == true', condition: cond }, { a: true, rounds: 0 })).toBe(false); + expect(answer({ when: 'a == true', condition: cond }, { rounds: 1 })).toBeUndefined(); + }); +}); + +describe('bothGates', () => { + it('lets one false limb decide, and one unanswered limb withhold', () => { + expect(bothGates(true, true)).toBe(true); + expect(bothGates(false, true)).toBe(false); + expect(bothGates(true, false)).toBe(false); + // A false enclosing gate settles the body even where the body's own gate is unanswered. + expect(bothGates(false, undefined)).toBe(false); + expect(bothGates(undefined, true)).toBeUndefined(); + expect(bothGates(true, undefined)).toBeUndefined(); + }); +}); + +describe('variablesWrittenIn', () => { + const site = (name: string, activityId: string, via: ProducerSite['via']): ProducerSite => + ({ name, via, stepId: 's', activityId, ordinal: 0 }); + + it('takes only the producer sites inside the named activity', () => { + const producers = [ + site('platform', 'intake', 'output'), + site('branch_prefix', 'intake', 'remap'), + site('review_mode', 'triage', 'checkpoint'), + ]; + expect(variablesWrittenIn(producers, 'intake')).toEqual(new Set(['platform', 'branch_prefix'])); + expect(variablesWrittenIn(producers, 'triage')).toEqual(new Set(['review_mode'])); + expect(variablesWrittenIn(producers, 'absent')).toEqual(new Set()); + }); + + it('reduces a dotted producer name to the bag entry it writes', () => { + expect(variablesWrittenIn([site('plan.tasks', 'intake', 'output')], 'intake')) + .toEqual(new Set(['plan'])); + }); +}); diff --git a/tests/reference-delivery.test.ts b/tests/reference-delivery.test.ts index 119c1deab..8b2eeb304 100644 --- a/tests/reference-delivery.test.ts +++ b/tests/reference-delivery.test.ts @@ -167,25 +167,50 @@ describe('reference-not-repeat delivery (B1)', () => { return result; } - describe('get_activity default mode is unchanged', () => { - it('repeats the full bundle on every call and never emits markers', async () => { + describe('get_activity in the default mode', () => { + it('delivers the worker bundle in full to an identity the server has not met', async () => { const session = await startSession({ workflow_id: 'work-package', agent_id: 'w1' }); const idx = session['session_index'] as string; await enterActivity(idx, 'start-work-package'); const first = splitActivityResponse(await getActivity(idx)); + expect(first.bundle['bundle_mode']).toBeUndefined(); + const techniques = first.bundle['techniques'] as Record; + expect(Object.keys(techniques).length).toBeGreaterThan(0); + for (const value of Object.values(techniques)) { + expect(isUnchangedMarker(value)).toBe(false); + } + }); + + it('refers an identity it has already delivered to back to the bundle it holds', async () => { + const session = await startSession({ workflow_id: 'work-package', agent_id: 'w1' }); + const idx = session['session_index'] as string; + await enterActivity(idx, 'start-work-package'); + await getActivity(idx); + + // Same agent_id, no context_mode declared: the orchestrator holds one identity for as long as + // a worker carries its batch, so a second delivery under it is that same context arriving again. const second = splitActivityResponse(await getActivity(idx)); + const techniques = second.bundle['techniques'] as Record; + for (const [key, value] of Object.entries(techniques)) { + expect(isUnchangedMarker(value), `expected a marker for ${key}`).toBe(true); + } + expect(isUnchangedMarker(second.bundle['rules'])).toBe(true); + expect(second.bundle['bundle_note']).toBeDefined(); + // The activity body is never collapsed, whatever the identity holds. + expect(second.bodyText.length).toBeGreaterThan(0); + }); - for (const parsed of [first, second]) { - expect(parsed.bundle['bundle_mode']).toBeUndefined(); - const techniques = parsed.bundle['techniques'] as Record; - expect(Object.keys(techniques).length).toBeGreaterThan(0); - for (const value of Object.values(techniques)) { - expect(isUnchangedMarker(value)).toBe(false); - } + it('re-delivers everything to a fresh identity in the same session', async () => { + const session = await startSession({ workflow_id: 'work-package', agent_id: 'w1' }); + const idx = session['session_index'] as string; + await enterActivity(idx, 'start-work-package'); + await getActivity(idx); + + const replacement = splitActivityResponse(await getActivity(idx, { agent_id: 'w2' })); + for (const value of Object.values(replacement.bundle['techniques'] as Record)) { + expect(isUnchangedMarker(value)).toBe(false); } - // Byte-identical repetition — the pre-B1 behaviour full mode preserves. - expect(responseText(await getActivity(idx))).toBe(responseText(await getActivity(idx))); }); }); @@ -390,10 +415,13 @@ describe('reference-not-repeat delivery (B1)', () => { } expect(isUnchangedMarker(referenced.bundle['rules'])).toBe(true); - // Omitting the opt-in returns to full delivery. + // Omitting the opt-in drops `bundle_mode`, and `bundle: "full"` is what re-delivers the + // bundle this identity has already been sent. const backToFull = splitActivityResponse(await getActivity(idx)); expect(backToFull.bundle['bundle_mode']).toBeUndefined(); - for (const value of Object.values(backToFull.bundle['techniques'] as Record)) { + + const forced = splitActivityResponse(await getActivity(idx, { bundle: 'full' })); + for (const value of Object.values(forced.bundle['techniques'] as Record)) { expect(isUnchangedMarker(value)).toBe(false); } }); diff --git a/tests/variable-seeding.test.ts b/tests/variable-seeding.test.ts index 0de51069e..f8e30c197 100644 --- a/tests/variable-seeding.test.ts +++ b/tests/variable-seeding.test.ts @@ -11,7 +11,7 @@ import { Client } from '@modelcontextprotocol/sdk/client/index.js'; import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js'; import { createServer } from '../src/server.js'; import { resolve, join } from 'node:path'; -import { mkdtempSync, readFileSync, rmSync } from 'node:fs'; +import { cpSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { seedDefaults, jsonTypeOf, isTemplateReference } from '../src/utils/variable-seed.js'; import { createInitialSessionFile } from '../src/schema/session.schema.js'; @@ -414,3 +414,77 @@ describe('B7 seeding + setVariable type validation (fixture corpus)', () => { }); }); }); + +/** + * Definition drift on resume: the bag is seeded from the declarations that were on disk when the + * session was created, and a resume runs whatever is on disk now. A declaration added since is + * absent from the bag, so a gate on it reads unbound. + * + * Runs against a COPY of the fixture corpus so the definition can be edited under a live session. + */ +describe('resume against an edited workflow definition', () => { + let client: Client; + let closeTransport: () => Promise; + let workspaceDir: string; + let corpusDir: string; + const slug = '2026-08-17-definition-drift'; + const planningFolder = () => join(workspaceDir, '.engineering/artifacts/planning', slug); + const readSession = () => JSON.parse(readFileSync(join(planningFolder(), 'session.json'), 'utf8')); + const workflowPath = () => join(corpusDir, 'seed-fixture/workflow.yaml'); + + // eslint-disable-next-line @typescript-eslint/no-explicit-any + async function call(name: string, args: Record): Promise { + const result = await client.callTool({ name, arguments: args }); + if (result.isError) throw new Error(`${name} failed: ${(result.content as { text: string }[])[0]?.text}`); + return result; + } + + beforeAll(async () => { + workspaceDir = mkdtempSync(join(tmpdir(), 'wf-drift-ws-')); + corpusDir = mkdtempSync(join(tmpdir(), 'wf-drift-corpus-')); + cpSync(resolve(import.meta.dirname, 'fixtures/variable-model'), corpusDir, { recursive: true }); + const server = createServer({ + workflowDir: corpusDir, + schemasDir: resolve(import.meta.dirname, '../schemas'), + workspaceDir, + serverName: 'test-workflow-server', + serverVersion: '1.0.0', + minCheckpointResponseSeconds: 0, + }); + const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); + await server.connect(serverTransport); + client = new Client({ name: 'test-client', version: '1.0.0' }, {}); + await client.connect(clientTransport); + closeTransport = async () => { await client.close(); }; + }); + + afterAll(async () => { + await closeTransport(); + rmSync(workspaceDir, { recursive: true, force: true }); + rmSync(corpusDir, { recursive: true, force: true }); + }); + + it('seeds declarations the bag lacks and re-stamps the recorded version', async () => { + await call('start_session', { + workflow_id: 'seed-fixture', agent_id: 'orchestrator', planning_folder: planningFolder(), + }); + const created = readSession(); + expect(created.workflowVersion).toBe('1.0.0'); + expect(created.variables.late_gate).toBeUndefined(); + // A run decision the resume must not overwrite with the declared default. + expect(created.variables.mode_label).toBe('standard'); + + const edited = readFileSync(workflowPath(), 'utf8') + .replace('version: 1.0.0', 'version: 1.1.0') + + '\n - name: late_gate\n type: boolean\n description: Declared after the session opened.\n defaultValue: false\n'; + writeFileSync(workflowPath(), edited); + + await call('start_session', { + workflow_id: 'seed-fixture', agent_id: 'orchestrator', planning_folder: planningFolder(), + }); + const resumed = readSession(); + expect(resumed.workflowVersion).toBe('1.1.0'); + expect(resumed.variables.late_gate).toBe(false); + expect(resumed.variables.mode_label).toBe('standard'); + }); +}); diff --git a/workflows b/workflows index 1921a6e57..cf4d07746 160000 --- a/workflows +++ b/workflows @@ -1 +1 @@ -Subproject commit 1921a6e570e3e310dbe0d6b7af95eac18c317930 +Subproject commit cf4d0774657e8fbc3f3c94e20130f9a497f62bb8