Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions docs/checkpoint_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,40 @@ Fields:

A checkpoint reused at several sites is declared once as a **checkpoint fragment** under `fragments.checkpoints` in the owning workflow's `workflow.yaml`, and each site imports it by reference — the step carries only `kind: checkpoint`, its site-local `id`, and `ref: [workflow::]name` (plus a `condition` when the fragment declares none). The loader materializes the fragment body into the step before delivery, so the yield/present/respond flow and every consumer below see an ordinary full checkpoint; the `check:fragments` guard rejects an inline body that duplicates a fragment (issue #166 B10).

## Where a Checkpoint Belongs

A checkpoint's position in the step list decides whether its answer can steer anything. Every step
gated on a variable the checkpoint decides has to run after it: a gate reading an unbound variable is
false, so the step is skipped, and the answer arrives with nothing left to apply it to. The run
completes, having asked a question that changed nothing.

`check:decision-order` holds the line mechanically — it reports a checkpoint whose decision a step
before it is already gated on. Five cases are exempt, because in each the earlier read has an answer
or loses nothing by not firing:

| Exempt | Why |
|--------|-----|
| The variable declares a `defaultValue` | Seeding puts it in the bag at session creation, so the earlier gate reads the default rather than nothing |
| The earlier gate reads by `exists` / `notExists` | A presence test answers on a missing variable; absence is one of its two answers |
| The earlier step only messages or logs | An announcement that does not fire costs nothing, and gating one on a not-yet-decided value is the ordinary way to stay quiet until it is known |
| The deciding option carries `transitionTo` | Re-entry sends the run back through the earlier step, which then reads what the option wrote |
| The two gates demand incompatible values of one variable | No single run reaches both steps, so the earlier one was never waiting on this decision |

The last two carve out the corpus's standard way of settling a value: a technique derives it, an
announcement reports it when the derivation was confident, a checkpoint decides it when the derivation
was ambiguous, and the announcement and the checkpoint carry opposite gates on the ambiguity flag. On
the corpus the guard was written against, the rule without its exemptions reports 14 pairs and 12 of
them are that shape or one of the other four; every exemption is load-bearing, and removing any single
one puts a working pattern back on the report.

Requirements come from conjuncts only. An `or` proves nothing about which branch a run took, so a
gate built from one contributes no exclusion — the guard reports rather than assumes.

Positioning interacts with the entry rule: `check:checkpoint-entry` refuses a checkpoint as an
activity's first step, because that dispatch pays full delivery and yields before doing any work. A
decision that has to precede all of an activity's work belongs at the preceding activity's tail or as
the orchestrator's precondition on dispatching at all.

## Why this Architecture?

1. **Clean UI Boundaries:** Sub-agents running in hidden background tasks never attempt to prompt the user directly, preventing frozen processes.
Expand Down
2 changes: 1 addition & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ Stdout is one JSON object with per-activity fresh/resume characters and the aggr

By default each run compares against the committed baseline
[`scripts/fixtures/token-benchmark-baseline.json`](../scripts/fixtures/token-benchmark-baseline.json)
(fresh mode, recorded 2026-08-17 against `workflows@34cd5429`). Stderr prints a
(fresh mode, recorded 2026-08-17 against `workflows@72db28ae`). Stderr prints a
compact scorecard; stdout JSON includes `vsReference` with absolute/percent deltas
and a **deliveryCostIndex** (baseline = 100, lower is better — sum of activity +
workflow + resource + technique chars).
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"check:fragments": "tsx scripts/check-fragments.ts",
"check:review-mode": "tsx scripts/check-review-mode-gating.ts",
"check:checkpoint-entry": "tsx scripts/check-checkpoint-entry.ts",
"check:decision-order": "tsx scripts/check-decision-order.ts",
"check:bootstrap": "tsx scripts/check-bootstrap-self-contained.ts",
"check:set-values": "tsx scripts/check-set-action-values.ts",
"check:harness-set": "tsx scripts/check-harness-adapter-set.ts",
Expand Down
217 changes: 217 additions & 0 deletions scripts/check-decision-order.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
/**
* check-decision-order — a checkpoint may not decide a value an earlier step already read (#469).
*
* A step gated on a variable no earlier step could have bound reads nothing, so it is skipped; the
* checkpoint that would have bound it runs later and its answer arrives too late to steer anything.
* The run completes, having asked a question that changed nothing.
*
* What the rule keys on, and why each exemption holds: docs/checkpoint_model.md § Where a Checkpoint
* Belongs.
*
* Run: npx tsx scripts/check-decision-order.ts [--root <workflows-dir>] [--json]
*/
import { readFileSync, readdirSync, existsSync, statSync } from 'node:fs';
import { join, relative, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { parse } from 'yaml';
import { parseWhen, type WhenAst } from '../src/schema/when-expression.js';
import { assertScanned, requireWorkflowsRoot } from './workflows-root.js';
import { runGuard, type Finding } from './guard-protocol.js';

const DIR = fileURLToPath(new URL('.', import.meta.url));
const DEFAULT_ROOT = resolve(join(DIR, '..', 'workflows'));

interface Step {
kind?: string;
id?: string;
when?: string;
condition?: unknown;
actions?: { action?: string }[];
options?: { effect?: { setVariable?: Record<string, unknown>; transitionTo?: string } }[];
}

/** A value a gate requires of one variable. Only conjuncts a run must satisfy to reach the step. */
interface Requirement {
variable: string;
negated: boolean;
value: string;
}

/** The bag entry a dotted path belongs to: writers name whole variables, gates read into them. */
function rootOf(path: string): string {
return path.split('.')[0] ?? path;
}

/**
* Variables a gate needs a *value* for. `exists` / `notExists` answer on a missing variable, so a
* presence read is not waiting on anything and is left out.
*/
function valueReads(step: Step): Set<string> {
const out = new Set<string>();
if (typeof step.when === 'string') {
const parsed = parseWhen(step.when);
if (parsed.ok) collectWhenReads(parsed.ast, out);
}
collectConditionReads(step.condition, out);
return out;
}

function collectWhenReads(ast: WhenAst, out: Set<string>): void {
switch (ast.kind) {
case 'literal':
return;
case 'truthy':
case 'cmp':
out.add(rootOf(ast.path));
return;
case 'not':
collectWhenReads(ast.expr, out);
return;
default:
collectWhenReads(ast.left, out);
collectWhenReads(ast.right, out);
}
}

function collectConditionReads(condition: unknown, out: Set<string>): void {
if (condition === null || typeof condition !== 'object') return;
const c = condition as Record<string, unknown>;
if (typeof c.variable === 'string' && c.operator !== 'exists' && c.operator !== 'notExists') {
out.add(rootOf(c.variable));
}
for (const sub of Array.isArray(c.conditions) ? c.conditions : []) collectConditionReads(sub, out);
collectConditionReads(c.condition, out);
}

/** Requirements provable from a gate: equality conjuncts only. An `or` proves nothing about a run. */
function requirements(step: Step): Requirement[] {
const out: Requirement[] = [];
if (typeof step.when === 'string') {
const parsed = parseWhen(step.when);
if (parsed.ok) collectWhenRequirements(parsed.ast, out);
}
collectConditionRequirements(step.condition, out);
return out;
}

function collectWhenRequirements(ast: WhenAst, out: Requirement[]): void {
if (ast.kind === 'and') {
collectWhenRequirements(ast.left, out);
collectWhenRequirements(ast.right, out);
return;
}
if (ast.kind !== 'cmp' || (ast.op !== '==' && ast.op !== '!=')) return;
out.push({ variable: rootOf(ast.path), negated: ast.op === '!=', value: String(ast.value) });
}

function collectConditionRequirements(condition: unknown, out: Requirement[]): void {
if (condition === null || typeof condition !== 'object') return;
const c = condition as Record<string, unknown>;
if (c.type === 'simple') {
if ((c.operator === '==' || c.operator === '!=') && typeof c.variable === 'string') {
out.push({ variable: rootOf(c.variable), negated: c.operator === '!=', value: String(c.value) });
}
return;
}
if (c.type !== 'and') return;
for (const sub of Array.isArray(c.conditions) ? c.conditions : []) {
collectConditionRequirements(sub, out);
}
}

/** Whether two gates demand incompatible values of one variable, so no run reaches both steps. */
function neverBothRun(a: Requirement[], b: Requirement[]): boolean {
for (const ra of a) {
for (const rb of b) {
if (ra.variable !== rb.variable) continue;
if (!ra.negated && !rb.negated) {
if (ra.value !== rb.value) return true;
} else if (ra.negated !== rb.negated && ra.value === rb.value) {
return true;
}
}
}
return false;
}

/** Whether a skipped run loses anything. An announcement that does not fire costs nothing. */
function doesWork(step: Step): boolean {
if (step.kind === 'technique' || step.kind === 'loop') return true;
if (step.kind !== 'action') return false;
const actions = step.actions ?? [];
return actions.some((a) => a.action !== 'message' && a.action !== 'log');
}

/** Variables a checkpoint's options bind, minus those bound by an option that re-enters. */
function decidedVariables(step: Step): Set<string> {
const decided = new Set<string>();
const reentrant = new Set<string>();
for (const option of step.options ?? []) {
const set = option.effect?.setVariable;
if (set === undefined) continue;
const target = typeof option.effect?.transitionTo === 'string' ? reentrant : decided;
for (const name of Object.keys(set)) target.add(name);
}
for (const name of reentrant) decided.delete(name);
return decided;
}

/** Declared variables carrying a `defaultValue`: an earlier read of one has the default to read. */
function defaultedVariables(workflowDir: string): Set<string> {
const path = join(workflowDir, 'workflow.yaml');
if (!existsSync(path)) return new Set();
const declared = (parse(readFileSync(path, 'utf-8')) as { variables?: unknown })?.variables;
const out = new Set<string>();
for (const v of Array.isArray(declared) ? declared : []) {
if (v !== null && typeof v === 'object' && typeof (v as { name?: unknown }).name === 'string') {
if ('defaultValue' in (v as object)) out.add((v as { name: string }).name);
}
}
return out;
}

export function collectFindings(root: string = DEFAULT_ROOT): Finding[] {
const findings: Finding[] = [];
let scanned = 0;
for (const workflow of readdirSync(root).sort()) {
const workflowDir = join(root, workflow);
const activitiesDir = join(workflowDir, 'activities');
if (!existsSync(activitiesDir) || !statSync(activitiesDir).isDirectory()) continue;
const defaulted = defaultedVariables(workflowDir);
for (const entry of readdirSync(activitiesDir).sort()) {
if (!entry.endsWith('.yaml') && !entry.endsWith('.yml')) continue;
const path = join(activitiesDir, entry);
const def = parse(readFileSync(path, 'utf-8')) as { id?: string; steps?: Step[] } | null;
scanned++;
const steps = def?.steps ?? [];
steps.forEach((checkpoint, index) => {
if (checkpoint.kind !== 'checkpoint') return;
const gate = requirements(checkpoint);
for (const name of decidedVariables(checkpoint)) {
if (defaulted.has(name)) continue;
const reader = steps
.slice(0, index)
.find((s) => doesWork(s) && valueReads(s).has(name) && !neverBothRun(requirements(s), gate));
if (reader === undefined) continue;
findings.push({
check: 'decides-after-use',
site: `${relative(root, path)}::${checkpoint.id ?? '?'}`,
detail: `checkpoint '${checkpoint.id ?? '?'}' decides '${name}', which step `
+ `'${reader.id ?? '?'}' is already gated on — that step runs first, reads nothing, and `
+ `is skipped. Move the checkpoint above it, or give '${name}' a producer that runs first.`,
});
}
});
}
}
assertScanned(scanned, 'activity files', root);
return findings;
}

const isMain = !!process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href;
if (isMain) {
await runGuard('decision-order', () => requireWorkflowsRoot(DEFAULT_ROOT), collectFindings, {
okMessage: 'no checkpoint decides a value an earlier step already read',
remedy: 'move the checkpoint above the step gated on its decision, or give that variable an earlier producer',
});
}
26 changes: 13 additions & 13 deletions scripts/fixtures/token-benchmark-baseline.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"label": "baseline",
"description": "Gate baseline: work-package / skip-optional / robot + hot-resource probe, context_mode fresh. Recorded 2026-08-17 against workflows@cf4d0774. The July A0 reference recorded 1,355,532 delivery characters at workflows@a1409d5b; the same walk measured 1,780,292 on 2026-08-17 before the delivery corrections in .engineering/artifacts/planning/2026-08-17-meta-and-work-package-workflow-optimisation/EVALUATION-REPORT.md.",
"description": "Gate baseline: work-package / skip-optional / robot + hot-resource probe, context_mode fresh. Recorded 2026-08-17 against workflows@72db28ae. Delivery is 5,559 characters above the workflows@cf4d0774 recording of 1,296,760, buying: a step that derives the build-artifact regeneration commands, a project-type clause on the two gates that present them, and a second binding of the issue-reference detector so the not-creating path binds a platform at all. The detector's second delivery is a full 4,054-character fetch rather than a ledger hit, because a technique bundled into get_activity does not satisfy a later standalone get_technique for the same content. The project-type clause drops the walk from 11 checkpoint round trips to 10, since a project with no build-dependent artifacts is no longer asked about them. The July A0 reference recorded 1,355,532 delivery characters at workflows@a1409d5b; the same walk measured 1,780,292 on 2026-08-17 before the delivery corrections in .engineering/artifacts/planning/2026-08-17-meta-and-work-package-workflow-optimisation/EVALUATION-REPORT.md.",
"contextMode": "fresh",
"workflowsRev": "cf4d0774",
"workflowsRev": "72db28ae",
"agentId": "bench-solo",
"path": [
"start-work-package",
Expand All @@ -24,29 +24,29 @@
"get_workflow": 1,
"next_activity": 12,
"get_activity": 12,
"get_technique": 23,
"get_technique": 24,
"get_resource": 162,
"yield_checkpoint": 11,
"respond_checkpoint": 11,
"resume_checkpoint": 11
"yield_checkpoint": 10,
"respond_checkpoint": 10,
"resume_checkpoint": 10
},
"chars": {
"get_activity": 518679,
"get_workflow": 108280,
"get_activity": 520075,
"get_workflow": 108356,
"get_resource": 527683,
"get_technique": 142118
"get_technique": 146205
},
"history": {
"technique_bundled": 66,
"technique_fetched": 23,
"technique_fetched": 24,
"resource_fetched": 146
},
"deliveredContentKeys": 265,
"resourceLedgerKeys": 77,
"unchangedResourceAnswers": 0,
"unchangedTechniqueAnswers": 0,
"getActivityChars": 518679,
"getWorkflowChars": 108280,
"getActivityChars": 520075,
"getWorkflowChars": 108356,
"getResourceChars": 527683,
"getTechniqueChars": 142118
"getTechniqueChars": 146205
}
8 changes: 8 additions & 0 deletions scripts/guards.ts
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,14 @@ export const GUARDS: GuardSpec[] = [
json: true,
proves: 'no activity opens with a checkpoint, so no dispatch exists only to ask a question',
},
{
id: 'decision-order',
script: 'scripts/check-decision-order.ts',
npmScript: 'check:decision-order',
scope: 'corpus',
json: true,
proves: 'no checkpoint decides a value a step before it is already gated on',
},
{
id: 'bootstrap-self-contained',
script: 'scripts/check-bootstrap-self-contained.ts',
Expand Down
Loading
Loading