Skip to content

docs(skills): orchestration layering — harness-native orchestration is layer one - #729

Merged
drewstone merged 1 commit into
mainfrom
docs/orchestration-layering
Aug 3, 2026
Merged

docs(skills): orchestration layering — harness-native orchestration is layer one#729
drewstone merged 1 commit into
mainfrom
docs/orchestration-layering

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Doctrine correction from review: foundation harnesses ship trained orchestration (Claude Code subagents, codex goal-mode, pi extensions) and prose is that layer's native API — instructing a claude-code node to fan out subagents invokes an in-distribution capability, not vibes. The graph/script layer exists only for what no single harness provides: cross-harness composition, one conserved budget across the tree, ledger evidence, resume, heterogeneous models.

Rule: outer layer coarse, inner layer maximal — one harness-sized node told to use its native fan-out beats N externally-choreographed thin nodes that each pay the harness floor while suppressing the model's trained orchestration.

Recorded as the sixth gen4 requirement, because it makes two current artifacts suspect: the unmeasured-harness case expectation (nodes=3) and v3's held N-roles-N-nodes lesson — the layer-aware answer to 'three probes on claude-code' may be one node running three native subagents. Which harnesses qualify is a supervisor-lab harness-KB row, not a guess.

…tration is layer one

Foundation harnesses ship TRAINED orchestration and prose is its native
API. The graph layer composes ACROSS harnesses (conserved budget, ledger
evidence, resume, heterogeneous models); it must not re-choreograph
WITHIN one. Outer coarse, inner maximal.

Consequence recorded as the sixth gen4 requirement: the unmeasured-harness
case expectation (nodes=3) and v3's held N-roles-N-nodes lesson are both
suspect — the layer-aware answer to 'three probes on claude-code' may be
ONE node instructed to run three native subagents. gen4 needs a case
that encodes this, with the qualifying-harness list owned by the
supervisor-lab harness KB.

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 9e332e29

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-08-03T21:02:33Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Value Audit — sound

Verdict sound
Concerns 2 (2 weak-concern)
Heuristic 0.0s
Duplication 0.0s
Interrogation 60.5s (2 bridge agents)
Total 60.5s

💰 Value — sound

Adds a substantive doctrinal requirement to gen3's held-ledger gen4 list — correct and in-grain; the live SKILL.md rule it challenges stays live until gen4 ships, which is the repo's intended loop-gated sequencing.

  • What it does: Appends one item to the gen4Requirements array inside verifierHold in skills/agent-graphs/generations/gen3.json:699. It records an orchestration-layering doctrine — foundation harnesses (Claude Code subagents, codex goal-mode, pi extensions) ship trained orchestration and prose is that layer's native API; the runGraph layer composes ACROSS harnesses (conserved budget, ledger evidence, resume,
  • Goals it achieves: Prevent gen4 from re-deriving the same wrong graph shape by recording, in the ledger that gen4 must read, that 'three probes on claude-code' may correctly be ONE node using native subagents — not three externally-choreographed thin nodes that each pay the harness floor while suppressing the model's trained orchestration. Better system state once merged: the next improvement generation has an expli
  • Assessment: Good change on its merits. The doctrine is substantive and correct: re-choreograph fan-out inside a harness that already has trained orchestration wastes the per-node floor and suppresses native behavior, and 'outer coarse / inner maximal' is the right principle. It is recorded in exactly the right place — verifierHold.gen4Requirements is the forward-looking requirement list for the HELD gen3 lo
  • Better / existing approach: none — this is the right approach. Searched skills/agent-graphs/, docs/, and all *.md for the doctrinal vocabulary and for prior gen-requirement lists; the only gen4Requirements array is the one being extended (skills/agent-graphs/generations/gen3.json:693), and the doctrine appears nowhere else. Editing the live SKILL.md 'MUST create N nodes' rule (skills/agent-graphs/SKILL.md:27,86) or the
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 2
  • Bridge warning: opencode/kimi-for-coding/k2p7: opencode: opencode error

🎯 Usefulness — sound

A coherent doctrine-record change landing in both the durable prose doc and the per-generation ledger, fitting the established pattern and grounded by a verified case-expectation claim.

  • Integration: gen4Requirements is a forward-looking ledger array; IMPROVE.md:54 documents it as the gen4 pointer and bench/src/agent-graphs-gen3.mts writes the file. Nothing parses the array at runtime, which is correct for a requirements record — it is consumed by the next gen4 authoring run, not by code. Reachable via the documented flow.
  • Fit with existing patterns: Matches the established ledger pattern exactly: the prior 5 gen4Requirements entries are the same shape (one string per concrete requirement), and landing durable doctrine in IMPROVE.md while recording the per-generation consequence in gen3.json is the intended split. The unmeasured-harness case (cases/unmeasured-harness.json:5) does carry nodes=3, so the doctrine's stated consequence is grounded
  • Real-world viability: Pure JSON-string + prose addition with no runtime, concurrency, or error-path surface. The one load-bearing factual claim (nodes=3 expectation) was verified against the case file. Holds up under realistic use because there is no happy path to fail — it is a record.
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

🎯 Usefulness Audit

🟡 New ledger entry bundles permanent doctrine with a checkable task into one string [ergonomics] ``

The other 5 gen4Requirements entries (gen3.json:694-698) are each one crisp, independently-closeable requirement. The new entry at gen3.json:699 packs (a) permanent layering doctrine, (b) the outer-coarse/inner-maximal rule, (c) a concrete gen4 case-rewrite task, and (d) harness-KB ownership into one ~600-char string. A gen4 author tracking 'which requirements are done' cannot mark this closed without conflating permanent doctrine (never done) with the case rewrite (checkable). Consider splittin

💰 Value Audit

🟡 Live SKILL.md 'MUST create N nodes' rule stays authoritative until gen4 ships [maintenance] ``

The new requirement makes skills/agent-graphs/SKILL.md:27 ('Identical-Role Parallelism: ... you MUST create N distinct worker nodes') and :86 ('If N parallel instances of a role are requested, create N nodes') suspect for harnesses with trained orchestration, but those lines remain the live, unconditional rule until gen4 promotes. This is by design (loop-gated skill mutation, per commit 5ee60d7) and the requirement is already recorded, so it does not gate this PR — flagging only so a reviewer is


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260803T210530Z

@tangletools

Copy link
Copy Markdown
Contributor

✅ No Blockers — 9e332e29

Review health 100/100 · Reviewer score 89/100 · Confidence 65/100 · 4 findings (4 low)

glm deepseek deepseek-flash aggregate
Readiness 89 95 89 89
Confidence 65 65 65 65
Correctness 89 95 89 89
Security 89 95 89 89
Testing 89 95 89 89
Architecture 89 95 89 89

Reviewer score is advisory once the run is complete and the verdict has no blockers.

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

🟡 LOW Doctrine references a 'supervisor-lab harness-KB row' that is not located in this shot — skills/agent-graphs/IMPROVE.md

Both files assert 'Which harnesses qualify for native fan-out is a supervisor-lab harness-KB row, not a guess.' This is stated as existing infrastructure, but no path or evidence for the harness-KB is given in the changed files. As doctrine this is acceptable (it is a gen4 requirement, not a claim of done work), but if a reader is expected to find this KB row today the reference is dangling. Non-blocking; flagging so gen4 authoring either links the KB or explicitly marks it as to-be-built.

🟡 LOW IMPROVE.md doctrine lacks pointer to the gen4 requirement record — skills/agent-graphs/IMPROVE.md

The header says '(doctrine, gates gen4)' but the section never names where the gen4 consequence lives (gen3.json verifierHold.gen4Requirements, 6th bullet: new one-node+native-fanout case). A reader of IMPROVE.md alone cannot locate the case-design action item. One-line cross-reference would close the loop.

🟡 LOW Casing inconsistency for TRAINED between the two files — skills/agent-graphs/generations/gen3.json

The gen3.json bullet writes 'ship TRAINED orchestration' (all-caps TRAINED) while IMPROVE.md line 41 writes the same phrase as 'ship trained orchestration' (lowercase). Same doctrine, two casings. Cosmetic only — pick one for consistency. Fix: lowercase 'TRAINED' in the gen3 bullet to match IMPROVE.md.

🟡 LOW New doctrine-motivated case could collide with blind-authoring holdout rule — skills/agent-graphs/generations/gen3.json

This requirement is explicitly motivated by measured failure (unmeasured-harness holdout scored 0.5 on 3/5 v3 reps; the held N-roles-N-nodes rule), while gen4Requirements also mandates 'holdout refresh: cases authored blind ... never failure history'. If gen4 encodes the doctrine by mutating the existing holdout case unmeasured-harness, it reproduces the exact case-design contamination pattern the verifier invalidated gen3 for. The bullet says gen4 'needs a case' but not whether it is a fresh blind-authored case vs a rewrite of the suspect holdout. Recommend stating the new case is authored blind to measured failure and added as a new id, not a mutation of unmeasured-harness.


tangletools · 2026-08-03T21:06:06Z · trace

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — 4 non-blocking findings — 9e332e29

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

Full immutable report for this review: trace

Summary comment for this run: full summary


tangletools · 2026-08-03T21:06:06Z · immutable trace

@drewstone
drewstone merged commit b24684c into main Aug 3, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants