Fan swarm's external voices out per lens cluster (0.7.0)

.claude-plugin/marketplace.json

@@ -29,8 +29,8 @@
     {
       "name": "swarm",
       "source": "./plugins/swarm",
-      "description": "Local mixture-of-agents code review for Claude Code. Fans a diff across Claude lenses plus the codex and grok CLIs (grok-4.5) with file-read + hardened web research under an OS secret-jail, merges by mechanism with cross-family consensus, verifies solo findings and all design suggestions, and presents one ranked report. Optional --fix / --loop applies the findings you agreed with; --pr reviews a GitHub PR diff and posts the result. Skills: /swarm:review, /swarm:agents.",
-      "version": "0.6.0"
+      "description": "Local mixture-of-agents code review for Claude Code. Fans a diff across Claude lenses plus the codex and grok CLIs (grok-4.5) — every voice running one call per gated lens cluster — with file-read + hardened web research under an OS secret-jail, merges by mechanism with cross-family consensus, verifies solo findings and all design suggestions, and presents one ranked report. Optional --fix / --loop applies the findings you agreed with; --pr reviews a GitHub PR diff and posts the result. Skills: /swarm:review, /swarm:agents.",
+      "version": "0.7.0"
     },
     {
       "name": "settings",

.claude/knowledge/features/swarm-review-pipeline.md

@@ -1,9 +1,9 @@
 ---
 title: "Swarm Review Pipeline (/swarm:review)"
 createdAt: 2026-07-08
-updatedAt: 2026-07-23
+updatedAt: 2026-07-27
 createdFrom: "PR #24"
-updatedFrom: "open-swarm-external-exploration"
+updatedFrom: "swarm-per-lens-externals"
 pluginVersion: 1.9.0
 prime: false
 reindexedAt: 2026-07-12
@@ -25,7 +25,7 @@ Grown from 5 topical lenses by importing `/code-review`'s other two
 decomposition axes — methodological (HOW to look) and design quality — all
 **default-on** (user directive 2026-07-15: maintainability reviewed on every
 run, not opt-in). `LENS_CLUSTERS` in the workflow is the **single source of
-truth** (the per-cluster externals follow-up consumes it):
+truth** — every voice's fan-out units come from it, Claude and externals alike:
 
 | cluster | lenses | guiding question |
 |---|---|---|
@@ -34,15 +34,17 @@ truth** (the per-cluster externals follow-up consumes it):
 | `design` | reuse, simplification, efficiency, altitude | is this good, maintainable code? |
 | `consistency` | style, conventions | does it fit the codebase? |
 
-- **The cluster is the Claude fan-out unit** (≤4 finders); `--max` splits to
-  one finder per lens (≤11) — the granularity ladder is `--quick` (future) =
+- **The cluster is the fan-out unit for EVERY voice** since 0.7.0 — Claude
+  finders (≤4) *and* codex/grok (one CLI call per gated cluster each);
+  `--max` splits all of them to one call per lens (≤11 units → ≤22 external
+  calls) — the granularity ladder is `--quick` (future) =
   one broad pass → default = per-cluster → `--max` = per-lens. The **gate
-  stays per-lens** (a fully-pruned cluster spawns no agent); design lenses are
+  stays per-lens** (a fully-pruned cluster spawns no agent for anyone); design lenses are
   first-class in the gate prompt, skipped only when the diff can't pay off.
   **Accepted tradeoff of the cluster default:** per-lens failure isolation is
-  gone — one crashed cluster finder drops that whole cluster's Claude coverage
-  for the round (a visible `backendError`, never silent); `--max` restores
-  per-lens isolation. Documented, not retried per-lens (minimal — the default
+  gone — one crashed cluster call drops that whole cluster's coverage *for that
+  voice* (a visible `backendError` carrying the unit + its lenses, never
+  silent); `--max` restores per-lens isolation. Documented, not retried per-lens (minimal — the default
   trades isolation for fewer agents).
 - **`kind` is derived from the lens name** (`design` vs `defect`) — no
   finding-schema change, so the 3-place schema mirror is untouched. A merged
@@ -63,9 +65,12 @@ truth** (the per-cluster externals follow-up consumes it):
   suggestion-shaped, but each has a falsifiable applicability core (reuse
   target exists? simpler form behavior-identical? claimed waste real?) — the
   same 3-state verifier runs with an applicability prompt, **even for design
-  clusters with cross-family consensus** (agreement ≠ applicability: externals
-  only see the diff, so two of them can agree on a nonexistent reuse target —
-  the first live swarm run over this very feature caught that gap). Bypassing
+  clusters with cross-family consensus** (agreement ≠ applicability: two voices
+  can agree on a nonexistent reuse target — the first live swarm run over this
+  very feature caught that gap. That was originally because externals were
+  diff-only; since 0.6.0 they read the repo, so the residual reason is
+  correlation, not blindness: the voices share a prompt frame, a cluster scope
+  and — per-cluster since 0.7.0 — the same lens briefs). Bypassing
   into an unverified "maintainability" section would have surfaced unchecked
   suggestions from precisely the noisiest lenses. Methodological lenses are
   factual → normal defect verify; defect consensus stays auto-accepted.
@@ -254,15 +259,68 @@ filled* — `gh pr diff <n>` (bare `--pr` resolves the current branch's PR via
   retired `DESIGN_LENSES` was): it runs only inside the already-`kind`-decided
   design branch and never moves a row between tables.
 
-## Future idea (P3+): per-cluster external prompts
+## Per-cluster external prompts (shipped 0.7.0)
 
-Today externals run ONE broad multi-lens review each (the prompt names all 11
-lenses); Claude fans out per cluster. Running externals **per cluster** too
-(re-scoped 2026-07-15 from the original per-lens idea — clusters cap the
-multiplier at 4×, not 11×) would add depth + symmetry + authoritative lens tags
-+ let the gate prune external calls. That is the `swarm-per-lens-externals`
-follow-up task, built on `LENS_CLUSTERS`; for routine depth prefer higher
-external `--effort` / grok `--best-of-n` (one call, more thinking) over N calls.
+Externals no longer run ONE broad multi-lens review each: codex and grok fan out
+over the **same gated clusters** as the Claude finders (`unitsFor()` builds the
+units once; `externalUnits` reuses `finderUnits` whenever a gate ran, so the two
+sides cannot drift). Cost is `live-backends × units` — ≤2×4 default, ≤2×11 under
+`--max` — logged at fan-out, never silently capped.
+
+Decisions worth keeping:
+
+- **Where the prompt is assembled.** `LENS_BRIEF` stays single-source in the
+  workflow, so the briefs must *travel*. The workflow sandbox cannot write files
+  and the skill's Bash prep runs **before** the gate exists (so it cannot know the
+  surviving clusters) — so the adapter grew `run --lens-instr <s>`, which prepends
+  the briefs to the fenced-diff prompt in **deterministic shell**. Handing that
+  write to the transport agent was the rejected alternative: it would put prompt
+  assembly inside an LLM, breaking the same "fencing/assembly is never an LLM step"
+  contract the diff fencing follows. Being on the adapter also makes it
+  backend-agnostic — a future voice (Kimi) inherits per-cluster prompts for free.
+  Consequence: the instruction rides as one single-quoted argv word in a command a
+  haiku transport retypes, so briefs are guarded apostrophe-/control-char-free and
+  the command is kept to one line. E2E: 8/8 calls round-tripped byte-exact.
+- **The SKILL.md HDR is now LENS-FREE** — the hand-mirrored lens list is gone
+  (that was the drift risk `test_lens_sync.py` existed to catch; the check flipped
+  to a *negative* one). The HDR must also stay lens-*agnostic* about finding kinds:
+  a leftover "report every design-quality improvement" line contradicts a
+  defect-only cluster's "one finding per distinct defect" instruction and invites
+  off-cluster tags.
+- **A gate that prunes for everyone needs a floor.** With externals gated too, a
+  lens the low-effort haiku gate drops is reviewed by *nobody* — the full-width
+  external calls used to absorb a mis-gate. `MANDATORY_LENSES`
+  (`security`, `adversarial`, `correctness`) is the code-level backstop, since the
+  gate's only other protection is a sentence in its own prompt (injection-reachable
+  via the diff it classifies). **Accepted cost:** flooring these pins the
+  `breakage` and `threat` clusters always-on, so the gate can only prune `design`
+  and `consistency` — a doc-only diff still pays 2 clusters × live voices. Chosen
+  deliberately: a clean report on the dimensions most costly to miss is worse than
+  the calls saved.
+- **The transport retype is guarded by a content checksum, not trust.** The instruction
+  rides as one argv word a haiku agent retypes; an EMPTY value is refused, and
+  `--lens-instr-sum` (an FNV-1a/32 of the exact text, REQUIRED alongside it) makes
+  a *reworded, paraphrased or truncated* one fail too — otherwise the backend would review a narrower scope
+  than the findings get labelled with, hollowing out "the voice IS its cluster".
+  8 hex chars survive a retype far better than 1 KB of prose. A byte COUNT was the first attempt and was not enough — `security`/`altitude`, `ONLY`/`ALSO` are same-length swaps that change the scope while the count still matches; the check must bind content, and must be refused-if-absent or a dropped flag voids it.
+- **Accepted residuals** (re-found by every review round — decided, not
+  overlooked): (a) [RESOLVED 0.7.0] the oversize skip is now
+  decided in the prep Bash (`EXTERNALS_OVERSIZE`), not by model arithmetic — the
+  constant stays in the skill but is pinned to `max_bytes` by `test_lens_sync.py`;
+  (b) one adapter process per unit re-runs
+  grok's process-local model probe per cluster (4× instead of 1×) — wasted
+  network calls, but they overlap the review calls, so wall-clock cost is ~0 and
+  caching would add staleness for no user-visible gain.
+- **The coverage line must partition the lens set.** Both halves are easy to get
+  wrong and neither fails loudly: lenses the gate lists in neither `run` nor
+  `skip` are dropped silently (seen live — a real run swallowed `adversarial`),
+  and a floored-in lens left out of `gate.run` *runs* while appearing in neither
+  report column (the external-only control run caught exactly that regression in
+  the floor's first version). The workflow now rewrites both fields and asserts
+  they partition `CANDIDATE_LENSES`.
+
+For routine depth prefer higher external `--effort` / grok `--best-of-n` (one
+call, more thinking) over more calls.
 
 ## Verified end-to-end (2026-07-05)
 

CHANGELOG.md

@@ -212,6 +212,17 @@ entries are grouped per plugin, newest first.
 
 ## swarm
 
+### 0.7.0 — 2026-07-27
+- **Per-cluster external voices (default):** `codex` and `grok` no longer run one broad multi-lens review each — they fan out over the **same gated lens clusters** as the Claude finders (one call per cluster; per lens under `--max`). The gate now prunes calls for *everyone*: a fully-gated-out cluster spawns nothing for any voice. Cost is `live-backends × units` external calls (≤2×4 default, ≤2×11 under `--max`) and is logged at fan-out — never silently capped.
+- **Authoritative lens tags:** each external voice *is* its cluster, so a finding's `[lens]` prefix no longer depends on a broad prompt self-tagging correctly. Untagged findings from a single-lens external unit now resolve to that lens (same rule the Claude finders already used) instead of falling back to `unspecified`.
+- **`LENS_BRIEF` becomes single-source:** new adapter flag `agents.sh run --lens-instr <s>` prepends the workflow-supplied cluster briefs to the fenced-diff prompt in **deterministic shell** — the same "assembly is never an LLM step" contract the diff fencing follows, and backend-agnostic, so a future voice inherits per-cluster prompts for free. The SKILL.md external-prompt HDR drops its hand-mirrored lens list; `test_lens_sync.py` flips that mirror check to a negative one and pins the `--lens-instr` wiring.
+- **Mandatory-lens floor.** Because the gate now prunes for every voice, a lens it wrongly drops is reviewed by nobody — the old full-width external calls used to absorb that, and the gate's only other protection is a sentence in its own prompt (injection-reachable via the diff it classifies). `MANDATORY_LENSES` (`security`, `adversarial`, `correctness`) can never be pruned; accepted cost is that the `breakage` and `threat` clusters always spawn, leaving the gate only `design`/`consistency` to prune. The report's coverage line now partitions the full lens set: floored-in lenses are written back to `gate.run`, and lenses the gate listed in neither field are materialized as gated-out (a live run silently swallowed `adversarial` this way).
+- **Transport integrity is content-bound.** `--lens-instr-sum` (FNV-1a/32 of the exact instruction, computed in pure JS by the workflow and recomputed in python3 by the adapter) replaces the first attempt at a byte-length check — `security`/`altitude` and `ONLY`/`ALSO` are same-length swaps that change the review scope while a byte count still matches. It is **required** whenever `--lens-instr` is given, so a transport cannot void the guard by dropping one flag. The prep block now also decides the oversize skip deterministically (`EXTERNALS_OVERSIZE`) instead of leaving the arithmetic to the model.
+- **Gate-floor and coverage-line integrity.** `MANDATORY_LENSES` is asserted to be a subset of the lens set at startup and in CI (an explicit list is a mirror: a renamed lens would silently void the floor). The gate coverage line drops hallucinated lens names and asserts run/skip stay disjoint, so every lens appears in exactly one column.
+- The `--max` profile now lifts **every** voice to per-lens granularity (previously Claude only). A `claude: false` control run keeps full-width external coverage (no gate exists to prune it), just split per cluster.
+- Grok no longer needs a diff-only brief variant: since 0.6.0 both externals read project files, so both get the full cluster briefs.
+- The skill's oversize threshold drops to 118784 bytes (4 KiB under the adapter's 120 KiB cap) — `exec` now sees lens instruction + diff, so a prompt that only just fit would otherwise fail per call as a backend error.
+
 ### 0.6.0 — 2026-07-20
 - **Posture change: external voices get file-read + always-on web research** (hardened egress). codex runs `-s read-only -C <repo> -c tools.web_search=true`; grok runs a strict `--tools` allowlist (`read_file,list_dir,grep,web_search,web_fetch`) + `--cwd <repo>` — no write/shell tools, no `--disable-web-search`. Enables out-of-diff bug finding and external knowledge (API docs, CVEs) without re-opening the secret-exfil hole.
 - **Keep + extend the OS secret-jail**: repo-**root** `.env*`, `data/`, `*.pem`, SSH id keys (`id_rsa*`/`id_ed25519*`/`id_ecdsa*`/`id_dsa*`), `*.key`, `.npmrc`, `.pypirc`, `credentials.json` join the HOME denylist — which itself gains `~/.gitconfig`, `~/.config/git`, and `~/.cargo/credentials.toml` (git keeps working via `GIT_CONFIG_GLOBAL/SYSTEM=/dev/null`, so the denied global config is never opened by git yet stays unreadable to a direct read). Root-level only — nested secrets via `SWARM_DENY_PATHS`, which also takes per-repo extras. New `test_sandbox_deny.py` regression-checks the denylist, the fail-closed degrade argv, and (e2e) blocks a temp `.env` when `sandbox-exec` works.

CLAUDE.md

@@ -21,7 +21,7 @@ This is a **Claude Code plugin marketplace** (monorepo) containing plugins that
 - **knowledge-system** (v1.9.x) — Knowledge management with three layers: Rules, Knowledge, Memory. Skills: `/init`, `/query`, `/curate`, `/reindex`, `/backfill-knowledge`, `/migrate`, `/statusline`
 - **work-system** (v1.10.x) — Task and worktree workflow. Skills: `/define`, `/kickoff`, `/adopt`, `/continue`, `/status`, `/close`, `/list`, `/statusline`
 - **pr-flow** (v1.3.x) — PR review feedback loop. Skills: `/open`, `/cycle`, `/check`, `/fix`, `/rebase`, `/merge`
-- **swarm** (v0.6.x) — Local mixture-of-agents code review (external `codex`/`grok` CLIs — grok-4.5 — plus Claude lenses: 11 in 4 clusters). Externals get file-read + web research under an OS secret-jail. P2: `/swarm:review` pipeline (scope→fan-out→merge→verify); P5: `--fix`/`--loop` apply the findings you agreed with. Skills: `/swarm:review`, `/swarm:agents`
+- **swarm** (v0.7.x) — Local mixture-of-agents code review (external `codex`/`grok` CLIs — grok-4.5 — plus Claude lenses: 11 in 4 clusters). Every voice fans out per gated cluster; externals get file-read + web research under an OS secret-jail. P2: `/swarm:review` pipeline (scope→fan-out→merge→verify); P5: `--fix`/`--loop` apply the findings you agreed with. Skills: `/swarm:review`, `/swarm:agents`
 - **settings** (v0.1.x) — Per-plugin TOML config resolved over schema defaults; each plugin owns its `schema/settings.schema.json`. Skill: `/settings` (list/show/get/set/validate). Phase 1: config surface only.
 
 ## Plugin Anatomy

README.md

@@ -30,7 +30,7 @@ PR review feedback loop. Create PRs with readiness checks, commit + push + trigg
 
 ### Swarm
 
-Local mixture-of-agents code review. Fans out one review across Claude lens subagents (11 lenses in 4 clusters — breakage, threat, design, consistency) plus the `codex` and `grok` CLIs (grok-4.5), merges and verifies their findings, and presents a single ranked report — defects and design suggestions kept apart — before anything is pushed. The external voices read project files and research online (finding out-of-diff bugs), jailed under an OS secret-deny sandbox with a hardened egress policy. With `--fix` / `--loop` it also applies the findings you agreed with (only Claude edits; the external agents stay review-only). `--pr [<number>]` runs the same ensemble against a GitHub PR's diff and posts the gated result as a PR comment (via your own `gh` auth, after one confirmation) — no CI or API-token setup. Complementary to PR Flow's GitHub-side loop. *(0.6.0: external file-read + web research under the secret-jail.)*
+Local mixture-of-agents code review. Fans out one review across Claude lens subagents (11 lenses in 4 clusters — breakage, threat, design, consistency) plus the `codex` and `grok` CLIs (grok-4.5) — every voice running one call per gated cluster, so the gate prunes work for all of them and each finding's lens tag is authoritative — merges and verifies their findings, and presents a single ranked report — defects and design suggestions kept apart — before anything is pushed. The external voices read project files and research online (finding out-of-diff bugs), jailed under an OS secret-deny sandbox with a hardened egress policy. With `--fix` / `--loop` it also applies the findings you agreed with (only Claude edits; the external agents stay review-only). `--pr [<number>]` runs the same ensemble against a GitHub PR's diff and posts the gated result as a PR comment (via your own `gh` auth, after one confirmation) — no CI or API-token setup. Complementary to PR Flow's GitHub-side loop. *(0.7.0: per-cluster external voices — codex/grok fan out over the same gated clusters as the Claude finders.)*
 
 **Commands:** `/swarm:review [--fix | --loop[=N]] [--max]`, `/swarm:review --pr [<number>]`, `/swarm:agents` *(planned: `/swarm:adversarial`, `/swarm:style`, `/swarm:security` — thin subset presets of the default lens set)*