Skip to content

fix(ci): agent-jury — gate the auto-merge, install gh, make failures diagnosable - #134

Open
todie wants to merge 2 commits into
mainfrom
fix/agent-jury-harden
Open

fix(ci): agent-jury — gate the auto-merge, install gh, make failures diagnosable#134
todie wants to merge 2 commits into
mainfrom
fix/agent-jury-harden

Conversation

@todie

@todie todie commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Same port applied to unsigned-gg/agentic (#97, #98), sourced from paas#2141 and cortex#44. This repo is the more urgent of the two.

agentic's jury never runs — it requests a runner label absent from the unsigned-gg org, so jobs queue forever. This repo is in cerebral-work where reverie IS valid, so the job is picked up and dies at the first gh pr view with exit 127. Its auto-merge is inert for exactly one reason: gh is missing. This PR adds gh, because nothing works without it — so the gate ships in the same commit, defaulting closed. Otherwise this would arm an unattended merge path on the estate's Linear CLI.

Four defects, all live:

  1. Ungated auto-merge — step named "(if not gated)", zero AGENT_JURY_AUTOMERGE references, squash-merges on an approved verdict plus a clean mergeable_state.
  2. No gh on the runner — pinned 2.76.1, idempotent.
  3. Undiagnosable failures + key through eval — the call was eval curl -sf ... $AUTH_HEADER. -f swallowed the status so the log read "review request failed: " with no reason; eval re-parsed a string containing $LITELLM_KEY. Now an argument array with explicit status capture (HTTP_CODE, body, model, URL; 000 = transport failure).
  4. Two workflows, one check name — both agent-jury.yml and code-review.yml declared a job named GLM-5.2 code review, producing indistinguishable checks. Jury jobs are now Agent Jury delivered / Agent Jury verdict, split into the three-tier shape so a gateway failure stops reading as a rejection.

Plus the 12000 → 60000 diff budget with truncation stated in the prompt.

Not verified by execution; the first proof is the next jury run on this repo.

Refs: OPS-1200, OPS-1203, paas#2141

todie added 2 commits August 15, 2026 17:02
…diagnosable

Same port applied to unsigned-gg/agentic (#97, #98), from paas#2141 and
cerebral-work/cortex#44. This repo is the more urgent of the two.

**Why more urgent.** agentic's jury never runs — it requests a runner label that
does not exist in the unsigned-gg org, so every job queues forever. This repo is
in cerebral-work, where the `reverie` label IS valid, so the job is picked up and
dies at the first `gh pr view` with exit 127. The auto-merge step is therefore
inert for exactly ONE reason: gh is missing from the runner image. Adding gh —
which this PR does, because nothing works without it — would arm an unattended
merge path on the estate's Linear CLI. The gate ships in the same commit for that
reason, defaulting closed.

Four defects, all live:

1. **Ungated auto-merge.** The step is named "(if not gated)" and had zero
   references to AGENT_JURY_AUTOMERGE. It squash-merges on an approved verdict
   plus a clean mergeable_state. Now requires the repo variable set to 'true'.

2. **No gh on the runner.** Every step drives the GitHub API through gh; the
   reverie runner image ships none. Pinned 2.76.1, idempotent.

3. **Undiagnosable failures + the key through `eval`.** The call was
   `eval curl -sf ... $AUTH_HEADER ...`. `-f` meant curl printed nothing on an
   HTTP error, and `eval` re-parsed a string containing $LITELLM_KEY. Now an
   argument array with an explicit status check; captures HTTP_CODE, body,
   model and URL, and 000 for transport failure.

4. **Two workflows, one check name.** agent-jury.yml and code-review.yml both
   declared a job named `GLM-5.2 code review`, producing indistinguishable
   checks. Jury jobs are now `Agent Jury delivered` / `Agent Jury verdict`, split
   into the three-tier shape so a gateway failure stops reading as a rejection.

Also: diff budget 12000 -> 60000 with truncation stated in the prompt.

NOT verified by execution — first proof is the next jury run on this repo.

Refs: OPS-1200, OPS-1203, paas#2141
Completes the tier split — without it the review job exits 0 on a gateway
failure and the error tier collapses into the fail tier. Ported from paas#2141's
'Fail when the review was not delivered', omitted from the first commit and
caught on the sibling agentic PR's own first run.
@todie

todie commented Aug 22, 2026

Copy link
Copy Markdown
Contributor Author

Bridge assessment — this is a bootstrap deadlock, and it is the safety-relevant half of the estate's jury problem

Found on an estate-wide sweep. Verified rather than assumed:

The gate you added defaults closed. if: vars.AGENT_JURY_AUTOMERGE == 'true' && steps.review.outputs.verdict == 'approved' && steps.review.outputs.review_failed != 'true' — the variable is unset, so the auto-merge step never runs. Merging this arms gh and ships the gate in the same commit, exactly as your description claims. That sequencing is the right call and it is why this is safe to land.

Your non-jury CI is fully green: typecheck · test · compile-smoke, CodeQL, and both Analyze jobs. The two jury checks are CANCELLED, not failed.

The deadlock, stated plainly

This repo's jury dies at the first gh pr view with exit 127. So the jury cannot deliver a verdict here until gh is installed — and the PR that installs gh is this one. Waiting does not resolve it; the gate is structurally incapable of reviewing its own fix. That is different from the jury being flaky, and it is the reason I am not treating "no verdict" as "not ready".

Why this matters beyond one repo

Today's estate sweep found the jury failing in four distinct ways that all read as "not green": HTTP 000 non-verdicts (its own 300s curl cap — fixed in cerebral-work/.github#8, merged but not live until the v1 tag moves), runs stuck action_required with zero checks reported, jobs queued forever against an unreachable runner group, and now CANCELLED here.

Against that, an ungated auto-merge is the dangerous half. The estate has separately recorded that the jury approves real defects — a prior sweep found four of five jury-approved PRs held real problems, one a remotely-triggerable DoS. An auto-merge armed on verdict == approved, in a repo whose gate emits false approvals, merges defective code unattended. Your PR closes exactly that, and defaults it off.

What I am doing and not doing

Not merging it. My standing position this session is that when a gate cannot run, I surface rather than substitute my own review for it — that ruling is still open with the operator (task #16 item 3a), and a CI-config change to another repo's merge machinery is not the place to make an exception quietly.

But I am flagging it as the strongest candidate in the queue for landing on a human decision: real CI green, gate verified fail-closed, and a deadlock that no amount of waiting clears. Filed for the operator.

— revenant bridge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant