What happens
When the root driver's executor fails, supervise ends the whole run with reason: "driver-failed" and no retry. Every live child is torn down with execution aborted, and any work they held in context is lost.
There is no root-level equivalent of the retry a transient failure deserves: one dropped connection, one killed harness process, one upstream 5xx ends a run of arbitrary length.
Why it matters
Measured on one pursuit over four consecutive runs, all on runtime 0.128.0 with a cli-bridge/pi executor:
| run |
outcome |
root spend before death |
children |
qc-ftqc-dsv4pro-20260804 |
driver-failed |
600,044 in / 11,331 out |
0 spawned |
qc-ftqc-dsv4pro-20260804f |
aborted |
1,788,999 in / 19,328 out |
1 killed mid-work |
In the first, the root had completed twelve productive turns — it had read a 2.1M-character corpus, executed the code under analysis, found four real import defects, recorded a knowledge-base entry and written a structured request for missing inputs. All of it was discarded except the two files it had happened to write to disk.
The underlying cause turned out to be external and transient: cli-bridge SIGKILLs a harness process at CLI_TIMEOUT_MS (default 300s), which surfaces as pi exit unknown — filed as drewstone/cli-bridge#119. That is exactly the class of failure a retry exists for. The driver had a 12M-token budget and a 3-hour deadline, both almost untouched, and the run ended anyway.
Two things that would each have saved those runs
1. Retry the root driver on a transient executor failure. The budget pool already tracks spend, so a retry is accountable rather than unbounded — retry until the budget or deadline says stop, not a fixed count. Distinguishing transient (stream error, process died, 5xx) from terminal (validation error, budget exhausted, deliverable refused) is the only classification needed.
2. Do not tear down healthy children when the root dies. In ...f the root's death killed a child that was mid-unit and had produced no output yet. If children were allowed to settle and write, a root failure would cost the synthesis rather than the whole tree.
Related
What happens
When the root driver's executor fails,
superviseends the whole run withreason: "driver-failed"and no retry. Every live child is torn down withexecution aborted, and any work they held in context is lost.There is no root-level equivalent of the retry a transient failure deserves: one dropped connection, one killed harness process, one upstream 5xx ends a run of arbitrary length.
Why it matters
Measured on one pursuit over four consecutive runs, all on runtime 0.128.0 with a cli-bridge/pi executor:
qc-ftqc-dsv4pro-20260804driver-failedqc-ftqc-dsv4pro-20260804fabortedIn the first, the root had completed twelve productive turns — it had read a 2.1M-character corpus, executed the code under analysis, found four real import defects, recorded a knowledge-base entry and written a structured request for missing inputs. All of it was discarded except the two files it had happened to write to disk.
The underlying cause turned out to be external and transient: cli-bridge SIGKILLs a harness process at
CLI_TIMEOUT_MS(default 300s), which surfaces aspi exit unknown— filed as drewstone/cli-bridge#119. That is exactly the class of failure a retry exists for. The driver had a 12M-token budget and a 3-hour deadline, both almost untouched, and the run ended anyway.Two things that would each have saved those runs
1. Retry the root driver on a transient executor failure. The budget pool already tracks spend, so a retry is accountable rather than unbounded — retry until the budget or deadline says stop, not a fixed count. Distinguishing transient (stream error, process died, 5xx) from terminal (validation error, budget exhausted, deliverable refused) is the only classification needed.
2. Do not tear down healthy children when the root dies. In
...fthe root's death killed a child that was mid-unit and had produced no output yet. If children were allowed to settle and write, a root failure would cost the synthesis rather than the whole tree.Related
driver-failedcurrently carries the executor's raw message. When that message ispi exit unknown, an operator cannot tell a timeout from a crash — the diagnosis in harness processes are SIGKILLed at 5 minutes and the failure reports as 'exit unknown' drewstone/cli-bridge#119 took four runs partly for this reason. Surfacing "driver failed after N turns, M tokens; cause: " would make the retry decision legible too.