Context
When audit: is enabled, the CLI runs the normal benchmark first (including finalize_benchmark() and accuracy scoring), then calls run_audit(). audit.only: true skips the preceding main benchmark.
PR #409 gives each AuditRunSpec subrun its own setup-inclusive run_timeout_s deadline so audit subruns match normal benchmark-run semantics. That is not an overall audit budget: a reference phase and fixed-sample phase can each consume the full value, and AuditTest.verify() runs afterward outside those per-subrun deadlines.
origin/main has no active run watchdog for audits; it only applies the existing response/metrics drain budgets inside each subrun.
Scope
Add a first-class overall audit deadline in a follow-up MR rather than overloading the per-benchmark run_timeout_s contract in PR #409. Define its ownership alongside phase configuration work in #449 and scoring/finalization cancellation work in #464.
Acceptance criteria
- One deadline begins before the first audit subrun setup and is shared across every planned audit subrun.
- Remaining budget, rather than a fresh full budget, is passed to each subrun.
- The contract explicitly covers or boundary-checks
AuditTest.verify() and result writing.
audit.only: true uses the same audit deadline semantics.
- Expiry produces a non-zero exit, honest incomplete/interrupted artifacts, and no leaked children.
- Existing per-subrun response drain, metrics drain, and interrupted teardown-grace semantics remain independently configurable.
- Integration coverage blocks deterministically during multiple audit lifecycle stages and proves the overall budget is not reset.
Context
When
audit:is enabled, the CLI runs the normal benchmark first (includingfinalize_benchmark()and accuracy scoring), then callsrun_audit().audit.only: trueskips the preceding main benchmark.PR #409 gives each
AuditRunSpecsubrun its own setup-inclusiverun_timeout_sdeadline so audit subruns match normal benchmark-run semantics. That is not an overall audit budget: a reference phase and fixed-sample phase can each consume the full value, andAuditTest.verify()runs afterward outside those per-subrun deadlines.origin/mainhas no active run watchdog for audits; it only applies the existing response/metrics drain budgets inside each subrun.Scope
Add a first-class overall audit deadline in a follow-up MR rather than overloading the per-benchmark
run_timeout_scontract in PR #409. Define its ownership alongside phase configuration work in #449 and scoring/finalization cancellation work in #464.Acceptance criteria
AuditTest.verify()and result writing.audit.only: trueuses the same audit deadline semantics.