Skip to content

Fence workflow cancellation against run start and active execution #529

Description

@chubes4

Problem

WP_Agent_Workflow_Request_Controller::cancel() can report and deliver a terminal cancelled operation without proving that the workflow runner accepted cancellation.

Two races share the same missing ownership contract:

  1. If cancellation arrives before WP_Agent_Workflow_Runner::start_run() creates the run-control row, WP_Agent_Run_Control::request_cancel() returns no accepted state. The controller ignores that result, records the workflow result as cancelled, and delivers terminal cancellation, while the worker may subsequently start and execute normally.
  2. If an active worker has already passed its final cancellation check, the controller does not fence cancellation against the worker lease. It can deliver cancelled while the worker subsequently persists succeeded or failed, leaving controller and recorder state inconsistent.

Reproduction

Cancel before run start

  1. Reserve a request-controller operation before its runner creates the workflow run-control row.
  2. Call cancel() for the operation.
  3. Observe a terminal cancelled response even though request_cancel() did not persist cancellation.
  4. Start the worker and observe normal execution.

Cancel after the worker's last check

  1. Advance an operation until the worker holds the operation lease and passes its final run-control cancellation check.
  2. Call cancel() concurrently.
  3. Observe cancelled terminal delivery.
  4. Allow the worker to finish and observe a conflicting terminal recorder result.

Impact

  • External consumers receive a false terminal cancellation.
  • Workflow side effects can occur after cancellation was reported complete.
  • Recorder and request-controller terminal states can disagree.
  • Cleanup can unschedule actions while the active worker still owns execution.

Acceptance criteria

  • Cancellation has a durable operation-level intent that exists before workflow run creation and is consumed when the runner starts.
  • An active execution lease is fenced so cancellation and worker terminalization have one authoritative winner.
  • cancel() reports accepted/terminal state only when durable cancellation is proven.
  • Controller, recorder, and run-control state cannot terminalize to conflicting outcomes.
  • Deterministic concurrent tests cover cancellation before run creation and after the worker's final cancellation check.

AI assistance

OpenAI GPT-5.6 Sol via OpenCode general coding and review subagents traced the request-controller and runner cancellation paths, identified the two races, searched existing trackers, and drafted this report. Chris Huber directed the orchestration and remains responsible for the issue and resulting implementation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions