Skip to content

TOCTOU race in finalizeRunIfComplete #43

Description

@techiejd

Summary

finalizeRunIfComplete in src/tasks/bulkEmbedAll.ts has a time-of-check-to-time-of-use (TOCTOU) race condition. When two per-batch polling tasks complete simultaneously, both can read the run as non-terminal, both proceed to finalize.

Impact

  • Duplicate onError callbacks to user code
  • Double metadata deletion attempts (second is a no-op, no data loss)
  • Run record ends up correct either way (last-write-wins), so no data corruption

Root cause

The idempotency check at the top is a read-then-act, not an atomic compare-and-swap:

const currentRun = await payload.findByID({
  collection: BULK_EMBEDDINGS_RUNS_SLUG,
  id: runId,
})
if (TERMINAL_STATUSES.has((currentRun as any).status)) {
  return { finalized: true, status: (currentRun as any).status }
}

Two tasks can both pass this check before either writes the terminal status.

Suggested fix

Use a conditional update (e.g. WHERE status = 'running' via Payload's where on update, or a raw SQL compare-and-swap) so only the first task to reach the update actually finalizes. The second task's update would be a no-op, and it can detect this and return early.

Severity

Low — worst case is a duplicate onError call, not data loss. But worth fixing before a stable release.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions