Skip to content

fix(scheduler): record who initiated an execution (#1970) - #1974

Merged
vybe merged 3 commits into
devfrom
fix/1970-scheduler-execution-origin
Aug 4, 2026
Merged

fix(scheduler): record who initiated an execution (#1970)#1974
vybe merged 3 commits into
devfrom
fix/1970-scheduler-execution-origin

Conversation

@dolho

@dolho dolho commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Problem

schedule_executions has carried five execution-origin columns for audit since AUDIT-001, and src/backend/database.py populates them on every path the backend owns. The scheduler is a separate service with its own DB module, and its create_execution() listed none of them:

INSERT INTO schedule_executions (
    id, schedule_id, agent_name, status, started_at, message, triggered_by,
    model_used, attempt_number, retry_of_execution_id
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

Nor did the signature accept them, so there was nowhere to put a caller even if one had been forwarded. Every scheduler-created row was written with all five NULL — observed across 100+ runs of an actively used schedule.

triggered_by='manual' recorded that a human ran something and never who. The attribution existed only in backend / MCP-server logs, bounded by log retention, so past a few weeks the durable record could not answer "did anyone trigger this run, and who?".

Not a vulnerability — nothing authorizes on these columns.

The identity was dropped at three points, not one

# Where What was missing
1 routers/schedules.py::trigger_schedule The delegating POST to the scheduler sent no body at all. The authenticated caller was in scope right there and never crossed the hop.
2 scheduler/main.py::_trigger_handler No parameter to receive an identity.
3 scheduler/database.py::create_execution Nowhere to put it.

Fixing only #3 — the line the issue points at — changes nothing observable, because nothing upstream would have had anything to pass.

Fix

An ExecutionOrigin value object threaded backend → _trigger_handler → _execute_manual_trigger → _execute_schedule_with_lock → create_execution.

One object rather than five parallel parameters at four call depths: five positional siblings is exactly how one of them silently stops being forwarded, which is the failure mode already on display here.

Cron ticks stay NULL. Attributing an autonomous fire to, say, the schedule's owner would make the column actively misleading — a blank reads as "unknown", a wrong name does not. Pinned by a test, because it is the half of the behaviour that must not change.

Two more paths the issue's fix would have left blank

Neither is mentioned in the issue, and both would have stayed anonymous after a literal reading of it:

  • Retry (triggered_by='retry') inherits the original run's origin. A retry has no caller of its own, but it exists only because someone started the original; a chain that drops the initiator makes attempt 1 the only attributable attempt. The read is fail-open — an audit lookup must not be able to stop a retry from running.
  • Reminder (triggered_by='reminder') inherits the provenance feat: agent self-reminders — agent-callable one-shot deferred self-trigger ("remind me to do X later") #1296 already persisted (owner_id, created_by_email, source_agent_name, source_mcp_key_id). This needed those four fields surfaced on the scheduler's Reminder model; the mapper guards them the way _row_to_execution guards its optional columns, so a table predating them doesn't take the fire path down.

Hardening the issue didn't ask for

Boundary validation of the trigger body. The scheduler's trigger endpoint takes an untrusted JSON body (as it already did for triggered_by). source_user_id is dropped rather than coerced when it is not an int — bool IS an int in Python, so True would have persisted as user 1: a real account attributed to a run it had nothing to do with. Strings are length-capped and blank-to-None, so "" and NULL are not two spellings of "unknown". A malformed body costs attribution, never the run.

Precedence flipped versus chat.py. The backend prefers the validated current_user.agent_name over the raw X-Source-Agent header:

"source_agent_name": current_user.agent_name or x_source_agent,

Header-first is fine for a collaboration hint; in an audit column it would let an agent-scoped caller pin its run on a sibling agent. The header now fills only the case where it is the sole signal (a user-scoped key, whose agent_name is None).

MCP third surface (Invariant #13). trigger_agent_schedule now forwards the same origin headers chat() already sends. Without it an MCP-triggered run attributes to the key owner but not to which key or which agent fired it — the part that identifies the actor when one human owns many of both. The generic extraHeaders hook this required explicitly refuses Authorization, so an attribution convenience can't become an auth-substitution surface.

Compatibility

Backward compatible in both rolling-deploy directions:

  • old scheduler + new backend → the extra body fields are ignored, triggered_by still read as before;
  • new scheduler + old backend → a bodyless POST parses to an empty origin, i.e. today's behaviour.

All five parameters are keyword-optional, so an un-migrated create_execution call site still inserts a valid row.

Verification

tests/unit/test_1970_execution_origin.py27 checks, 25 of which fail against the pre-fix tree:

$ git stash push -- src/ && pytest unit/test_1970_execution_origin.py -q
25 failed, 2 passed
$ git stash pop && pytest unit/test_1970_execution_origin.py -q
27 passed

The 2 that pass either way are deliberate: the cron-stays-NULL guard and the optional-parameter back-compat guard both assert behaviour the fix must preserve, so passing before and after is the correct result for them.

The DB assertions run against a real temp SQLite file whose table declares the five columns — a mock would have happily accepted the pre-fix signature, and a table without the columns would let a regressed INSERT pass by having nothing to fill.

Also green: tsc --noEmit on the MCP server, and the scheduler-adjacent suites (test_1808, test_schedule_status_observability, test_1945, test_1713 — 70 tests).

Acceptance criteria

  • create_execution() accepts and writes all five source_* columns
  • The backend forwards the authenticated caller across the scheduler hop
  • _trigger_handler reads it and passes it down
  • Cron-triggered runs keep passing NULL, which is correct for them

Sibling issues

#1968 (execution_id: undefined) and #1969 (lock-denied cron tick leaves no record) are untouched here. The issue notes both are "fixed most cleanly by creating the execution record synchronously in _trigger_handler()" — a restructure of the trigger path that this PR deliberately does not attempt, so the attribution fix can land without carrying that risk. The ExecutionOrigin object is already in scope at _trigger_handler, so it composes with that change rather than competing with it.

Related to #1970

🤖 Generated with Claude Code

Fixes #1970

`schedule_executions` has carried five origin columns for audit since
AUDIT-001, and the backend populates them on every path it owns. The
scheduler is a separate service with its own DB module, and its
`create_execution()` listed none of them in the INSERT — nor accepted
them in its signature, so there was nowhere to put a caller even if one
had been forwarded. Every scheduler-created row was written with all
five NULL.

`triggered_by='manual'` therefore recorded *that* a human ran something
and never *who*. The attribution lived only in backend/MCP-server logs,
bounded by log retention, so past a few weeks the durable record could
not answer "did anyone trigger this run, and who?".

The identity was dropped at three points, not one:

  1. the backend's delegating POST sent no body at all, so the
     authenticated caller — in scope right there — never crossed the hop;
  2. `_trigger_handler` had no parameter to receive one;
  3. `create_execution()` had nowhere to put it.

An `ExecutionOrigin` value object is threaded through all three. One
object rather than five parallel parameters at four call depths: five
positional siblings is how one of them silently stops being forwarded.

Two paths the DB fix alone would have left blank are covered too. A
retry inherits the original run's origin — it has no caller of its own,
but a chain of retries that drops the initiator makes the first attempt
the only attributable one; the read is fail-open, since an audit lookup
must not be able to stop a retry from running. A reminder inherits the
provenance #1296 already persisted.

Cron ticks stay NULL. Attributing an autonomous fire to, say, the
schedule's owner would make the column actively misleading — a blank
reads as "unknown", a wrong name does not.

Also hardened while here:

- the untrusted trigger body is validated at the scheduler boundary.
  `source_user_id` is dropped rather than coerced when it is not an int:
  `bool` IS an `int` in Python, so `True` would have persisted as user
  1, a real account attributed to a run it had nothing to do with.
  Strings are length-capped and blank-to-None, so "" and NULL are not
  two spellings of "unknown".
- the backend prefers the validated `current_user.agent_name` over the
  raw `X-Source-Agent` header — the reverse of chat.py's precedence,
  which is fine for a collaboration hint but would let a caller pin its
  run on a sibling agent in an audit column.
- the MCP trigger tool forwards the origin headers `chat()` already
  sends (Invariant #13). Without it an MCP-triggered run attributes to
  the key OWNER but not to which key or agent fired it — the part that
  identifies the actor when one human owns many of both.

Not a vulnerability: nothing authorizes on these columns.

Backward compatible in both rolling-deploy directions — an old scheduler
ignores the new body fields, and a new scheduler treats a bodyless POST
as an unattributed manual trigger.

tests/unit/test_1970_execution_origin.py — 27 checks, 25 of which fail
against the pre-fix tree.

Related to #1970

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

⚠️ Nightly unit-suite check skipped — merge conflict against dev.

Resolve by running git merge dev locally and pushing the result. The next nightly run will re-test once the conflict is gone.

@dolho

dolho commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

/review — self-review (PR #1974, #1970)

Branch: fix/1970-scheduler-execution-origindev · 9 files, +834/−23
Scope: CLEAN. The issue's suggested fix named 3 files; the diff touches 7 source files, all justified — the two extra non-cron paths (retry, reminder) and the MCP third surface (Invariant #13). No unrelated changes.
Plan completion: 4 AC DONE, 0 partial.

Self-review, so discount accordingly. Nothing blocking found.

[I1] The retry-origin read is a new DB call on the retry path (Confidence: 8/10)

original = self.db.get_execution(original_execution_id)

Trading "the retry is unattributed" for "the retry never runs" would be a bad deal, so this is wrapped fail-open with a WARNING. Worth naming because the first version I wrote had the if original guard but no try — a missing row was handled, a failing read was not. Those are different failures.

[I2] Attribution fields are caller-supplied at the scheduler boundary (Confidence: 7/10)

/api/schedules/{id}/trigger has no auth (platform network only), so a caller on that network could post a false source_user_id. Not a new exposuretriggered_by has been caller-supplied on this same endpoint all along — and nothing authorizes on these columns. Mitigated at the boundary: wrong types are dropped rather than coerced (bool IS an int in Python, so True would otherwise persist as real user 1) and strings are length-capped. Recording it explicitly so "audit column" is not later mistaken for "trustworthy identity".

[I3] The 255-char cap is not derived from the column (Confidence: 6/10)

source_user_email/source_mcp_key_name are TEXT — no DB constraint to match. 255 is a judgement call, not a derived bound. Fine, but it is a magic number.

Clean categories

  • SQL safety — the INSERT stays qmark-parameterised; the _PgCursor ?%s rewrite covers the 5 new placeholders.
  • Race conditions — no new shared state; ExecutionOrigin is a frozen-in-practice value object passed by argument, never mutated.
  • Auth boundaries — no new endpoint. The backend endpoint keeps AuthorizedAgent; the three X-* headers are attribution only, and precedence deliberately prefers the validated current_user.agent_name over the raw header (the reverse of chat.py) so a caller cannot pin its run on a sibling agent.
  • Credential exposure — no values logged; the scheduler log line carries the user id, not the email, because it lands in Vector-captured platform logs.
  • Enum completeness — no new enum/status values.
  • Migrations — none needed; all five columns already exist (AUDIT-001).

Summary

  • Critical: 0
  • Informational: 3 — none blocking
  • Scope: clean

CI: 18/18 green.

…ases)

Two real bugs in this PR's own `ExecutionOrigin`, found by a boundary +
property pass over `from_payload` and fixed here.

**1. Range was never checked.** The parser dropped wrong-TYPED values and
capped string LENGTH, but accepted any `int`. `source_user_id` lands in an
INTEGER column, and SQLite raises `OverflowError: Python int too large to
convert to SQLite INTEGER` rather than truncating — verified end-to-end
against a real `create_execution`. On this PR's path the exception lands in
`_execute_manual_trigger`'s blanket `except`, so the run is logged and
silently lost *after* the endpoint already answered `"triggered"`. A field
of an untrusted payload decided whether the schedule ran, which is the exact
thing this function exists to prevent — the range check is the missing
sibling of a guard that was otherwise present.

**2. `is_empty()` used truthiness.** `not any((self.user_id, ...))` reports
an origin carrying `user_id=0` as empty. Hypothesis shrank it to
`{"source_user_id": 0}`. Latent — nothing in `src/` calls it yet — which is
why it was worth fixing now, before the first caller inherited a helper that
lies about a valid value.

Both lived inside code with **100% statement and branch coverage** from
`test_1970_execution_origin.py`. Full coverage says every line ran, not that
every value class was tried; that gap is the whole reason for a separate
edge-case pass.

tests/unit/test_execution_origin_properties.py — 31 boundary rows + 6
Hypothesis properties.

Related to #1970

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dolho

dolho commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

/edge-cases — 3 real bugs found across today's PRs (2 in this one)

Boundary-value analysis + Hypothesis over ExecutionOrigin.from_payload, the validation boundary for the scheduler's unauthenticated manual-trigger endpoint. Both bugs below were in code that already had 100% statement and branch coverage from test_1970_execution_origin.py — full coverage says every line ran, not that every value class was tried.

[B1] Range was never checked — a payload field could silently lose a run

from_payload dropped wrong-typed values and capped string length, but accepted any int. source_user_id lands in an INTEGER column, and SQLite raises rather than truncating:

OverflowError: Python int too large to convert to SQLite INTEGER

Verified end-to-end against a real create_execution. On this PR's path the exception lands in _execute_manual_trigger's blanket except — so the run is logged and silently lost, after the endpoint already answered "triggered". (On #1976's path it is caught and 500s.)

The range check is the missing sibling of a guard that was otherwise present — the function already drops unusable input for exactly this reason.

[B2] is_empty() used truthiness — user_id=0 read as no attribution

not any((self.user_id, ...))0 is falsy. Hypothesis shrank it to {"source_user_id": 0}.

Latent: nothing in src/ calls it yet. That is exactly why it was worth fixing now, rather than leaving a public helper that lies about a valid value for the first caller to gate on.

Fixed here

Both, in 8c8f0b1, with tests/unit/test_execution_origin_properties.py — 31 boundary rows + 6 properties (total function, output types match the columns, strings stripped/capped/never-blank, is_empty agrees with the fields, parsing idempotent and non-mutating).

Branch coverage of src/scheduler/models.py: 100% (158 stmts, 6 branches, 0 partial) — before and after, which is the point.

Method notes

  • Mutation gate not run (--mutate is off by default); the coverage-vs-bugs result above already makes the honest case better than a mutation score would.
  • hypothesis==6.161.5 is already a pinned dep in tests/requirements-test.txt; the local venv was just stale. No new dependency.

…sts/registry.json — rebuild: dev entries + this PR's two entries)

@vybe vybe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validated via /validate-pr — complete three-surface fix (backend router + scheduler + MCP client/tool), 25/27 regression tests fail pre-fix plus Hypothesis properties, retry inheritance and rolling-deploy compatibility verified both directions, extraHeaders hook refuses Authorization. I added the missing 'Fixes #1970' keyword and resolved the registry.json conflict (rebuild: dev + this PR's two entries); scheduler/service.py auto-merged cleanly against #1975 as predicted. Full CI green post-rebase. Note: stacked #1976 auto-retargets to dev on branch delete. Known follow-up: architecture.md schedule_executions DDL omits the source_* columns (pre-existing drift).

@vybe
vybe merged commit 4e7372b into dev Aug 4, 2026
21 checks passed
dolho added a commit that referenced this pull request Aug 5, 2026
…uced

My rebase of this branch onto dev left `src/scheduler/models.py` with TWO
`class ExecutionOrigin` definitions: this PR's at line 110 and dev's copy
(#1974, as merged) at 187. Python keeps the last one, so the second shadowed
the first and both of this PR's fixes became dead code — the None-comparing
`is_empty` and the SQLite-range guard on `source_user_id`. Three
`test_execution_origin_properties` cases went red on head, correctly.

Git did not flag it. Both sides added the class at different offsets, so the
textual auto-merge took both hunks and reported no conflict — a semantic
duplicate that only a reader or an importer would notice.

Two things on my side let it through:

- I re-read only the files git marked as conflicted, not the whole merged
  result of a 4-commit rebase.
- My post-rebase check was `-k "1968 or ent326 or timeline or scheduler or
  executions"`, which does not match `test_execution_origin_properties`. The
  filter was narrower than the blast radius, so 129 tests passed and said
  nothing about the file I had just broken.

Kept the first copy: it is a strict superset (identical fields, `is_empty`
comparing against None so `user_id=0` is not reported empty, and the
`_SQLITE_INT_MIN/MAX` range check that stops an out-of-range id reaching the
INSERT and taking the dispatch down). Deleted dev's older copy.

Swept the other six branches I rebased for the same class of damage —
duplicate top-level defs in any changed .py — all clean.

350 passed across origin/scheduler/1968/1969/1970/execution.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dolho added a commit that referenced this pull request Aug 11, 2026
…uced

My rebase of this branch onto dev left `src/scheduler/models.py` with TWO
`class ExecutionOrigin` definitions: this PR's at line 110 and dev's copy
(#1974, as merged) at 187. Python keeps the last one, so the second shadowed
the first and both of this PR's fixes became dead code — the None-comparing
`is_empty` and the SQLite-range guard on `source_user_id`. Three
`test_execution_origin_properties` cases went red on head, correctly.

Git did not flag it. Both sides added the class at different offsets, so the
textual auto-merge took both hunks and reported no conflict — a semantic
duplicate that only a reader or an importer would notice.

Two things on my side let it through:

- I re-read only the files git marked as conflicted, not the whole merged
  result of a 4-commit rebase.
- My post-rebase check was `-k "1968 or ent326 or timeline or scheduler or
  executions"`, which does not match `test_execution_origin_properties`. The
  filter was narrower than the blast radius, so 129 tests passed and said
  nothing about the file I had just broken.

Kept the first copy: it is a strict superset (identical fields, `is_empty`
comparing against None so `user_id=0` is not reported empty, and the
`_SQLITE_INT_MIN/MAX` range check that stops an out-of-range id reaching the
INSERT and taking the dispatch down). Deleted dev's older copy.

Swept the other six branches I rebased for the same class of damage —
duplicate top-level defs in any changed .py — all clean.

350 passed across origin/scheduler/1968/1969/1970/execution.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vybe pushed a commit that referenced this pull request Aug 11, 2026
…ted (#1968) (#1976)

* fix(scheduler): record who initiated an execution (#1970)

`schedule_executions` has carried five origin columns for audit since
AUDIT-001, and the backend populates them on every path it owns. The
scheduler is a separate service with its own DB module, and its
`create_execution()` listed none of them in the INSERT — nor accepted
them in its signature, so there was nowhere to put a caller even if one
had been forwarded. Every scheduler-created row was written with all
five NULL.

`triggered_by='manual'` therefore recorded *that* a human ran something
and never *who*. The attribution lived only in backend/MCP-server logs,
bounded by log retention, so past a few weeks the durable record could
not answer "did anyone trigger this run, and who?".

The identity was dropped at three points, not one:

  1. the backend's delegating POST sent no body at all, so the
     authenticated caller — in scope right there — never crossed the hop;
  2. `_trigger_handler` had no parameter to receive one;
  3. `create_execution()` had nowhere to put it.

An `ExecutionOrigin` value object is threaded through all three. One
object rather than five parallel parameters at four call depths: five
positional siblings is how one of them silently stops being forwarded.

Two paths the DB fix alone would have left blank are covered too. A
retry inherits the original run's origin — it has no caller of its own,
but a chain of retries that drops the initiator makes the first attempt
the only attributable one; the read is fail-open, since an audit lookup
must not be able to stop a retry from running. A reminder inherits the
provenance #1296 already persisted.

Cron ticks stay NULL. Attributing an autonomous fire to, say, the
schedule's owner would make the column actively misleading — a blank
reads as "unknown", a wrong name does not.

Also hardened while here:

- the untrusted trigger body is validated at the scheduler boundary.
  `source_user_id` is dropped rather than coerced when it is not an int:
  `bool` IS an `int` in Python, so `True` would have persisted as user
  1, a real account attributed to a run it had nothing to do with.
  Strings are length-capped and blank-to-None, so "" and NULL are not
  two spellings of "unknown".
- the backend prefers the validated `current_user.agent_name` over the
  raw `X-Source-Agent` header — the reverse of chat.py's precedence,
  which is fine for a collaboration hint but would let a caller pin its
  run on a sibling agent in an audit column.
- the MCP trigger tool forwards the origin headers `chat()` already
  sends (Invariant #13). Without it an MCP-triggered run attributes to
  the key OWNER but not to which key or agent fired it — the part that
  identifies the actor when one human owns many of both.

Not a vulnerability: nothing authorizes on these columns.

Backward compatible in both rolling-deploy directions — an old scheduler
ignores the new body fields, and a new scheduler treats a bodyless POST
as an unattributed manual trigger.

tests/unit/test_1970_execution_origin.py — 27 checks, 25 of which fail
against the pre-fix tree.

Related to #1970

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scheduler): return a real execution_id, and 409 when nothing started (#1968)

`_trigger_handler` was fire-and-forget: it spawned the run with
`asyncio.create_task` and responded immediately, *before* the execution
record existed. So it had no id to return. The backend relayed the same
id-less fields, and the MCP tool interpolated the missing key — telling
every agent `Execution started with ID 'undefined'`, on every trigger,
while the execution ran fine. Callers could not correlate a trigger with
its run, poll it, or fetch its result; the workaround was to guess from
`list_recent_executions` by timestamp.

The same ordering hid a second problem. The response was emitted before
`_execute_manual_trigger` had even attempted the distributed lock, so a
trigger suppressed because the schedule was already running still
answered `"status": "triggered"`. A suppressed trigger and a real one
were byte-identical to the caller.

The handler now acquires the lock and creates the row synchronously,
then hands both to the background task. That makes two facts sayable
that simply did not exist yet at response time: which execution this is,
and whether one was started at all.

  * 200 carries a real `execution_id`, valid the moment the caller
    receives it — a fast poller must not 404.
  * 409 `already_running` replaces the false "triggered", with no id and
    no row, because nothing ran.

Exactly one row per trigger: `_execute_schedule_with_lock` takes the
pre-created execution and skips its own create. Two rows would hand the
caller an id naming a row that never runs while a second did the work.

Because a row can now exist before a gate decides not to run, an
abandoned run FAILs its pre-created row rather than leaving it `running`
forever — canary E-01's exact signature, and a task the UI would show
indefinitely.

The handler also now holds the lock across a DB write, which is new, so
every exit from that window releases it: creation raising, creation
returning None, the run raising, and normal completion — exactly once
each. A second release is the dangerous one, since a lock re-acquired by
the next run in between would be freed out from under it.

Relayed through the remaining surfaces:

  * the backend forwards `execution_id` (and records it on the audit row,
    so a trigger and its run are joinable after the fact) and maps 409
    rather than flattening it into "Failed to trigger schedule" — a
    worse lie than the original, since it claims failure where the
    schedule is healthily busy;
  * the MCP tool returns a structured `already_running` instead of
    throwing, so an agent gets a decision it can act on, and GUARDS the
    id instead of interpolating it — an older backend still omits the
    field, and swapping one confident lie for another is not a fix;
  * `ScheduleTriggerResult.execution_id` becomes optional. Typing it as
    a required `string` while the wire never sent it is precisely why
    the compiler stayed happy through every `undefined`;
  * the UI reads 409 as "already running" rather than "nothing was
    changed — try again", and the CLI prints the id it was already
    fetching and discarding.

tests/unit/test_1968_trigger_execution_id.py — 22 checks, 17 of which
fail against the pre-fix tree.

Related to #1968

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scheduler): keep a strong reference to the spawned trigger task (#1968)

Self-review finding on this PR. The event loop holds only a WEAK reference
to a task, so a bare `asyncio.create_task(...)` whose result nobody keeps
can be garbage-collected mid-flight — the asyncio docs say so outright.

The bare call predates this PR, but this PR changes what it costs. Before,
a collected task meant the run silently did not happen. Now the lock is
acquired and the execution row created BEFORE the task is spawned, so a
collected task strands a `running` execution whose id the caller already
holds and pins the schedule's lock until its Redis TTL.

Uses the `_inflight` set + `add_done_callback(discard)` shape the #1083
result-callback path already established
(`agent_server/services/result_callback.py`), so the set cannot grow
without bound.

Guarded by `test_the_spawned_task_is_strongly_referenced`, which checks the
reference is held in the window BEFORE the task runs — the only window
where collection is possible — and released after.

Related to #1968

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(learnings): record the create_task-owns-state class surfaced by #1968 review

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(scheduler): remove the duplicate ExecutionOrigin my rebase introduced

My rebase of this branch onto dev left `src/scheduler/models.py` with TWO
`class ExecutionOrigin` definitions: this PR's at line 110 and dev's copy
(#1974, as merged) at 187. Python keeps the last one, so the second shadowed
the first and both of this PR's fixes became dead code — the None-comparing
`is_empty` and the SQLite-range guard on `source_user_id`. Three
`test_execution_origin_properties` cases went red on head, correctly.

Git did not flag it. Both sides added the class at different offsets, so the
textual auto-merge took both hunks and reported no conflict — a semantic
duplicate that only a reader or an importer would notice.

Two things on my side let it through:

- I re-read only the files git marked as conflicted, not the whole merged
  result of a 4-commit rebase.
- My post-rebase check was `-k "1968 or ent326 or timeline or scheduler or
  executions"`, which does not match `test_execution_origin_properties`. The
  filter was narrower than the blast radius, so 129 tests passed and said
  nothing about the file I had just broken.

Kept the first copy: it is a strict superset (identical fields, `is_empty`
comparing against None so `user_id=0` is not reported empty, and the
`_SQLITE_INT_MIN/MAX` range check that stops an out-of-range id reaching the
INSERT and taking the dispatch down). Deleted dev's older copy.

Swept the other six branches I rebased for the same class of damage —
duplicate top-level defs in any changed .py — all clean.

350 passed across origin/scheduler/1968/1969/1970/execution.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(webhooks): a busy schedule is a healthy 202, not a 503 (#1968)

The blast-radius table listed three consumers of the scheduler's `/trigger`
endpoint and asserted none regress. There is a fourth —
`routers/webhooks.py`, the unauthenticated public trigger — and it is the one
that does.

With #1968's 409, `if response.status_code not in (200, 202)` caught a
HEALTHY, busy schedule and turned it into an ERROR log line plus a 503
'Trigger failed — try again later' to a public caller, advice that only hits
the same lock. Worse, the `except HTTPException` arm then called
`idempotency_service.fail(idem)`, releasing the #525 dedup claim for a delivery
that never failed — so a retry could fire a second execution the moment the
lock cleared. Before #1968 that delivery returned 202.

Now a 409 is recognised as its own outcome: INFO log, the claim COMPLETED with
a snapshot recording what happened, and 202 with
`status: "already_running"` and a message saying the delivery was coalesced
into the run in flight rather than claiming a fresh execution started. Genuine
scheduler errors (500/502/400) still surface as 503 — the carve-out is on 409
alone, and a test pins that it does not widen.

tests/test_webhook_triggers.py asserts `status_code in (202, 503)` throughout,
which is why CI accepted the regression silently, so the new tests name the
status instead of tolerating a set. They run in-process (TestClient + faked
scheduler) rather than against a live instance:

  carve-out removed:  3 failed, 3 passed
  as shipped:         6 passed

Two of the non-blocking notes, both one-liners:

* `_abandon_precreated_execution` now passes `expected_status=RUNNING` to a new
  optional CAS precondition on the scheduler's `update_execution_status`. It
  was safe by argument (every abandon gate runs before dispatch); #1082 exists
  to retire exactly that kind of argument, and the clause makes it safe by
  construction. Default None keeps every other caller byte-identical.
* `schedules.ts` logged `execution_id: undefined` above the `!result.execution_id`
  guard — the exact string this issue is named after. Moved below it.

Related to #1968

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants