feat: async prompt dispatch (kind:"prompt" leaf) - #170
Conversation
Add the design for a `kind:"prompt"` leaf that dispatches a free-form prompt over the existing async KEDA queue, mirroring the `solve` precedent. Realizes issue rossoctl#168. Key decisions (ADR-0028): - dedicated LeafResult discriminant `{status:"responded";text}`, not a `done`+text overload — preserves the union's one-discriminant→one- payload invariant - full `/turn` parity via a shared `executeTurn` core, parameterized only by session-open policy (createIfAbsent) so `/turn`'s 404 contract is preserved while the leaf gets create-or-resume - leaf-family model precedence; `/turn` sandbox model (no solve pool lease) No new queue, worker, scaler, route, or status endpoint; zero `/runs` route changes. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Extend the leaf-job-runner processOne return union with "responded" (a forced consequence of the new LeafResult member; a prompt leaf is a terminal success that writes a record + acks). leaf-result-store's responded mapping is a later task. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…sage runTurn becomes a thin createIfAbsent:false wrapper over executeTurn, so a prompt leaf can run the identical /turn stack. The 404-on-missing-session contract is preserved and now pinned by a regression test. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
DRY the assistant-message branch-usage summation that Task 3 had mirrored verbatim into executeTurn and runSolveLeaf. One source of truth in run-turn.ts; run-leaf.ts consumes it over the existing value-import edge (no new cycle). Both call sites keep their best-effort try/catch guard. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…ecuteTurn Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Add 'solved|responded' to poll_status terminal states and a Claim 7 that dispatches a bare kind:prompt async envelope and asserts status=responded with non-empty .text. Gated by ASYNC_LIVE_SMOKE=1. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Address final-review findings for kind:prompt leaves: - I1: resolveRunWorkload keeps the workloadId existence/404 gate but no longer injects sandboxPoolSelector for kind:prompt (executeTurn ignores it by design, ADR 0028); logs a warn instead of silently dropping it. - M2: split the stale spec §2 'no change' row — leaf-job-runner had a type-level touch (processOne return union + 'responded'). - M3: add an explicit max_tokens -> responded test for the prompt path. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
pdettori
left a comment
There was a problem hiding this comment.
✅ Reviewed — would APPROVE (self-approval blocked by GitHub, so posting as a comment).
Clean, well-documented async prompt-dispatch feature. Exactly one discriminant (prompt kind → responded status → text payload) threaded end-to-end, reusing every existing seam. Verified "responded" is terminal end-to-end: classifyOutcome uses a default-ack model, so no new branching was needed there. Model-selection precedence, the import type cycle guard, and the selector-drop boundary guard all check out, with thorough unit coverage. All CI green; 9 commits, all signed-off.
One non-blocking suggestion inline (CodeQL log-injection cleanup). The executeTurn err.message.includes("no session in backend") string-match is brittle but you've already noted the hermetic 404-contract test as a follow-up — right call.
Assisted-By: Claude Code
| // The workloadId still gates existence (404 above), but its pool selector is intentionally | ||
| // ignored here rather than injected and then silently dropped by executeTurn downstream. | ||
| if (record.sandboxSelector) { | ||
| console.warn(`workload '${body.workloadId}': sandbox pool selector ignored for kind:prompt leaf (ADR 0028)`); |
There was a problem hiding this comment.
CodeQL flags this as log injection (alert #45). Not practically exploitable — findWorkload is an exact-key lookup and workload names are validated against WORKLOAD_NAME (RFC-1123, ≤50 chars) at creation, so reaching this line means the id is already validated. But logging the stored record.workloadId instead of raw body.workloadId makes that safety self-evident and clears the alert without a suppression.
There was a problem hiding this comment.
Done in 6733c12 — the boundary warning now logs the resolved record.workloadId (the exact key findWorkload matched, already validated against WORKLOAD_NAME at creation) instead of the raw body.workloadId. Clears alert #45 without a suppression; no behavior change. The existing I1 test asserts only that the warn fires, so it stayed green.
…on (alert rossoctl#45) The kind:prompt boundary warning interpolated the raw request field body.workloadId. Log the resolved record.workloadId instead — the exact key findWorkload matched, already validated against WORKLOAD_NAME (RFC-1123, <=50 chars) at creation. Clears the alert without a suppression and makes the safety self-evident. No behavior change; the existing I1 test still asserts the warn fires. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
Summary
Adds a
kind:"prompt"leaf envelope that dispatches a free-form prompt through the existing async KEDA/Redis-Streams queue, so a/turn-style interaction can be backgrounded and polled for completion — mirroring the existingkind:"solve"precedent.Governing principle: cleanest design over fewest changes. Exactly one new discriminant is threaded end-to-end —
promptkind →respondedresult status →textpayload — reusing every existing seam (async enqueue, KEDAScaledJobdrain,/runs/statuspolling, terminal classification)./turnparity comes from a shared exportedexecuteTurn()core (parameterized only bycreateIfAbsent), withrunTurnreduced to a thin wrapper, plus a sharedsumBranchUsagehelper (no third copy).Design docs:
docs/specs/2026-08-25-async-prompt-dispatch-design.mdanddocs/adrs/0028-async-prompt-dispatch.md.What's in it
LeafEnvelopegainskind:"prompt"+prompt;LeafResultgains{ status:"responded"; text; usage? }(one status → one payload).leaf-result-storepersiststext: string | nullwith a dedicatedrespondedbranch intoResultRecord.executeTurn(input)extracted fromrunTurn; behavior-preserving — the/turn404 "no session in backend" contract is preserved bit-for-bit (createIfAbsent:false).runPromptLeafmapsstopReason→responded(withaborted/errorspecial-cased); wired intorunLeafwith an injectabledeps.executeTurn.isPromptEnvelopefolds intoisRunEnvelope; a bare{sessionId, kind:"prompt", prompt, async:true}enqueues →202 {status:"accepted"};handleLeafStatusemits therespondedwire shape. Zero/runsroute branching change.workloadIdon a prompt envelope still gates existence (404 if absent) but its pool selector is intentionally not injected — the API boundary logs a warning rather than injecting a selectorexecuteTurnwould silently drop (ADR 0028; prompt leaves inherit/turn's sandbox model, no per-leaf pool isolation).Testing
Full suite green (
make test, EXIT:0):@sh/knative-server179 passed,harness201 passed / 3 skipped, all other packages passed. Skips are the pre-existing gated live-model / live-Redis suites — not regressions. Includes new unit coverage for the prompt envelope, therespondedrecord, themax_tokens → respondedmapping, and the workload-gated-but-selector-ignored boundary.Follow-up (non-blocking)
executeTurn404-contract test with an injected backend, so the contract is asserted unconditionally in a Redis-less CI leg (the contract is currently also pinned byserver.ts:94).Resolves #168.
🤖 Generated with Claude Code