Skip to content

Classify sandbox execution failures on execute spans - #1726

Merged
RhysSullivan merged 1 commit into
mainfrom
execute-error-kind
Aug 24, 2026
Merged

Classify sandbox execution failures on execute spans#1726
RhysSullivan merged 1 commit into
mainfrom
execute-error-kind

Conversation

@RhysSullivan

Copy link
Copy Markdown
Collaborator

Sandbox script failures ride the MCP success channel as ExecuteResult.error, so mcp.execute spans read OK and the failure class is unqueryable — over the last 7 days zero spans carried a wrong-shape TypeError signature despite ~97k executions.

  • @executor-js/codemode-core: adds ExecuteErrorKind and an optional errorKind on ExecuteResult, classified from the thrown error's name (never message content).
  • dynamic-worker runtime: tags the tool-dispatch rethrow (ExecutionToolError) and in-sandbox deadline (ExecutionTimeoutError) with dedicated names, populates errorKind from the serialized error, and maps its typed failures (compile → syntax_error, runtime → serialization_error/resource_limit, host timeout → timeout).
  • execution engine: annotates mcp.execute / mcp.execute.resume spans with mcp.execute.outcome (ok/fail/paused) and mcp.execute.error_kind on failures.
  • prod-telemetry skill doc updated with the new attributes.

Error messages shown to the model are unchanged; only error names inside the sandbox differ (ExecutionToolError/ExecutionTimeoutError instead of Error).

Gives prod a queryable baseline for wrong-shape response guesses (error_kind=type_error) ahead of the inferred output-shape work.

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Cloudflare preview

Torn down — the PR is closed.

@pkg-pr-new

pkg-pr-new Bot commented Aug 24, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@1726

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@1726

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@1726

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@1726

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@1726

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@1726

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@1726

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@1726

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@1726

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@1726

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@1726

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@1726

executor

npm i https://pkg.pr.new/executor@1726

commit: b0c2f86

@RhysSullivan
RhysSullivan marked this pull request as ready for review August 24, 2026 23:41
@RhysSullivan
RhysSullivan merged commit 6cad3d2 into main Aug 24, 2026
42 of 44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant