Skip to content

refactor(optimizer): delegate model calls to runtime owners - #541

Merged
drewstone merged 3 commits into
mainfrom
refactor/runtime-owned-model-calls-20260803
Aug 4, 2026
Merged

refactor(optimizer): delegate model calls to runtime owners#541
drewstone merged 3 commits into
mainfrom
refactor/runtime-owned-model-calls-20260803

Conversation

@drewstone

@drewstone drewstone commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Closes #537

What changed

  • replace Eval-owned provider credentials and HTTP calls with one caller-owned canonical model callback
  • translate official GEPA, SkillOpt, and DSPy OpenAI-compatible child requests at the Eval boundary
  • require a typed canonical response, exact cost receipt, and finite execution evidence for every invoked call
  • persist callback identity and model execution observations in benchmark artifacts
  • remove the legacy direct-provider path and public benchmark credential flags

Contract

  • requests carry a stable call ID, deeply immutable canonical ChatRequest, abort signal, and endpoint format
  • responses carry canonical ChatResponse, CostReceiptInput, and JSON execution evidence
  • input tokens exclude cached tokens; cache-write tokens remain separate
  • owner failures are returned through the local child endpoint without leaking provider bodies or credentials

Proof

  • pnpm test: 4,742 passed, 3 skipped
  • pnpm typecheck; pnpm typecheck:examples; pnpm lint
  • pnpm build; pnpm verify:package
  • full pinned GEPA + SkillOpt Python suite passed
  • official GEPA and SkillOpt request fixtures captured and replayed, including SkillOpt dynamic system-role messages
  • DSPy bridge: 35 passed
  • local fake-provider boundary suite: 43 passed

tangletools
tangletools previously approved these changes Aug 4, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — f66212b7

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-08-04T02:05:19Z

tangletools
tangletools previously approved these changes Aug 4, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 03c46190

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-08-04T02:08:41Z

@drewstone
drewstone merged commit fd43de8 into main Aug 4, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(analyst): move DSPy RLM execution behind a caller-owned callback

2 participants