Skip to content

Add function call resolution loop and withMessages() to PromptBuilder - #273

Open
gziolo wants to merge 11 commits into
trunkfrom
add/function-call-resolution-loop
Open

Add function call resolution loop and withMessages() to PromptBuilder#273
gziolo wants to merge 11 commits into
trunkfrom
add/function-call-resolution-loop

Conversation

@gziolo

@gziolo gziolo commented Aug 5, 2026

Copy link
Copy Markdown
Member

Fixes #272

What

This PR adds two things to PromptBuilder, as proposed in #272:

  1. withMessages(Message ...$messages) to append full messages to the conversation. It is the counterpart to withHistory(), which prepends. This gives developers a public way to continue a conversation, for example to build a manual function call loop.

  2. An automatic function call resolution loop, enabled with a resolver:

$result = AiClient::prompt('What is this site about?')
    ->usingFunctionDeclarations(...$declarations)
    ->usingFunctionCallResolver($resolver)
    ->usingMaxFunctionCallIterations(3) // default 5
    ->generateTextResult();

The example uses generateTextResult() on purpose. When the loop stops early, the final response usually contains function calls and no text, and generateText() throws in that case. generateTextResult() hands the response back so the caller can handle it.

The resolver interface

The extension point is a small interface with two methods:

interface FunctionCallResolverInterface
{
    public function canResolve(FunctionCall $functionCall): bool;
    public function resolve(FunctionCall $functionCall): FunctionResponse;
}

The two steps are separate on purpose. canResolve() must be free of side effects. It is called for every function call in a model response before any call is executed. This way, a round is either executed completely or handed back to the caller untouched. resolve() executes one call. Execution errors should be encoded in the returned FunctionResponse, so the model can react to them.

How the loop works

  • Each round executes the function calls requested by the model through the resolver, appends the model message and a user message with the function responses to a copy of the conversation, and requests a follow-up response. The builder's own message list is not changed.
  • The loop stops when the model answers without function calls (completed), when the round limit is reached (maxIterations), when the response looks incomplete (incompleteFunctionCalls, see below), or when the resolver cannot resolve a requested call (unresolvedFunctionCalls, so the caller can handle custom functions).
  • A response is treated as incomplete when its finish reason is not toolCalls (for example a truncated response that hit the token limit) or when a function call has no name. Such calls are never executed. Provider adapters must map their finish reasons correctly for this check; the official OpenAI provider still needs the follow-up described below.
  • When a resolver is set, model discovery also requires the chat history capability, because follow-up rounds send multi-message conversations.
  • BeforeGenerateResultEvent and AfterGenerateResultEvent fire for every round, so consumers can observe the loop or abort it.
  • Token usage is summed across all rounds.
  • The number of rounds, the stop reason, the resolved calls, and the full conversation are exposed under the functionCallResolution key of the additional data of the final result.
  • The loop follows the first response candidate and only applies to text generation. Other capabilities ignore the resolver.

Why in the SDK

WordPress core is adding an ability resolution loop to the WP AI Client (Trac ticket 64865, wordpress-develop PR #12658). Because the SDK had no way to append messages or run the loop, that PR captures the messages and the resolved model from BeforeGenerateResultEvent and calls the model directly for later rounds. With this PR, the WordPress side shrinks to a thin resolver: canResolve() checks that the function name maps to a registered ability, and resolve() executes it. The workaround disappears. Any other consumer of the SDK gets the same loop for free.

Note on scope

docs/REQUIREMENTS.md says the client must not include agents. My reading is that this loop is not an agent framework. It is a small, bounded loop on top of the existing function calling feature, and it stays fully under the control of the caller. Happy to adjust the docs wording as part of this PR if you agree, or to change the approach if you read the scope differently.

OpenAI provider follow-up

This PR also updates the SDK's OpenAI-compatible Chat Completions implementation. A user message with several function responses, which is what the loop sends after parallel function calls, is now expanded into one tool message per response. Mixing function responses with other parts in one message is rejected with a clear error. The official OpenAI provider, which uses the Responses API, still needs a separate follow-up to normalize parallel top-level function-call items into one candidate, preserve an incomplete response's finish reason, and expand parallel calls and responses into top-level input items. No official-provider code is changed here.

Testing

composer test

The new tests in tests/unit/Builders/PromptBuilderFunctionCallResolutionTest.php cover the happy path, transcript ordering and roles, multiple calls in one response, the no-execution guarantee when a call cannot be resolved, the iteration limit, token usage aggregation, zero-round completion, invalid options, non-text capabilities, and withMessages() ordering. composer lint (PHPCS and PHPStan) passes.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.89%. Comparing base (20a1a6d) to head (02b40a5).

Additional details and impacted files
@@             Coverage Diff              @@
##              trunk     #273      +/-   ##
============================================
+ Coverage     86.54%   86.89%   +0.34%     
- Complexity     1381     1416      +35     
============================================
  Files            69       69              
  Lines          4438     4555     +117     
============================================
+ Hits           3841     3958     +117     
  Misses          597      597              
Flag Coverage Δ
unit 86.89% <100.00%> (+0.34%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support in PromptBuilder for (1) appending messages to an existing conversation and (2) automatically resolving model-requested function calls via a bounded multi-round loop, returning loop metadata in the final result’s additional data.

Changes:

  • Add PromptBuilder::withMessages() to append full Message instances (counterpart to withHistory() which prepends).
  • Introduce FunctionCallResolverInterface and integrate an optional function call resolution loop into text generation, including token-usage aggregation and transcript exposure.
  • Add comprehensive unit tests plus supporting mocks/helpers for scripted multi-round text generation.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/Builders/PromptBuilder.php Adds withMessages(), resolver configuration, and the multi-round function call resolution loop with transcript + token aggregation.
src/Tools/Contracts/FunctionCallResolverInterface.php Defines the resolver extension point (canResolve() + resolve()) used by the loop.
tests/unit/Builders/PromptBuilderFunctionCallResolutionTest.php Covers loop behavior (happy path, transcript ordering, unresolved calls, max iterations, token aggregation, non-text capabilities) and withMessages() ordering.
tests/traits/MockModelCreationTrait.php Adds a scripted text-generation mock model helper to simulate multi-round responses deterministically.
tests/mocks/MockFunctionCallResolver.php Adds a mock resolver that records checked/resolved calls and can be customized via callbacks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/Builders/PromptBuilder.php Outdated
@jeffpaul jeffpaul added this to the 1.5.0 milestone Aug 26, 2026
gziolo and others added 5 commits September 3, 2026 12:02
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@gziolo
gziolo force-pushed the add/function-call-resolution-loop branch 2 times, most recently from 6fcc530 to 1d0f4df Compare September 3, 2026 10:03
gziolo and others added 5 commits September 3, 2026 13:17
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gziolo
gziolo marked this pull request as ready for review September 3, 2026 11:52
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: gziolo <gziolo@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Thought-token aggregation must preserve unknown usage instead of reporting a misleading partial total.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/Builders/PromptBuilder.php
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Type] Enhancement A suggestion for improvement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a function call resolution loop and a way to append messages to PromptBuilder

3 participants