Skip to content

fix(anthropic): strip FORMAT_TOOL from re-encoded history to fix resume() HTTP 400 - #2863

Open
devteamaegis wants to merge 3 commits into
Mirascope:mainfrom
devteamaegis:fix/anthropic-format-tool-resume-400-error
Open

fix(anthropic): strip FORMAT_TOOL from re-encoded history to fix resume() HTTP 400#2863
devteamaegis wants to merge 3 commits into
Mirascope:mainfrom
devteamaegis:fix/anthropic-format-tool-resume-400-error

Conversation

@devteamaegis

Copy link
Copy Markdown

Problem

response.resume() fails with HTTP 400 when using Anthropic with structured outputs (format=SomeModel):

BadRequestError: 400 - tool_use ids were found without tool_result blocks immediately after

Fixes #2503

Root cause

Anthropic structured-output mode injects a private FORMAT_TOOL_NAME tool. The model response contains a tool_use block for that tool. When the assistant message appears in the history of a resume() call:

  1. _encode_message detects the format tool (raw_message_has_format_toolTrue) and falls through to re-encode from content parts instead of using the raw message verbatim.
  2. encode_content encodes every tool_call part as a ToolUseBlockParam — including the FORMAT_TOOL_NAME call.
  3. Anthropic rejects the request because there is no tool_result in the following user message (there never is — the format tool is a Mirascope detail that is never actually invoked).

Fix

In encode_content, skip any tool_call part whose name starts with FORMAT_TOOL_NAME:

elif part.type == "tool_call":
    if part.name.startswith(FORMAT_TOOL_NAME):
        continue   # ← new guard
    blocks.append(ToolUseBlockParam(...))

This preserves all real tool_use blocks while stripping the implementation-detail format tool from re-encoded history.

Tests

Added to python/tests/llm/providers/anthropic/test_anthropic_encoding.py:

test what it checks
test_encode_content_skips_format_tool_call FORMAT_TOOL tool_use absent from re-encoded blocks
test_encode_content_keeps_real_tool_calls genuine tool_use blocks still emitted
pytest tests/llm/providers/anthropic/test_anthropic_encoding.py -v
3 passed

# the next user message — Anthropic rejects the request with 400:
# "tool_use ids were found without tool_result blocks immediately after"
# See https://github.com/Mirascope/mirascope/issues/2503
if part.name.startswith(FORMAT_TOOL_NAME):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this mean the content of this final tool call isn't included at all? Shouldn't we be re-encoding it as text in this case?

Per review (@willbakst): skipping the format-tool call entirely lost the
assistant's actual structured response. Now re-encode its arguments (the
JSON the model produced) as a text block — content preserved, and still no
orphaned tool_use that would 400 on resume. Test asserts both text parts.
@devteamaegis

Copy link
Copy Markdown
Author

Good catch @willbakst — you're right, continue dropped the content entirely. The format tool's arguments are the assistant's actual structured response, so instead of skipping I now re-encode them as a text block: the JSON the model produced is preserved in the history, and there's still no orphaned tool_use block to trigger the 400 on resume. Updated the test to assert both text parts survive (["Here is the result.", '{"value": 7}']). Tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

response.resume broken when using structured outputs with Anthropic in tool mode

2 participants