workers-ai-provider / Workers AI: gpt-oss leaks forced tool calls into message.content as text (empty tool_calls, finish_reason "stop")

[threepointone]

Summary

On Workers AI, the gpt-oss models (@cf/openai/gpt-oss-120b, @cf/openai/gpt-oss-20b) sometimes emit a forced tool call as raw JSON text in message.content instead of a structured tool_calls entry. When this happens:

• choices[0].message.content is a JSON string like {"name":"<something>","path":"..."}
• choices[0].message.tool_calls is [] (empty)
• choices[0].finish_reason is "stop" (not "tool_calls")

This breaks any OpenAI-compatible consumer (including workers-ai-provider + the Vercel AI SDK): a step that was supposed to force a tool call silently produces no executable tool call, and the "answer" is malformed JSON rather than either a tool call or real prose.

This is adjacent to but distinct from #560 (which is about the SDK→provider tool_choice mapping). #560 is fixed on the client by sending the named-function form; this issue is a serving-side harmony rendering bug that the client cannot fully fix.

Affected models

• @cf/openai/gpt-oss-120b
• @cf/openai/gpt-oss-20b

Other families I tested (Llama 3.1/3.3/4, Qwen QwQ-32B, Qwen3-30B, Gemma-4) do not exhibit this — they return structured tool_calls.

When it happens

It is load-bearing on fit: when the forced tool is a reasonable fit for the conversation, gpt-oss returns a correct structured tool call. The leak reproduces when the forced tool is a poor fit for the conversation (the model "wants" to answer in prose but is forced to call a tool).

Works correctly (well-fitting forced tool)

Request to POST /accounts/{account}/ai/run/@cf/openai/gpt-oss-120b:

{
  "messages": [{ "role": "user", "content": "What is 123 + 456? Use the calculator tool." }],
  "tools": [{ "type": "function", "function": { "name": "calculator", "description": "Add two numbers and return their sum.", "parameters": { "type": "object", "properties": { "a": { "type": "number" }, "b": { "type": "number" } }, "required": ["a", "b"] } } }],
  "max_tokens": 2000,
  "tool_choice": { "type": "function", "function": { "name": "calculator" } }
}

Response (correct — structured tool call, finish_reason: "tool_calls"):

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{ "id": "chatcmpl-tool-...", "type": "function", "function": { "name": "calculator", "arguments": "{\"a\": 123, \"b\": 456}" } }],
      "reasoning_content": "User asks to compute 123 + 456 using calculator tool..."
    },
    "finish_reason": "tool_calls"
  }]
}

Broken (ill-fitting forced tool)

Same endpoint, a prose-tempting prompt forced to call an unrelated tool:

{
  "messages": [
    { "role": "system", "content": "You are a warm coach." },
    { "role": "user", "content": "That went really well! How do you think I did?" }
  ],
  "tools": [{ "type": "function", "function": { "name": "read_skill_resource", "description": "Read a bundled resource file.", "parameters": { "type": "object", "properties": { "name": { "type": "string" }, "path": { "type": "string" } }, "required": ["name", "path"] } } }],
  "max_tokens": 2000,
  "tool_choice": { "type": "function", "function": { "name": "read_skill_resource" } }
}

Observed responses across repeated runs (note the empty tool_calls, finish_reason: "stop", and the tool call serialized as a JSON string in content):

// gpt-oss-120b, run 1
{ "message": { "content": "{ \"name\": \"read_skill_resource\", \"path\": \"coaching_feedback_template.txt\" }", "tool_calls": [] }, "finish_reason": "stop" }

// gpt-oss-120b, run 2  — "name" is a HARMONY CHANNEL, not a tool
{ "message": { "content": "{\"name\":\"analysis\",\"path\":\"No external resources needed.\"}", "tool_calls": [] }, "finish_reason": "stop" }

// gpt-oss-120b, run 3  — "name" is a ROLE
{ "message": { "content": "{\"name\":\"assistant\",\"path\":\"final\"}", "tool_calls": [] }, "finish_reason": "stop" }

// gpt-oss-20b, run 1
{ "message": { "content": "{\"name\":\"read_skill_resource\",\"path\":\"warm_coach_supporting_response.txt\"}", "tool_calls": [] }, "finish_reason": "stop" }

// gpt-oss-20b, run 2 — hallucinated tool name
{ "message": { "content": "{\"name\":\"warm_coach_feedback\",\"path\":\"feedback.txt\"}", "tool_calls": [] }, "finish_reason": "stop" }

The same happens when streaming ("stream": true): the JSON arrives as choices[0].delta.content text deltas (gpt-oss does stream via SSE for this case), with finish_reason: "stop" and no delta.tool_calls.

Why this is a serving-side bug

The harmony format routes tool calls through the commentary channel and final answers through the final channel. In the broken cases the model's reasoning_content shows it is reasoning about a prose answer, and the would-be tool call lands in the final channel as a JSON string instead of being parsed into a structured tool_calls entry. So the harmony parser (in the vLLM/serving layer) is not detecting/lifting the tool call when the model is "torn" between answering and calling the forced tool.

The unreliability of the name field is the key tell: it is variously the real tool name, a channel name ("analysis"), a role ("assistant"), or a hallucination. A correct harmony parse would never surface a channel/role name as a tool name.

Expected behavior

For a forced tool_choice ("required" or { "type": "function", "function": { "name": ... } }), gpt-oss should return the call in choices[0].message.tool_calls with finish_reason: "tool_calls", exactly as it does for well-fitting prompts — never serialize the call as content text with finish_reason: "stop".

Repro environment

• Direct REST: POST https://api.cloudflare.com/client/v4/accounts/{account}/ai/run/@cf/openai/gpt-oss-120b (and -20b)
• tool_choice as the named-function form or "required"
• Reproduces both non-streaming and streaming
• max_tokens: 2000 (not a truncation issue — finish_reason is "stop", and reasoning_content is fully present)

Client-side mitigation (and its limits)

In workers-ai-provider we added a conservative salvage: when a tool was forced, tool_calls is empty, and the content parses to JSON whose name matches a requested tool, we reinterpret it as a structured tool call (and report finishReason: "tool-calls"). We deliberately do not salvage when name is a channel/role/hallucination, to avoid fabricating bogus calls.

This only recovers the envelope. In the broken cases the arguments are typically hallucinated (e.g. {"path":"coaching_feedback.txt"} invented wholesale), so even a successful salvage yields low-quality arguments. The robust fix needs to be in the harmony parser server-side.

Ask

Fix the harmony tool-call parsing for gpt-oss so forced tool calls are always emitted as structured tool_calls with finish_reason: "tool_calls", instead of being serialized into the final/content channel as text.