GPTRunner discards text response when there is tool call

[vantaa89]

Hi, thank you for the great work!

While working on your benchmark, I discovered that GPTRunner discards the GPT's text output whenever a tool call occurs.

ComplexFuncBench/runner/gpt_runner.py

Line 62 in c37b284

self.model.messages.append({"role": "assistant", "content": None, "tool_calls": llm_response.tool_calls})

However, it seems that this is not the case for other models, such as claude

ComplexFuncBench/runner/claude_runner.py

Line 80 in c37b284

self.model.messages.append({"role": "assistant", "content": llm_response.content})

, which, in turn, discards the function call history while maintaining the text output. I am wondering if this is intended, or if I am misunderstanding the logic.

Thank you.

[lucenzhong]

You're right. The GPTRunner also needs to add content output. When we tested the gpt series models last year, tool_calls and content did not exist simultaneously.

Hi, thank you for the great work!

While working on your benchmark, I discovered that GPTRunner discards the GPT's text output whenever a tool call occurs.

ComplexFuncBench/runner/gpt_runner.py

Line 62 in c37b284

self.model.messages.append({"role": "assistant", "content": None, "tool_calls": llm_response.tool_calls})
However, it seems that this is not the case for other models, such as claude

ComplexFuncBench/runner/claude_runner.py

Line 80 in c37b284

self.model.messages.append({"role": "assistant", "content": llm_response.content})

, which, in turn, discards the function call history while maintaining the text output. I am wondering if this is intended, or if I am misunderstanding the logic.
Thank you.

You're right. The GPTRunner also needs to add content output. When we tested the gpt series models last year, tool_calls and content did not exist simultaneously.