OpenAIResponsesChatClient drops content-level RawRepresentation for System and Developer messages

[TheGrayFrost]

Description

OpenAIResponsesChatClient discards content-level AIContent.RawRepresentation for System and Developer messages, while preserving it for User messages.

In ToOpenAIResponseItems the system/developer branch flattens the message to a single string and rebuilds the item from it:

if (input.Role == ChatRole.System ||
    input.Role == OpenAIClientExtensions.ChatRoleDeveloper)
{
    string text = input.Text;
    if (!string.IsNullOrWhiteSpace(text))
    {
        yield return input.Role == ChatRole.System ?
            ResponseItem.CreateSystemMessageItem(text) :
            ResponseItem.CreateDeveloperMessageItem(text);
    }

    continue;
}

Because the item is reconstructed from input.Text, any ResponseContentPart a caller attached as RawRepresentation on the message's contents is dropped. The User branch immediately below does honour it, both for contents that map to a whole ResponseItem ({ RawRepresentation: ResponseItem rawRep } => rawRep) and for contents that map to ResponseContentParts grouped via ResponseItem.CreateUserMessageItem(parts).

The failure is silent: no exception is thrown, the request succeeds, and the customization simply is not on the wire.

Why this matters

RawRepresentation is the documented escape hatch for provider-specific fields that Microsoft.Extensions.AI does not model. Setting it on a content part is the only supported way to add a property to an input_text part.

The concrete case that led us here is OpenAI's explicit prompt caching for GPT-5.6, which requires a prompt_cache_breakpoint property on a content part:

{
  "type": "input_text",
  "text": "...",
  "prompt_cache_breakpoint": { "mode": "explicit" }
}

Under prompt_cache_options.mode = "explicit" nothing is cached unless a content part carries that marker, and a breakpoint caches everything up to and including the part it sits on. The natural placement for the first breakpoint is the end of the system prompt. That is currently impossible through MEAI — the property is stripped before serialization, and the request caches nothing.

The service itself accepts a breakpoint on a system message: sending the same body by hand to /openai/v1/responses on a gpt-5.6-terra deployment produced cached_tokens: 9106. Only the MEAI conversion path loses it.

Reproduction

Minimal console app, net10.0, referencing Microsoft.Extensions.AI.OpenAI 10.6.0 and OpenAI 2.10.0. A capturing HttpMessageHandler records the outgoing body; no network access or API key is needed.

using System.ClientModel;
using System.ClientModel.Primitives;
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Responses;

class CapturingHandler : HttpMessageHandler
{
    public string? Body;
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage req, CancellationToken ct)
    {
        Body = req.Content is null ? "" : await req.Content.ReadAsStringAsync(ct);
        return new HttpResponseMessage(System.Net.HttpStatusCode.OK)
        {
            Content = new StringContent(
                """{"id":"r","object":"response","created_at":0,"status":"completed","model":"m","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}""",
                Encoding.UTF8, "application/json"),
        };
    }
}

static class Program
{
    // Adds an arbitrary property to a ResponseContentPart by round-tripping its JSON,
    // because IJsonModel<T>.Create replaces rather than merges.
    static T AugmentJsonModel<T>(T model, string key, object value) where T : IJsonModel<T>
    {
        using var doc = JsonDocument.Parse(ModelReaderWriter.Write(model, ModelReaderWriterOptions.Json));
        using var ms = new MemoryStream();
        using (var writer = new Utf8JsonWriter(ms))
        {
            writer.WriteStartObject();
            foreach (var p in doc.RootElement.EnumerateObject())
            {
                p.WriteTo(writer);
            }

            writer.WritePropertyName(key);
            JsonSerializer.Serialize(writer, value);
            writer.WriteEndObject();
        }

        return ModelReaderWriter.Read<T>(new BinaryData(ms.ToArray()), ModelReaderWriterOptions.Json)!;
    }

    static TextContent Marked(string text)
    {
        var part = AugmentJsonModel(
            ResponseContentPart.CreateInputTextPart(text),
            "prompt_cache_breakpoint",
            new Dictionary<string, string> { ["mode"] = "explicit" });

        return new TextContent(text) { RawRepresentation = part };
    }

    static async Task Main()
    {
        var handler = new CapturingHandler();
        var options = new OpenAIClientOptions
        {
            Endpoint = new Uri("https://example.invalid/openai/v1"),
            Transport = new HttpClientPipelineTransport(new HttpClient(handler)),
        };

        IChatClient client = new OpenAIClient(new ApiKeyCredential("k"), options)
            .GetResponsesClient()
            .AsIChatClient("gpt-5.6-terra");

        await client.GetResponseAsync(
        [
            new ChatMessage(ChatRole.System, new List<AIContent> { Marked("SYSTEM") }),
            new ChatMessage(ChatRole.User,   new List<AIContent> { Marked("USER") }),
        ]);

        Console.WriteLine(handler.Body);
    }
}

Actual output

{"model":"gpt-5.6-terra","input":[
  {"type":"message","role":"system","content":[{"type":"input_text","text":"SYSTEM"}]},
  {"type":"message","role":"user","content":[{"type":"input_text","text":"USER","prompt_cache_breakpoint":{"mode":"explicit"}}]}
]}

Both messages had an identical RawRepresentation attached. It survives on user and is gone on system.

The same happens for ChatRoleDeveloper.

Expected output

prompt_cache_breakpoint present on the system content part as well.

Suggested fix

Have the system/developer branch reuse the caller's ResponseContentPart when one is present, instead of unconditionally rebuilding from input.Text — mirroring the parts handling already used for user messages. Something along the lines of collecting parts from input.Contents (falling back to CreateInputTextPart when RawRepresentation is not a ResponseContentPart) and passing them to the system/developer item factory.

Note that flattening to input.Text also silently coalesces multiple TextContent items into one part, which may be worth preserving separately.

Workaround

Insert a synthetic user message containing "." immediately after the system prompt and put the breakpoint on that instead, since the user path preserves RawRepresentation. This works but adds a real token to every request and puts a spurious turn in the conversation.

Environment

Microsoft.Extensions.AI	10.6.0
Microsoft.Extensions.AI.Abstractions	10.6.0
Microsoft.Extensions.AI.OpenAI	10.6.0
OpenAI	2.10.0
TFM	net10.0
OS	macOS (arm64)

Also reproduces against Azure OpenAI (*.openai.azure.com) via AzureOpenAIClient, since the conversion is shared.