diff --git a/docs/web-search-provider-capability.md b/docs/web-search-provider-capability.md index 7d8c68cf5a..748e2f5e01 100644 --- a/docs/web-search-provider-capability.md +++ b/docs/web-search-provider-capability.md @@ -19,10 +19,11 @@ # Provider-hosted web search capability -Status: implemented for OpenAI Responses and Anthropic Messages -`web_search_20250305`; other provider-native wires remain explicit follow-ups. +Status: implemented for OpenAI Responses, Alibaba Token Plan Open Responses, +and Anthropic Messages `web_search_20250305`; other provider-native wires +remain explicit follow-ups. Verified against public provider documentation and shipped client/SDK behavior -on 2026-08-04. +through 2026-08-24. ## Problem @@ -59,8 +60,15 @@ set `capabilities.webSearch`; otherwise narrow provider/model rules apply. The implemented native adapters are: - `openai-responses` for Codex-style `web_search`; +- `openai-responses` over the Alibaba Token Plan Open Responses profile for + `qwen3.8-max` Harness `web_search`; - `anthropic-messages` for Claude Code-compatible `web_search_20250305`. +Alibaba Token Plan stays deliberately narrower than the provider's published +model matrix: the current Runtime routes only `qwen3.8-max` through its verified +Responses profile. Qwen 3.7 models remain on Chat in Maka and therefore do not +receive a provider-native search tool yet. + ## Execution surfaces All production `AiSdkBackend` composition roots use the same @@ -105,8 +113,11 @@ turn-start tool surface The search request uses the existing model credential. It does not duplicate the secret into web-search settings, start a nested model request, or expose the secret to the renderer. Provider search call/results are marked -`providerExecuted`, preserved separately from local ToolRuntime execution, and -replayed through the native provider-tool shape. +`providerExecuted` and preserved separately from local ToolRuntime execution. +Adapters that can round-trip the pair replay its native provider-tool shape. +Alibaba Open Responses currently keeps the durable episode and grounded text but +omits the pair from the next provider request, because that adapter cannot yet +round-trip provider-executed results without producing a dangling output. The durable event keeps normalized `result` data for the canonical read model, UI, and exports, plus opaque `providerOutput` data for provider-protocol replay. @@ -125,6 +136,15 @@ stream. DeepSeek returned search actions but no structured source rows or URL annotations in that response, so Maka preserves citations when supplied but does not synthesize or invent them. +After DeepSeek moved to the plaintext `@ai-sdk/open-responses` adapter, native +search was temporarily disabled because that adapter dropped provider-defined +tools. Live revalidation on 2026-08-24 against first-party +`deepseek-v4-flash` completed six provider-executed actions (`search`, +`open_page`, and `find_in_page`) and a grounded final answer. The shared Open +Responses mapper preserves those actions while the existing replay boundary +continues to omit the provider-owned pair from the next request instead of +emitting a dangling function output. + The same key was also verified through DeepSeek's Anthropic-compatible endpoint: one real Maka turn sent `web_search_20250305`, received one provider-executed WebSearch result containing ten source rows, and completed the final answer @@ -187,7 +207,8 @@ search-heavy workflows that value source visibility over cache economics. | OpenAI API | Responses `web_search` tool | Maka currently enables the native path for GPT-5 families, whose runtime wire is already Responses | Integrated through `openai-responses` | | Custom Responses relay | Responses `web_search` tool when explicitly declared by model metadata | `openai-responses-compatible` connections with `apiProtocol=openai-responses` and `capabilities.webSearch=true` | Integrated through `openai-responses` | | xAI API / OAuth | Responses Agent Tools `web_search` | Maka currently enables the verified Grok 4.5 Responses route | Integrated through `openai-responses` | -| Alibaba Model Studio | Responses `web_search` | Qwen 3.5 Plus/Flash provider support is recorded | Provider supports it; Maka Responses adapter pending | +| Alibaba Token Plan | Responses Harness `web_search` | `qwen3.8-max` on the Token Plan China/Singapore access paths | Integrated through the Alibaba Open Responses profile; live smoke pending | +| Alibaba Model Studio pay-as-you-go | Responses `web_search` | Provider model and deployment support varies | Provider supports it; Maka pay-as-you-go Responses adapter pending | | Anthropic / Claude subscription | Messages `web_search_20250305` | Current Claude Opus/Sonnet/Haiku/Fable families | Integrated through `anthropic-messages` | | MiniMax API / Coding Plan | Anthropic-compatible `web_search_20250305` | MiniMax M2.7/M3 families | Integrated through `anthropic-messages`; live provider verification pending | | Google Gemini | Gemini API grounding with Google Search | Supported Gemini 2.0+ model families vary by release | Provider supports it; Maka adapter pending | diff --git a/packages/core/src/__tests__/model-metadata.test.ts b/packages/core/src/__tests__/model-metadata.test.ts index 9eb6090bf8..4e1c54b4e7 100644 --- a/packages/core/src/__tests__/model-metadata.test.ts +++ b/packages/core/src/__tests__/model-metadata.test.ts @@ -104,4 +104,12 @@ describe('openAiAdapterApiProtocol', () => { assert.equal(openAiAdapterApiProtocol('muse-spark-1.2-contributor', 'opencode'), 'openai-chat'); assert.equal(openAiAdapterApiProtocol('minimax-m3', 'opencode-go'), 'openai-chat'); }); + + it('routes only Qwen3.8 Max through Alibaba Token Plan Responses', () => { + for (const providerType of ['alibaba-token-plan-cn', 'alibaba-token-plan'] as const) { + assert.equal(openAiAdapterApiProtocol('qwen3.8-max', providerType), 'openai-responses'); + assert.equal(openAiAdapterApiProtocol('qwen3.7-max', providerType), 'openai-chat'); + } + assert.equal(openAiAdapterApiProtocol('qwen3.8-max', 'alibaba-cn'), 'openai-chat'); + }); }); diff --git a/packages/core/src/__tests__/model-web-search.test.ts b/packages/core/src/__tests__/model-web-search.test.ts index 47b13b2953..252d754de8 100644 --- a/packages/core/src/__tests__/model-web-search.test.ts +++ b/packages/core/src/__tests__/model-web-search.test.ts @@ -26,11 +26,11 @@ describe('hosted web search capability', () => { it('enables the implemented Responses path only for declared model families', () => { assert.deepEqual(resolveHostedWebSearchCapability('deepseek', undefined, 'deepseek-v4-flash'), { adapter: 'openai-responses', - implemented: false, + implemented: true, }); assert.deepEqual(resolveHostedWebSearchCapability('deepseek', undefined, 'deepseek-v4-pro'), { adapter: 'openai-responses', - implemented: false, + implemented: true, }); assert.equal(resolveHostedWebSearchCapability('deepseek', undefined, 'deepseek-chat'), null); assert.deepEqual(resolveHostedWebSearchCapability('openai', undefined, 'gpt-5.5'), { @@ -49,6 +49,19 @@ describe('hosted web search capability', () => { adapter: 'openai-responses', implemented: false, }); + for (const providerType of ['alibaba-token-plan-cn', 'alibaba-token-plan'] as const) { + assert.deepEqual(resolveHostedWebSearchCapability(providerType, undefined, 'qwen3.8-max'), { + adapter: 'openai-responses', + implemented: true, + }); + assert.equal(resolveHostedWebSearchCapability(providerType, undefined, 'qwen3.7-max'), null); + assert.equal(resolveHostedWebSearchCapability(providerType, undefined, 'qwen3.7-plus'), null); + assert.equal( + resolveHostedWebSearchCapability(providerType, undefined, 'qwen3.8-max-preview'), + null, + ); + assert.equal(resolveHostedWebSearchCapability(providerType, undefined, 'qwen3.6-plus'), null); + } assert.equal(resolveHostedWebSearchCapability('openai', undefined, 'gpt-4.1'), null); }); @@ -124,7 +137,7 @@ describe('hosted web search capability', () => { it('keeps dual-wire providers on the configured connection protocol', () => { assert.deepEqual(resolveHostedWebSearchCapability('deepseek', undefined, 'deepseek-v4-flash'), { adapter: 'openai-responses', - implemented: false, + implemented: true, }); assert.deepEqual( resolveHostedWebSearchCapability( @@ -155,6 +168,14 @@ describe('hosted web search capability', () => { ), null, ); + assert.equal( + resolveHostedWebSearchCapability( + 'alibaba-token-plan-cn', + [{ id: 'qwen3.8-max', apiProtocol: 'openai-chat' }], + 'qwen3.8-max', + ), + null, + ); }); it('publishes native search for both first-party DeepSeek V4 models', () => { diff --git a/packages/core/src/__tests__/provider-catalog-contract.test.ts b/packages/core/src/__tests__/provider-catalog-contract.test.ts index d5a57da1f0..ef03f43fa2 100644 --- a/packages/core/src/__tests__/provider-catalog-contract.test.ts +++ b/packages/core/src/__tests__/provider-catalog-contract.test.ts @@ -78,6 +78,19 @@ describe('provider catalog contract — structural invariants over CATALOG_PROVI ); } }); + + it('delegates Alibaba Token Plan execution through one explicit Runtime profile', () => { + const delegated = Object.entries(PROVIDER_REGISTRY).flatMap(([providerType, definition]) => { + const adapter = definition.runtimeAdapter; + return adapter.kind === 'openai-compatible' && adapter.runtimeProfile + ? [{ providerType, runtimeProfile: adapter.runtimeProfile }] + : []; + }); + assert.deepEqual(delegated, [ + { providerType: 'alibaba-token-plan-cn', runtimeProfile: 'alibaba-token-plan' }, + { providerType: 'alibaba-token-plan', runtimeProfile: 'alibaba-token-plan' }, + ]); + }); }); describe('retired provider contract', () => { diff --git a/packages/core/src/llm-connections.ts b/packages/core/src/llm-connections.ts index 75bbe31742..199fb601f5 100644 --- a/packages/core/src/llm-connections.ts +++ b/packages/core/src/llm-connections.ts @@ -44,6 +44,7 @@ import { type ProviderCategory, type ProviderDefaults, type ProviderRuntimeAdapter, + type ProviderRuntimeProfileId, type ProviderResponsesContract, type ProviderType, } from './provider-registry.js'; @@ -64,6 +65,7 @@ export type { ProviderCategory, ProviderDefaults, ProviderRuntimeAdapter, + ProviderRuntimeProfileId, ProviderResponsesContract, ProviderType, }; diff --git a/packages/core/src/model-metadata.ts b/packages/core/src/model-metadata.ts index 5d65b8fc4a..79f9acbeb7 100644 --- a/packages/core/src/model-metadata.ts +++ b/packages/core/src/model-metadata.ts @@ -125,11 +125,10 @@ export function lookupModelProviderOverride( /** * The request wire a model served over the OpenAI adapter must use. * - * OpenAI's `gpt-5*` families and xAI's `grok-4.5` are served only over the - * Responses API; every other model on the native OpenAI adapter uses Chat - * Completions. This is the single declared source of that protocol split, - * expressed through the {@link ModelInfo.apiProtocol} seam. It is consumed by - * the runtime model factory and the conformance matrix. + * Provider/model routing facts live here even when the concrete Responses SDK + * and replay policy are delegated to a Runtime profile. This is the single + * declared source of the default protocol split, expressed through the + * {@link ModelInfo.apiProtocol} seam. */ export function openAiAdapterApiProtocol( modelId: string, @@ -138,6 +137,8 @@ export function openAiAdapterApiProtocol( const id = modelId.trim(); return (providerType === 'deepseek' && deepSeekModelSupportsResponses(id)) || (providerType === 'opencode-go' && id === 'muse-spark-1.2-contributor') || + ((providerType === 'alibaba-token-plan-cn' || providerType === 'alibaba-token-plan') && + id === 'qwen3.8-max') || /^gpt-5/i.test(id) || ((providerType === 'xai' || providerType === 'xai-oauth') && id === 'grok-4.5') ? 'openai-responses' diff --git a/packages/core/src/model-web-search.ts b/packages/core/src/model-web-search.ts index 5372a8aa79..e56e535dc5 100644 --- a/packages/core/src/model-web-search.ts +++ b/packages/core/src/model-web-search.ts @@ -77,10 +77,7 @@ function providerHostedWebSearchAdapter( ): HostedWebSearchCapability | null { switch (providerType) { case 'deepseek': - // @ai-sdk/open-responses currently serializes function tools only. - // Mark native search unavailable so routing never hands it a provider - // tool that would be silently filtered from the request. - return { adapter: 'openai-responses', implemented: false }; + return { adapter: 'openai-responses', implemented: true }; case 'openai': case 'openai-responses-compatible': case 'xai': @@ -89,6 +86,9 @@ function providerHostedWebSearchAdapter( case 'alibaba': case 'alibaba-cn': return { adapter: 'openai-responses', implemented: false }; + case 'alibaba-token-plan-cn': + case 'alibaba-token-plan': + return { adapter: 'openai-responses', implemented: true }; case 'anthropic': case 'MiniMax': case 'MiniMax-cn': @@ -127,6 +127,11 @@ function providerDefaultHostedWebSearchCapability( case 'alibaba': case 'alibaba-cn': return /^qwen3\.5-(?:plus|flash)(?:[.-]|$)/i.test(modelId) ? capability : null; + case 'alibaba-token-plan-cn': + case 'alibaba-token-plan': + // Keep the hosted tool on the exact model that the Token Plan runtime + // routes through Responses. Aliases and Qwen 3.7 still use Chat here. + return modelId === 'qwen3.8-max' ? capability : null; case 'anthropic': return /^claude-(?:[\d.]+-)*(?:opus|sonnet|haiku|fable)\b/i.test(modelId) ? capability : null; case 'MiniMax': diff --git a/packages/core/src/provider-registry.ts b/packages/core/src/provider-registry.ts index 68243afa64..36c3c44d0a 100644 --- a/packages/core/src/provider-registry.ts +++ b/packages/core/src/provider-registry.ts @@ -31,6 +31,13 @@ export type ProviderCatalogGroup = 'recommended' | 'plans' | 'api' | 'aggregator export type ApplyPatchProtocol = 'openai-structured' | 'codex-v4a-freeform'; +/** + * Stable reference to provider execution policy implemented by `@maka/runtime`. + * Core owns only this protocol-level delegation; SDK selection, replay + * carriers, and request mutation remain Runtime implementation details. + */ +export type ProviderRuntimeProfileId = 'alibaba-token-plan'; + export type ProviderResponsesContract = | { readonly adapter: 'openai'; @@ -41,6 +48,29 @@ export type ProviderResponsesContract = readonly reasoningReplay: 'plaintext-content'; }; +type OpenAiCompatibleRuntimeAdapterBase = { + kind: 'openai-compatible'; + name: 'provider' | 'connection'; + includeUsage?: boolean; + requireBaseUrl?: boolean; + replayAssistantReasoningAs?: 'reasoning'; + replayAssistantReasoningDetails?: true; +}; + +type OpenAiCompatibleRuntimeAdapter = OpenAiCompatibleRuntimeAdapterBase & + ( + | { + /** Presence enables a complete Core-owned Responses contract. */ + responses?: ProviderResponsesContract; + runtimeProfile?: never; + } + | { + responses?: never; + /** Explicitly delegates concrete execution policy to `@maka/runtime`. */ + runtimeProfile: ProviderRuntimeProfileId; + } + ); + type ProviderRuntimeAdapterDefinition = | { kind: 'anthropic'; auth: 'api-key' | 'bearer'; normalizeBaseUrl: boolean } /** @@ -53,16 +83,7 @@ type ProviderRuntimeAdapterDefinition = | { kind: 'google'; normalizeBaseUrl?: boolean } | { kind: 'github-copilot' } | { kind: 'cohere' } - | { - kind: 'openai-compatible'; - name: 'provider' | 'connection'; - includeUsage?: boolean; - requireBaseUrl?: boolean; - /** Presence enables Responses and fixes the only supported SDK/replay pairing. */ - responses?: ProviderResponsesContract; - replayAssistantReasoningAs?: 'reasoning'; - replayAssistantReasoningDetails?: true; - }; + | OpenAiCompatibleRuntimeAdapter; export type ProviderRuntimeAdapter = ProviderRuntimeAdapterDefinition & { /** Provider wire contract for ApplyPatch. Model support is resolved separately. */ @@ -1664,7 +1685,11 @@ const providerRegistry = { fallbackModels: [...alibabaTokenPlanModelIds], status: 'ready', protocol: 'openai', - runtimeAdapter: { kind: 'openai-compatible', name: 'provider' }, + runtimeAdapter: { + kind: 'openai-compatible', + name: 'provider', + runtimeProfile: 'alibaba-token-plan', + }, modelDiscovery: { kind: 'protocol' }, category: 'domestic', catalogGroup: 'plans', @@ -1684,7 +1709,11 @@ const providerRegistry = { fallbackModels: [...alibabaTokenPlanModelIds], status: 'ready', protocol: 'openai', - runtimeAdapter: { kind: 'openai-compatible', name: 'provider' }, + runtimeAdapter: { + kind: 'openai-compatible', + name: 'provider', + runtimeProfile: 'alibaba-token-plan', + }, modelDiscovery: { kind: 'protocol' }, category: 'overseas', catalogGroup: 'plans', diff --git a/packages/runtime/src/__tests__/ai-sdk-backend.test.ts b/packages/runtime/src/__tests__/ai-sdk-backend.test.ts index de1647618a..c057a4eddb 100644 --- a/packages/runtime/src/__tests__/ai-sdk-backend.test.ts +++ b/packages/runtime/src/__tests__/ai-sdk-backend.test.ts @@ -11379,6 +11379,670 @@ describe('AiSdkBackend thinking persistence', () => { ); }); + test('Alibaba Responses keeps multiple streamed reasoning items distinct through replay', async () => { + const chunks: LanguageModelV4StreamPart[] = [ + { type: 'stream-start', warnings: [] }, + { type: 'reasoning-start', id: 'reasoning-item-1' }, + { type: 'reasoning-delta', id: 'reasoning-item-1', delta: 'first summary' }, + { + type: 'reasoning-end', + id: 'reasoning-item-1', + providerMetadata: { + 'alibaba-token-plan-cn': { + itemId: 'reasoning-item-1', + reasoningSummary: [{ type: 'summary_text', text: 'first summary' }], + reasoningContent: null, + }, + }, + }, + { type: 'reasoning-start', id: 'reasoning-item-2' }, + { type: 'reasoning-delta', id: 'reasoning-item-2', delta: 'second summary' }, + { + type: 'reasoning-end', + id: 'reasoning-item-2', + providerMetadata: { + 'alibaba-token-plan-cn': { + itemId: 'reasoning-item-2', + reasoningSummary: [{ type: 'summary_text', text: 'second summary' }], + reasoningContent: null, + }, + }, + }, + { type: 'reasoning-start', id: 'reasoning-item-empty' }, + { + type: 'reasoning-end', + id: 'reasoning-item-empty', + providerMetadata: { + 'alibaba-token-plan-cn': { + itemId: 'reasoning-item-empty', + reasoningSummary: [], + reasoningContent: null, + }, + }, + }, + { type: 'text-start', id: 'message-item' }, + { type: 'text-delta', id: 'message-item', delta: 'answer' }, + { type: 'text-end', id: 'message-item' }, + { + type: 'finish', + finishReason: { unified: 'stop', raw: 'stop' }, + usage: { + inputTokens: { total: 1, noCache: 1, cacheRead: 0, cacheWrite: 0 }, + outputTokens: { total: 3, text: 1, reasoning: 2 }, + }, + }, + ]; + const firstModel = new MockLanguageModelV4({ + doStream: { + stream: simulateReadableStream({ chunks, initialDelayInMs: null, chunkDelayInMs: null }), + }, + }); + const tokenPlanConnection = { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + } as const; + const firstBackend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async () => {}, + connection: tokenPlanConnection, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => firstModel, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + const firstEvents: SessionEvent[] = []; + for await (const event of firstBackend.send({ + turnId: 'turn-prev', + text: 'question', + context: [], + })) { + firstEvents.push(event); + } + const thinkingCompletes = firstEvents.filter( + (event): event is Extract => + event.type === 'thinking_complete', + ); + assert.deepEqual( + thinkingCompletes.map((event) => [ + event.text, + (event.providerOptions?.makaResponses as { itemId?: unknown } | undefined)?.itemId, + ]), + [ + ['first summary', 'reasoning-item-1'], + ['second summary', 'reasoning-item-2'], + ['', 'reasoning-item-empty'], + ], + ); + + const ctx = { + sessionId: 'session-1', + invocationId: 'inv-1', + runId: 'run-prev', + turnId: 'turn-prev', + now: () => 7, + newId: idGenerator(), + } as unknown as InvocationContext; + const memory = createSessionEventMapMemory(); + const runtimeContext = firstEvents.map((event) => + mapSessionEventToRuntimeEvent(event, ctx, memory), + ); + const secondModel = completionModel(); + const secondBackend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async () => {}, + connection: tokenPlanConnection, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => secondModel, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + + await drain( + secondBackend.send({ + turnId: 'turn-current', + text: 'follow up', + context: [], + runtimeContext, + }), + ); + + const prompt = compactPrompt(secondModel) as ModelMessage[]; + const assistant = prompt.find( + (message) => message.role === 'assistant' && Array.isArray(message.content), + ); + assert.ok(assistant && Array.isArray(assistant.content)); + assert.deepEqual( + assistant.content + .filter((part) => part.type === 'reasoning') + .map((part) => [ + part.text, + (part.providerOptions?.['alibaba-token-plan-cn'] as { itemId?: unknown } | undefined) + ?.itemId, + ]), + [ + ['first summary', 'reasoning-item-1'], + ['second summary', 'reasoning-item-2'], + ['', 'reasoning-item-empty'], + ], + ); + }); + + test('Alibaba Responses fails when streamed reasoning differs from the final summary', async (t) => { + // The early stop tears down the SDK stream while its settlement promises + // are still in flight; when those rejections land is scheduler-owned (on + // Windows they were observed after the test boundary). Trap unhandled + // rejections for the lifetime of this turn and assert the mismatch path + // leaves none behind, on every event loop, not just the one that raced. + const leakedRejections: unknown[] = []; + const trapUnhandledRejection = (reason: unknown): void => { + leakedRejections.push(reason); + }; + process.on('unhandledRejection', trapUnhandledRejection); + t.after(() => { + process.off('unhandledRejection', trapUnhandledRejection); + }); + const appended: AssistantMessage[] = []; + const model = new MockLanguageModelV4({ + doStream: { + stream: simulateReadableStream({ + chunks: [ + { type: 'stream-start', warnings: [] }, + { type: 'reasoning-start', id: 'reasoning-item' }, + { type: 'reasoning-delta', id: 'reasoning-item', delta: 'streamed text' }, + { + type: 'reasoning-end', + id: 'reasoning-item', + providerMetadata: { + 'alibaba-token-plan-cn': { + itemId: 'reasoning-item', + reasoningSummary: [{ type: 'summary_text', text: 'different final summary' }], + reasoningContent: null, + }, + }, + }, + ] as LanguageModelV4StreamPart[], + initialDelayInMs: null, + chunkDelayInMs: null, + }), + }, + }); + const backend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async (message) => { + if (message.type === 'assistant') appended.push(message); + }, + connection: { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + }, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => model, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + + const events: SessionEvent[] = []; + for await (const event of backend.send({ turnId: 'turn-1', text: 'question', context: [] })) { + events.push(event); + } + + assert.equal( + events.some((event) => event.type === 'error'), + true, + ); + assert.equal(events.find((event) => event.type === 'complete')?.stopReason, 'error'); + assert.equal(JSON.stringify(appended).includes('makaResponses'), false); + + const ctx = { + sessionId: 'session-1', + invocationId: 'inv-1', + runId: 'run-1', + turnId: 'turn-1', + now: () => 7, + newId: idGenerator(), + } as unknown as InvocationContext; + const memory = createSessionEventMapMemory(); + const runtimeContext = events.map((event) => mapSessionEventToRuntimeEvent(event, ctx, memory)); + const recoveryModel = completionModel(); + const recoveryBackend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async () => {}, + connection: { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + }, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => recoveryModel, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + + await drain( + recoveryBackend.send({ + turnId: 'turn-2', + text: 'recover', + context: [], + runtimeContext, + }), + ); + assert.ok(compactPrompt(recoveryModel)); + // Let SDK teardown settle across macrotask cycles so a leaked rejection + // is caught before the trap comes off. + await new Promise((resolve) => setTimeout(resolve, 100)); + assert.deepEqual( + leakedRejections, + [], + 'reasoning-mismatch teardown must not leak unhandled rejections', + ); + }); + + test('Alibaba Responses keeps a finalized item valid when the next item id is unsafe', async () => { + const invalidItemId = 'invalid\nreasoning-item'; + const model = new MockLanguageModelV4({ + doStream: { + stream: simulateReadableStream({ + chunks: [ + { type: 'stream-start', warnings: [] }, + { type: 'reasoning-start', id: 'reasoning-item-a' }, + { type: 'reasoning-delta', id: 'reasoning-item-a', delta: 'valid summary' }, + { + type: 'reasoning-end', + id: 'reasoning-item-a', + providerMetadata: { + 'alibaba-token-plan-cn': { + itemId: 'reasoning-item-a', + reasoningSummary: [{ type: 'summary_text', text: 'valid summary' }], + reasoningContent: null, + }, + }, + }, + { type: 'reasoning-start', id: invalidItemId }, + { type: 'reasoning-delta', id: invalidItemId, delta: 'unsafe item summary' }, + { + type: 'reasoning-end', + id: invalidItemId, + providerMetadata: { + 'alibaba-token-plan-cn': { + itemId: invalidItemId, + reasoningSummary: [{ type: 'summary_text', text: 'unsafe item summary' }], + reasoningContent: null, + }, + }, + }, + ] as LanguageModelV4StreamPart[], + initialDelayInMs: null, + chunkDelayInMs: null, + }), + }, + }); + const appended: AssistantMessage[] = []; + const connection = { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + } as const; + const backend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async (message) => { + if (message.type === 'assistant') appended.push(message); + }, + connection, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => model, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + const events: SessionEvent[] = []; + for await (const event of backend.send({ turnId: 'turn-1', text: 'question', context: [] })) { + events.push(event); + } + + assert.equal(events.find((event) => event.type === 'complete')?.stopReason, 'error'); + const parts = appended[0]?.thinking?.parts; + assert.deepEqual( + parts?.map((part) => [ + part.text, + (part.providerOptions?.makaResponses as { itemId?: unknown } | undefined)?.itemId, + ]), + [ + ['valid summary', 'reasoning-item-a'], + ['unsafe item summary', undefined], + ], + ); + + const ctx = { + sessionId: 'session-1', + invocationId: 'inv-1', + runId: 'run-1', + turnId: 'turn-1', + now: () => 7, + newId: idGenerator(), + } as unknown as InvocationContext; + const memory = createSessionEventMapMemory(); + const runtimeContext = events.map((event) => mapSessionEventToRuntimeEvent(event, ctx, memory)); + const recoveryModel = completionModel(); + const recoveryBackend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async () => {}, + connection, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => recoveryModel, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + + await drain( + recoveryBackend.send({ + turnId: 'turn-2', + text: 'recover', + context: [], + runtimeContext, + }), + ); + + const prompt = compactPrompt(recoveryModel) as ModelMessage[]; + const assistant = prompt.find( + (message) => message.role === 'assistant' && Array.isArray(message.content), + ); + assert.ok(assistant && Array.isArray(assistant.content)); + assert.deepEqual( + assistant.content + .filter((part) => part.type === 'reasoning') + .map((part) => [ + part.text, + (part.providerOptions?.['alibaba-token-plan-cn'] as { itemId?: unknown } | undefined) + ?.itemId, + ]), + [['valid summary', 'reasoning-item-a']], + ); + }); + + test('Alibaba Responses isolates a same-id delta that arrives after item completion', async () => { + const connection = { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + } as const; + const model = new MockLanguageModelV4({ + doStream: { + stream: simulateReadableStream({ + chunks: [ + { type: 'stream-start', warnings: [] }, + { type: 'reasoning-start', id: 'reasoning-item-a' }, + { type: 'reasoning-delta', id: 'reasoning-item-a', delta: 'valid summary' }, + { + type: 'reasoning-end', + id: 'reasoning-item-a', + providerMetadata: { + 'alibaba-token-plan-cn': { + itemId: 'reasoning-item-a', + reasoningSummary: [{ type: 'summary_text', text: 'valid summary' }], + reasoningContent: null, + }, + }, + }, + { type: 'reasoning-delta', id: 'reasoning-item-a', delta: 'late duplicate' }, + { + type: 'finish', + finishReason: { unified: 'stop', raw: 'stop' }, + usage: { + inputTokens: { total: 1, noCache: 1, cacheRead: 0, cacheWrite: 0 }, + outputTokens: { total: 2, text: 0, reasoning: 2 }, + }, + }, + ] as LanguageModelV4StreamPart[], + initialDelayInMs: null, + chunkDelayInMs: null, + }), + }, + }); + const appended: AssistantMessage[] = []; + const firstBackend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async (message) => { + if (message.type === 'assistant') appended.push(message); + }, + connection, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => model, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + const events: SessionEvent[] = []; + for await (const event of firstBackend.send({ + turnId: 'turn-1', + text: 'question', + context: [], + })) { + events.push(event); + } + + assert.deepEqual( + appended[0]?.thinking?.parts?.map((part) => [ + part.text, + (part.providerOptions?.makaResponses as { itemId?: unknown } | undefined)?.itemId, + ]), + [ + ['valid summary', 'reasoning-item-a'], + ['late duplicate', undefined], + ], + ); + + const ctx = { + sessionId: 'session-1', + invocationId: 'inv-1', + runId: 'run-1', + turnId: 'turn-1', + now: () => 7, + newId: idGenerator(), + } as unknown as InvocationContext; + const memory = createSessionEventMapMemory(); + const runtimeContext = events.map((event) => mapSessionEventToRuntimeEvent(event, ctx, memory)); + const recoveryModel = completionModel(); + const recoveryBackend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async () => {}, + connection, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => recoveryModel, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + + await drain( + recoveryBackend.send({ + turnId: 'turn-2', + text: 'follow up', + context: [], + runtimeContext, + }), + ); + const prompt = compactPrompt(recoveryModel) as ModelMessage[]; + assert.equal( + prompt.some( + (message) => + message.role === 'assistant' && + Array.isArray(message.content) && + message.content.some( + (part) => part.type === 'reasoning' && part.text === 'valid summary', + ), + ), + true, + ); + }); + + test('Alibaba Responses skips reasoning it cannot safely replay', async () => { + const foreignSummary = 'summary issued by a different provider profile'; + const futureSummary = 'summary issued by a future durable state version'; + const model = completionModel(); + const backend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async () => {}, + connection: { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + }, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => model, + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + const runtimeContext: RuntimeEvent[] = [ + runtimeEvent({ + id: 'e1', + turnId: 'turn-prev', + role: 'model', + author: 'agent', + content: { + kind: 'thinking', + text: 'summary without a provider item identity', + }, + refs: { providerEventId: 'm1' }, + }), + runtimeEvent({ + id: 'e2', + turnId: 'turn-prev', + role: 'model', + author: 'agent', + content: { + kind: 'thinking', + text: foreignSummary, + providerOptions: { + makaResponses: { + version: 1, + profile: 'alibaba-token-plan', + itemId: 'foreign-reasoning-item', + summaryPartLengths: [foreignSummary.length], + }, + }, + }, + refs: { providerEventId: 'm2' }, + }), + runtimeEvent({ + id: 'e3', + turnId: 'turn-prev', + role: 'model', + author: 'agent', + content: { + kind: 'thinking', + text: futureSummary, + providerOptions: { + makaResponses: { + version: 2, + profile: 'alibaba-token-plan-cn', + itemId: 'future-reasoning-item', + summaryPartLengths: [futureSummary.length], + }, + }, + }, + refs: { providerEventId: 'm3' }, + }), + ]; + + await drain( + backend.send({ + turnId: 'turn-current', + text: 'follow up', + context: [], + runtimeContext, + }), + ); + + const prompt = compactPrompt(model) as ModelMessage[]; + assert.equal( + prompt.some( + (message) => + message.role === 'assistant' && + Array.isArray(message.content) && + message.content.some((part) => part.type === 'reasoning'), + ), + false, + ); + }); + + test('Alibaba Responses rejects malformed state owned by its profile', async () => { + const backend = createTestAiSdkBackend({ + sessionId: 'session-1', + header: header(), + appendMessage: async () => {}, + connection: { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + }, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => completionModel(), + tools: [], + newId: idGenerator(), + now: monotonicClock(), + }); + const runtimeContext: RuntimeEvent[] = [ + runtimeEvent({ + id: 'e1', + turnId: 'turn-prev', + role: 'model', + author: 'agent', + content: { + kind: 'thinking', + text: 'current-profile reasoning with a widened state', + providerOptions: { + makaResponses: { + version: 1, + profile: 'alibaba-token-plan-cn', + itemId: 'reasoning-item', + raw: 'must not persist', + }, + }, + }, + refs: { providerEventId: 'm1' }, + }), + ]; + + await assert.rejects( + drain( + backend.send({ + turnId: 'turn-current', + text: 'follow up', + context: [], + runtimeContext, + }), + ), + /Malformed durable plaintext Responses reasoning state/, + ); + }); + test('passes DeepSeek max reasoning through as the provider-native effort', async () => { let requestBody: Record | undefined; const fetch = (async (_url: string | URL | Request, init?: RequestInit) => { diff --git a/packages/runtime/src/__tests__/model-adapter-onerror.test.ts b/packages/runtime/src/__tests__/model-adapter-onerror.test.ts index 0b2dd3de7c..c789c6548a 100644 --- a/packages/runtime/src/__tests__/model-adapter-onerror.test.ts +++ b/packages/runtime/src/__tests__/model-adapter-onerror.test.ts @@ -44,6 +44,21 @@ function newAdapter(): ModelAdapter { }); } +function newAlibabaAdapter(): ModelAdapter { + return new ModelAdapter({ + connection: { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + }, + apiKey: 'test', + modelId: 'qwen3.8-max', + modelFactory: () => ({}), + newId: () => 'id', + now: () => 0, + }); +} + describe('settleModelStepOutcome', () => { test('explicit stream failure takes precedence over finish metadata', () => { const failure = { @@ -65,9 +80,179 @@ describe('settleModelStepOutcome', () => { if (outcome.kind !== 'retryable-failure') return; assert.equal(outcome.failure, failure); }); + + test('classifies a raw error finish without widening provider retry policy', () => { + const outcome = settleModelStepOutcome({ + aborted: false, + sawFinish: true, + finishReason: 'error', + rawFinishReason: '503', + request: {}, + }); + + assert.equal(outcome.kind, 'terminal-failure'); + if (outcome.kind !== 'terminal-failure') return; + assert.equal(outcome.failure.kind, 'provider_unavailable'); + assert.equal(outcome.failure.code, '503'); + assert.equal(outcome.failure.retryable, false); + }); }); describe('ModelAdapter.startStream onError', () => { + test('preserves an already-emitted failure when reasoning flush has no final metadata', async () => { + const providerError = new APICallError({ + message: 'rate limited', + url: 'https://provider.invalid/v1/responses', + requestBodyValues: {}, + statusCode: 429, + responseHeaders: { 'retry-after-ms': '2500' }, + }); + const model = new MockLanguageModelV4({ + doStream: async () => ({ + stream: convertArrayToReadableStream([ + { type: 'stream-start', warnings: [] }, + { type: 'reasoning-start', id: 'reasoning-1' }, + { type: 'reasoning-delta', id: 'reasoning-1', delta: 'partial reasoning' }, + { type: 'error', error: providerError }, + // This test owns the ModelAdapter boundary sequence. Separate pinned + // SDK fixtures below prove which raw SSE terminal events produce the + // metadata-less reasoning trailer. + { type: 'reasoning-end', id: 'reasoning-1' }, + { + type: 'finish', + finishReason: { unified: 'error', raw: 'provider_error' }, + usage: ZERO_USAGE, + }, + ]), + }), + }); + const result = await newAlibabaAdapter().startStream({ + model, + messages: [{ role: 'user', content: 'hi' }], + tools: {}, + activeTools: [], + onStreamActivity: () => {}, + abortSignal: new AbortController().signal, + repairToolCall: async () => null, + }); + + const failures = []; + for await (const event of result.events) { + if (event.kind === 'error') { + failures.push(event.failure); + // AiSdkBackend stops consuming after the first error. ModelAdapter + // must settle outcome before yielding this deferred validation error. + break; + } + } + + assert.deepEqual(failures, [ + { + type: 'model_failure', + kind: 'rate_limit', + code: '429', + message: 'Rate limit exceeded', + retryable: true, + retryAfterMs: 2500, + }, + ]); + const outcome = await requireAlreadySettled(result.outcome); + assert.equal(outcome.kind, 'retryable-failure'); + if (outcome.kind !== 'retryable-failure') return; + assert.deepEqual(outcome.failure, failures[0]); + }); + + test('fails closed when a successful reasoning stream has no final metadata', async () => { + const model = new MockLanguageModelV4({ + doStream: async () => ({ + stream: convertArrayToReadableStream([ + { type: 'stream-start', warnings: [] }, + { type: 'reasoning-start', id: 'reasoning-1' }, + { type: 'reasoning-delta', id: 'reasoning-1', delta: 'partial reasoning' }, + // A completed response without output_item.done reaches Maka as the + // same metadata-less flush trailer as a failed response. + { type: 'reasoning-end', id: 'reasoning-1' }, + { + type: 'finish', + finishReason: { unified: 'stop', raw: 'stop' }, + usage: ZERO_USAGE, + }, + ]), + }), + }); + const result = await newAlibabaAdapter().startStream({ + model, + messages: [{ role: 'user', content: 'hi' }], + tools: {}, + activeTools: [], + onStreamActivity: () => {}, + abortSignal: new AbortController().signal, + repairToolCall: async () => null, + }); + + const failures = []; + for await (const event of result.events) { + if (event.kind === 'error') { + failures.push(event.failure); + break; + } + } + + assert.deepEqual(failures, [ + { + type: 'model_failure', + kind: 'unknown', + message: 'Plaintext Responses reasoning item is missing final summary metadata', + retryable: false, + }, + ]); + const outcome = await requireAlreadySettled(result.outcome); + assert.equal(outcome.kind, 'terminal-failure'); + if (outcome.kind !== 'terminal-failure') return; + assert.deepEqual(outcome.failure, failures[0]); + }); + + test('classifies a raw provider error finish ahead of an unfinalized reasoning trailer', async () => { + const model = new MockLanguageModelV4({ + doStream: async () => ({ + stream: convertArrayToReadableStream([ + { type: 'stream-start', warnings: [] }, + { type: 'reasoning-start', id: 'reasoning-1' }, + { type: 'reasoning-delta', id: 'reasoning-1', delta: 'partial reasoning' }, + { type: 'reasoning-end', id: 'reasoning-1' }, + { + type: 'finish', + finishReason: { unified: 'error', raw: 'rate_limit_exceeded' }, + usage: ZERO_USAGE, + }, + ]), + }), + }); + const result = await newAlibabaAdapter().startStream({ + model, + messages: [{ role: 'user', content: 'hi' }], + tools: {}, + activeTools: [], + onStreamActivity: () => {}, + abortSignal: new AbortController().signal, + repairToolCall: async () => null, + }); + + for await (const _event of result.events) void _event; + const outcome = await result.outcome; + + assert.equal(outcome.kind, 'terminal-failure'); + if (outcome.kind !== 'terminal-failure') return; + assert.deepEqual(outcome.failure, { + type: 'model_failure', + kind: 'rate_limit', + retryable: false, + message: 'Rate limit exceeded', + code: 'rate_limit_exceeded', + }); + assert.equal(outcome.usage?.rawFinishReason, 'rate_limit_exceeded'); + }); + test('normalizes provider retry eligibility and Retry-After at the adapter boundary', async () => { const model = new MockLanguageModelV4({ doStream: async () => { @@ -332,3 +517,10 @@ async function settle( for await (const _event of result.events) void _event; return await result.outcome; } + +async function requireAlreadySettled(promise: Promise): Promise { + const pending = Symbol('pending'); + const value = await Promise.race([promise, Promise.resolve(pending)]); + assert.notEqual(value, pending, 'model outcome must settle before the consumer stops iterating'); + return value as T; +} diff --git a/packages/runtime/src/__tests__/model-adapter.test.ts b/packages/runtime/src/__tests__/model-adapter.test.ts index 99b08f8991..d6a9871e3c 100644 --- a/packages/runtime/src/__tests__/model-adapter.test.ts +++ b/packages/runtime/src/__tests__/model-adapter.test.ts @@ -170,6 +170,136 @@ describe('ModelAdapter stream and error normalization', () => { }); }); + test('supports summary-item Responses reasoning replay for Alibaba Token Plan', () => { + const adapter = new ModelAdapter({ + connection: { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + }, + apiKey: 'alibaba-token', + modelId: 'qwen3.8-max', + modelFactory: () => ({}), + newId: idGenerator(), + now: monotonicClock(), + }); + + assert.deepEqual(adapter.runtimeEventReplaySupport(), { + toolCalls: true, + toolResults: true, + providerExecutedTools: false, + signedThinking: false, + unsignedThinking: false, + responsesReasoning: { + kind: 'plaintext-item', + profile: 'alibaba-token-plan-cn', + providerOptionsKey: 'alibaba-token-plan-cn', + }, + }); + }); + + test('normalizes Alibaba stream item ids into bounded durable state from item start', () => { + const providerType = 'alibaba-token-plan-cn'; + const adapter = new ModelAdapter({ + connection: { slug: providerType, providerType, defaultModel: 'qwen3.8-max' }, + apiKey: 'token', + modelId: 'qwen3.8-max', + modelFactory: () => ({}), + newId: idGenerator(), + now: monotonicClock(), + }); + type Chunk = Parameters[0]; + const providerOptions = { + makaResponses: { + version: 1, + profile: providerType, + itemId: 'alibaba-reasoning-item', + summaryPartLengths: [7], + }, + }; + assert.deepEqual( + adapter.translateChunk({ type: 'reasoning-start', id: 'alibaba-reasoning-item' } as Chunk), + [{ kind: 'thinking', text: '', reasoningItemId: 'alibaba-reasoning-item' }], + ); + assert.deepEqual( + adapter.translateChunk({ + type: 'reasoning-delta', + id: 'alibaba-reasoning-item', + delta: 'summary', + } as Chunk), + [{ kind: 'thinking', text: 'summary', reasoningItemId: 'alibaba-reasoning-item' }], + ); + assert.deepEqual( + adapter.translateChunk({ + type: 'reasoning-end', + id: 'alibaba-reasoning-item', + providerMetadata: { + [providerType]: { + itemId: 'alibaba-reasoning-item', + reasoningSummary: [{ type: 'summary_text', text: 'summary' }], + }, + }, + } as Chunk), + [ + { + kind: 'thinking', + text: '', + providerOptions, + reasoningItemId: 'alibaba-reasoning-item', + reasoningSummaryText: 'summary', + }, + ], + ); + assert.throws( + () => + adapter.translateChunk({ + type: 'reasoning-end', + id: 'unfinished-flush', + } as Chunk), + /missing final summary metadata/, + ); + assert.throws( + () => + adapter.translateChunk({ + type: 'reasoning-end', + id: 'missing-final-summary', + providerMetadata: { + [providerType]: { itemId: 'missing-final-summary' }, + }, + } as Chunk), + /missing final summary metadata/, + ); + }); + + test('keeps DeepSeek plaintext replay on the main content-only behavior', () => { + const adapter = new ModelAdapter({ + connection: { slug: 'deepseek', providerType: 'deepseek', defaultModel: 'deepseek-v4-flash' }, + apiKey: 'token', + modelId: 'deepseek-v4-flash', + modelFactory: () => ({}), + newId: idGenerator(), + now: monotonicClock(), + }); + type Chunk = Parameters[0]; + + assert.deepEqual( + adapter.translateChunk({ + type: 'reasoning-delta', + id: 'deepseek-reasoning-item', + delta: 'plaintext reasoning', + } as Chunk), + [{ kind: 'thinking', text: 'plaintext reasoning' }], + ); + assert.deepEqual( + adapter.translateChunk({ + type: 'reasoning-end', + id: 'deepseek-reasoning-item', + providerMetadata: { deepseek: { itemId: 'deepseek-reasoning-item' } }, + } as Chunk), + [], + ); + }); + test('translates provider text, reasoning, tool calls, and errors into ModelStreamEvents', () => { const adapter = newAdapter(); type Chunk = Parameters[0]; diff --git a/packages/runtime/src/__tests__/native-web-search-tool.test.ts b/packages/runtime/src/__tests__/native-web-search-tool.test.ts index 9b111a77c1..458aa83aad 100644 --- a/packages/runtime/src/__tests__/native-web-search-tool.test.ts +++ b/packages/runtime/src/__tests__/native-web-search-tool.test.ts @@ -55,7 +55,13 @@ test('turn-start routing falls back explicitly when native search is unavailable slug: 'deepseek', providerType: 'deepseek' as const, defaultModel: 'deepseek-v4-flash', - models: [{ id: 'deepseek-v4-flash', capabilities: { webSearch: true } }], + models: [ + { + id: 'deepseek-v4-flash', + apiProtocol: 'openai-chat' as const, + capabilities: { webSearch: true }, + }, + ], }; const native = routeWebSearchTools({ @@ -145,7 +151,29 @@ test('turn-start routing compiles Claude models to the CC-compatible Anthropic t }); }); -test('root surfaces do not advertise unsupported DeepSeek native search', () => { +test('turn-start routing compiles Qwen3.8 Max Token Plan to Responses web search', () => { + const routed = routeWebSearchTools({ + tools: [], + settings: { enabled: true, defaultProvider: 'model' }, + connection: { + slug: 'alibaba-token-plan-cn', + providerType: 'alibaba-token-plan-cn', + defaultModel: 'qwen3.8-max', + }, + model: 'qwen3.8-max', + tavilyReady: false, + allowAddNative: true, + }); + + assert.equal(routed.length, 1); + assert.equal(routed[0]?.name, NATIVE_WEB_SEARCH_TOOL_NAME); + assert.deepEqual(routed[0]?.providerTool, { + kind: 'openai-web-search', + searchContextSize: 'medium', + }); +}); + +test('root surfaces advertise DeepSeek native search without widening child tools', () => { const connection = { slug: 'deepseek', providerType: 'deepseek' as const, @@ -159,7 +187,12 @@ test('root surfaces do not advertise unsupported DeepSeek native search', () => tavilyReady: false, allowAddNative: true, }); - assert.deepEqual(root, []); + assert.equal(root.length, 1); + assert.equal(root[0]?.name, NATIVE_WEB_SEARCH_TOOL_NAME); + assert.deepEqual(root[0]?.providerTool, { + kind: 'openai-web-search', + searchContextSize: 'medium', + }); const child = routeWebSearchTools({ tools: [], diff --git a/packages/runtime/src/__tests__/open-responses-compatibility.test.ts b/packages/runtime/src/__tests__/open-responses-compatibility.test.ts new file mode 100644 index 0000000000..64021e37d3 --- /dev/null +++ b/packages/runtime/src/__tests__/open-responses-compatibility.test.ts @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import assert from 'node:assert/strict'; +import { test } from 'node:test'; +import { createOpenResponsesCompatibilityFinalizer } from '../open-responses-compatibility.js'; + +test('applies the declared Open Responses body policies', () => { + const finalize = createOpenResponsesCompatibilityFinalizer('alibaba-token-plan'); + assert.ok(finalize); + assert.deepEqual(finalize({ model: 'qwen3.8-max', store: true, tool_choice: 'auto' }), { + model: 'qwen3.8-max', + store: false, + tool_choice: 'auto', + }); + assert.deepEqual(finalize({ model: 'qwen3.8-max' }), { + model: 'qwen3.8-max', + store: false, + }); + for (const toolChoice of ['required', { type: 'function', name: 'lookup' }]) { + assert.throws( + () => finalize({ tool_choice: toolChoice }), + /does not support forced tool_choice/, + ); + } +}); diff --git a/packages/runtime/src/__tests__/openai-responses-plaintext-reasoning.test.ts b/packages/runtime/src/__tests__/openai-responses-plaintext-reasoning.test.ts index f28f382ce6..34c1f02683 100644 --- a/packages/runtime/src/__tests__/openai-responses-plaintext-reasoning.test.ts +++ b/packages/runtime/src/__tests__/openai-responses-plaintext-reasoning.test.ts @@ -50,7 +50,12 @@ const ANSWER = 'No — 91 is 7 x 13.'; * translator that dropped the message entirely would be the worst failure this * code can have, and only an assertion on the reply can see it. */ -function deepseekReasoningStream(deltas: string[], answer = ANSWER): string { +function plaintextReasoningStream( + deltas: string[], + answer = ANSWER, + finalSummary: Array<{ type: 'summary_text'; text: string }> = [], + model = 'deepseek-v4-flash', +): string { const events: Array> = [ { type: 'response.created', response: { id: 'r' } }, { @@ -81,7 +86,7 @@ function deepseekReasoningStream(deltas: string[], answer = ANSWER): string { id: ITEM_ID, status: 'completed', content: [{ type: 'reasoning_text', text: deltas.join('') }], - summary: [], + summary: finalSummary, }, }, { @@ -119,7 +124,7 @@ function deepseekReasoningStream(deltas: string[], answer = ANSWER): string { id: 'r', object: 'response', created_at: 0, - model: 'deepseek-v4-flash', + model, status: 'completed', output: [], usage: { input_tokens: 1, output_tokens: 1 }, @@ -177,6 +182,55 @@ function standardFunctionCallStream(): string { return `${events.map((event) => `data: ${JSON.stringify(event)}`).join('\n\n')}\n\ndata: [DONE]\n\n`; } +function unfinalizedReasoningStream(terminal: 'completed' | 'failed'): string { + const response = { + id: 'r', + object: 'response', + created_at: 0, + model: 'qwen3.8-max', + status: terminal, + output: [], + usage: { input_tokens: 1, output_tokens: 1 }, + ...(terminal === 'failed' + ? { error: { code: 'rate_limit_exceeded', message: 'rate limited' } } + : {}), + }; + const events = [ + { type: 'response.created', response: { id: 'r' } }, + { + type: 'response.output_item.added', + output_index: 0, + item: { type: 'reasoning', id: ITEM_ID, status: 'in_progress', content: [], summary: [] }, + }, + { + type: 'response.reasoning_text.delta', + content_index: 0, + delta: 'unfinished reasoning', + item_id: ITEM_ID, + output_index: 0, + }, + { type: `response.${terminal}`, response }, + ]; + return `${events.map((event) => `data: ${JSON.stringify(event)}`).join('\n\n')}\n\ndata: [DONE]\n\n`; +} + +async function alibabaStreamParts(body: string) { + const connection = conn('alibaba-token-plan-cn'); + const model = getAIModel({ + connection, + apiKey: 'test-key', + modelId: 'qwen3.8-max', + fetch: sseFetch(body), + }); + const { stream } = await model.doStream({ + prompt: [{ role: 'user', content: [{ type: 'text', text: 'hi' }] }], + providerOptions: buildProviderOptions(connection, 'qwen3.8-max', 'high'), + }); + const parts = []; + for await (const part of stream) parts.push(part); + return parts; +} + /** * Chunks are cut from the encoded bytes, not from the string: slicing the * string would hand every chunk a whole character and quietly make multi-byte @@ -226,9 +280,51 @@ async function streamParts( } describe('open responses plaintext reasoning', () => { + test('Alibaba raw SSE content deltas match the final summary metadata', async () => { + const deltas = ['Inspect the request. ', 'Call the Maka tool.']; + const summary = [{ type: 'summary_text' as const, text: deltas.join('') }]; + const parts = await alibabaStreamParts( + plaintextReasoningStream(deltas, ANSWER, summary, 'qwen3.8-max'), + ); + const streamed = parts + .filter((part) => part.type === 'reasoning-delta') + .map((part) => part.delta) + .join(''); + const reasoningEnd = parts.find((part) => part.type === 'reasoning-end'); + assert.ok(reasoningEnd && reasoningEnd.type === 'reasoning-end'); + const provider = reasoningEnd.providerMetadata?.['alibaba-token-plan-cn'] as + | { reasoningSummary?: Array<{ type: string; text: string }> } + | undefined; + assert.deepEqual(provider?.reasoningSummary, [ + { type: 'summary_text', text: 'Inspect the request. Call the Maka tool.' }, + ]); + assert.equal(streamed, provider?.reasoningSummary?.map((part) => part.text).join('')); + }); + + test('the pinned SDK flushes an unfinalized item without provider metadata', async () => { + const parts = await alibabaStreamParts(unfinalizedReasoningStream('completed')); + const reasoningEnd = parts.find((part) => part.type === 'reasoning-end'); + const finish = parts.find((part) => part.type === 'finish'); + + assert.ok(reasoningEnd); + assert.equal(reasoningEnd.providerMetadata, undefined); + assert.equal(finish?.finishReason.unified, 'stop'); + }); + + test('the pinned SDK keeps response.failed ahead of its unfinalized trailer', async () => { + const parts = await alibabaStreamParts(unfinalizedReasoningStream('failed')); + const reasoningEnd = parts.find((part) => part.type === 'reasoning-end'); + const finish = parts.find((part) => part.type === 'finish'); + + assert.ok(reasoningEnd); + assert.equal(reasoningEnd.providerMetadata, undefined); + assert.equal(finish?.finishReason.unified, 'error'); + assert.equal(finish?.finishReason.raw, 'rate_limit_exceeded'); + }); + test('streamed reasoning text reaches the model stream', async () => { const deltas = ['The user asks if 91 is prime. ', '91 = 7 x 13, ', 'so it is composite.']; - const parts = await streamParts('deepseek', sseFetch(deepseekReasoningStream(deltas))); + const parts = await streamParts('deepseek', sseFetch(plaintextReasoningStream(deltas))); assert.equal(parts.reasoning, deltas.join('')); assert.equal(parts.text, ANSWER); }); @@ -239,7 +335,7 @@ describe('open responses plaintext reasoning', () => { // dropping message frames wholesale would otherwise leave the suite green. const parts = await streamParts( 'deepseek', - sseFetch(deepseekReasoningStream(['thinking'], 'The answer is 42.')), + sseFetch(plaintextReasoningStream(['thinking'], 'The answer is 42.')), ); assert.equal(parts.text, 'The answer is 42.'); }); @@ -251,7 +347,7 @@ describe('open responses plaintext reasoning', () => { // was asked in, and a 7-byte chunk cuts these characters mid-sequence, so // this also pins the decoder's cross-chunk state. const deltas = ['用户问 91 是不是质数。', '91 = 7 × 13，', '所以它是合数。']; - const parts = await streamParts('deepseek', sseFetch(deepseekReasoningStream(deltas), 7)); + const parts = await streamParts('deepseek', sseFetch(plaintextReasoningStream(deltas), 7)); assert.equal(parts.reasoning, deltas.join('')); assert.equal(parts.text, ANSWER); }); @@ -260,7 +356,7 @@ describe('open responses plaintext reasoning', () => { // The transport is mounted per provider, not per wire. xAI reaches the same // Responses wire but its reasoning shape has not been measured, so nothing // should rewrite its stream on the strength of the wire alone. - const parts = await streamParts('xai', sseFetch(deepseekReasoningStream(['ignored']))); + const parts = await streamParts('xai', sseFetch(plaintextReasoningStream(['ignored']))); assert.equal(parts.reasoning, ''); assert.equal(parts.text, ANSWER); }); diff --git a/packages/runtime/src/__tests__/provider-contract-matrix.test.ts b/packages/runtime/src/__tests__/provider-contract-matrix.test.ts index 4f3b95ffe5..0da26ce942 100644 --- a/packages/runtime/src/__tests__/provider-contract-matrix.test.ts +++ b/packages/runtime/src/__tests__/provider-contract-matrix.test.ts @@ -23,6 +23,7 @@ import { after, describe, test } from 'node:test'; import type { LlmConnection } from '@maka/core/llm-connections'; import { PROVIDER_CONTRACT_MATRIX_PLAN, + listProviderContractCells, type ProviderContractDiscoveryPlan, type ProviderContractRow, type ProviderContractGeneratedCell, @@ -49,6 +50,16 @@ const plan = PROVIDER_CONTRACT_MATRIX_PLAN; after(closeAllJsonServers); +test('provider override cells and executable bindings are a bijection', () => { + const plannedKeys = listProviderContractCells(plan) + .flatMap(({ cell }) => (cell.state === 'override' ? [cell.overrideKey] : [])) + .sort(); + const bindingKeys = PROVIDER_CONTRACT_OVERRIDE_BINDINGS.flatMap(({ keys }) => keys).sort(); + assert.deepEqual(duplicateValues(plannedKeys), [], 'override cells must be unique'); + assert.deepEqual(duplicateValues(bindingKeys), [], 'override bindings must be unique'); + assert.deepEqual(bindingKeys, plannedKeys); +}); + describe('provider conformance matrix — override cells execute their bound contract', () => { for (const binding of PROVIDER_CONTRACT_OVERRIDE_BINDINGS) { test(`${binding.keys.join(' + ')} · ${binding.title}`, async () => { @@ -57,6 +68,10 @@ describe('provider conformance matrix — override cells execute their bound con } }); +function duplicateValues(values: readonly string[]): string[] { + return values.filter((value, index) => values.indexOf(value) !== index); +} + describe('provider conformance matrix — discovery', () => { for (const row of plan.rows) { const cell = row.cells.discovery; diff --git a/packages/runtime/src/__tests__/provider-contract-matrix.ts b/packages/runtime/src/__tests__/provider-contract-matrix.ts index 274c7851fb..449cdb8f39 100644 --- a/packages/runtime/src/__tests__/provider-contract-matrix.ts +++ b/packages/runtime/src/__tests__/provider-contract-matrix.ts @@ -51,6 +51,7 @@ import { type ProviderRuntimeAdapter, type ProviderType, } from '@maka/core/provider-registry'; +import { resolveRuntimeProviderAdapter } from '../provider-runtime-policy.js'; export const PROVIDER_CONTRACT_DIMENSIONS = [ 'discovery', @@ -214,7 +215,7 @@ function usesOpenAiResponsesWire( def: ProviderDefaults, modelId: string, ): boolean { - const adapter = def.runtimeAdapter; + const adapter = resolveRuntimeProviderAdapter(def.runtimeAdapter); const supportsResponses = adapter.kind === 'openai' || (adapter.kind === 'openai-compatible' && adapter.responses !== undefined); @@ -389,7 +390,7 @@ function reasoningReplayCell( providerType: ProviderType, def: ProviderDefaults, ): ProviderContractCell { - const adapter = def.runtimeAdapter; + const adapter = resolveRuntimeProviderAdapter(def.runtimeAdapter); if (adapter.kind === 'unavailable') { return { state: 'not-applicable', @@ -434,17 +435,12 @@ function reasoningReplayCell( }, }; } - if ( - providerType === 'volcengine-agent-plan' && - adapter.kind === 'openai' && - adapter.apiProtocol === 'openai-responses' - ) { + if (adapter.kind === 'openai' && adapter.apiProtocol === 'openai-responses') { return { state: 'override', dimension: 'reasoning-replay', overrideKey: overrideKeyFor(providerType, 'reasoning-replay'), - contract: - 'Stateless OpenAI Responses reasoning items retain their encrypted content across Maka-owned durable replay', + contract: 'Native OpenAI Responses reasoning items retain their provider continuation state', }; } // Native Anthropic / OpenAI / Google / Cohere SDKs own signed reasoning replay diff --git a/packages/runtime/src/__tests__/provider-contract-overrides.ts b/packages/runtime/src/__tests__/provider-contract-overrides.ts index 6a25c8b36a..ca7c049a2f 100644 --- a/packages/runtime/src/__tests__/provider-contract-overrides.ts +++ b/packages/runtime/src/__tests__/provider-contract-overrides.ts @@ -97,6 +97,42 @@ export const PROVIDER_CONTRACT_OVERRIDE_BINDINGS: readonly ProviderContractOverr title: 'ZenMux replays signed reasoning details in the streamed runtime tool loop', run: runZenMuxSignedReasoningReplay, }, + { + keys: ['deepseek:reasoning-replay'], + title: 'DeepSeek replays plaintext reasoning content on its Responses wire', + run: () => + runOpenAIResponsesWire({ + providerType: 'deepseek', + slug: 'deepseek', + name: 'DeepSeek', + basePath: '/v1', + modelId: 'deepseek-v4-flash', + apiKey: 'deepseek-test-key', + plaintextReasoning: true, + }), + }, + { + keys: ['xai:reasoning-replay', 'xai-oauth:reasoning-replay'], + title: 'xAI API-key and OAuth paths retain encrypted Responses reasoning', + run: async () => { + for (const input of [ + { providerType: 'xai' as const, slug: 'xai', apiKey: 'xai-test-key' }, + { + providerType: 'xai-oauth' as const, + slug: 'xai-oauth', + apiKey: 'xai-oauth-test-token', + }, + ]) { + await runOpenAIResponsesWire({ + ...input, + name: input.slug, + basePath: `/${input.slug}/v1`, + modelId: 'grok-4.5', + statelessReasoning: true, + }); + } + }, + }, { keys: [ 'openai-responses-compatible:exact-model-id', @@ -133,8 +169,138 @@ export const PROVIDER_CONTRACT_OVERRIDE_BINDINGS: readonly ProviderContractOverr statelessReasoning: true, }), }, + { + keys: ['alibaba-token-plan-cn:reasoning-replay', 'alibaba-token-plan:reasoning-replay'], + title: 'Alibaba Token Plan replays plaintext summary items on its Responses wire', + run: runAlibabaTokenPlanResponsesWire, + }, ]; +async function runAlibabaTokenPlanResponsesWire(): Promise { + for (const input of [ + { + providerType: 'alibaba-token-plan-cn' as const, + slug: 'alibaba-token-plan-cn', + basePath: '/cn/compatible-mode/v1', + apiKey: 'alibaba-token-plan-cn-key', + }, + { + providerType: 'alibaba-token-plan' as const, + slug: 'alibaba-token-plan', + basePath: '/global/compatible-mode/v1', + apiKey: 'alibaba-token-plan-global-key', + }, + ]) { + const modelId = 'qwen3.8-max'; + const requestBodies: Array> = []; + const server = await startJsonServer(async (request, response) => { + assert.equal(request.method, 'POST'); + assert.equal(request.url, `${input.basePath}/responses`); + assert.equal(request.headers.authorization, `Bearer ${input.apiKey}`); + requestBodies.push(JSON.parse(await readBody(request)) as Record); + if (requestBodies.length === 1) { + respondJson(response, 200, { + id: 'resp_alibaba_tool', + object: 'response', + created_at: 1, + status: 'completed', + model: modelId, + output: [ + { + type: 'reasoning', + id: 'rs_alibaba_tool', + summary: [{ type: 'summary_text', text: 'Use echo.' }], + }, + { + type: 'function_call', + id: 'fc_alibaba_echo', + call_id: 'call_alibaba_echo', + name: 'echo', + arguments: '{"text":"hello"}', + status: 'completed', + }, + ], + usage: { input_tokens: 8, output_tokens: 4, total_tokens: 12 }, + }); + return; + } + respondJson(response, 200, { + id: 'resp_alibaba_final', + object: 'response', + created_at: 2, + status: 'completed', + model: modelId, + output: [ + { + type: 'message', + id: 'msg_alibaba_final', + status: 'completed', + role: 'assistant', + content: [ + { type: 'output_text', text: 'Echoed hello.', annotations: [], logprobs: [] }, + ], + }, + ], + usage: { input_tokens: 14, output_tokens: 3, total_tokens: 17 }, + }); + }); + const connection: LlmConnection = { + slug: input.slug, + name: input.slug, + providerType: input.providerType, + baseUrl: `${server.url}${input.basePath}`, + defaultModel: modelId, + enabled: true, + createdAt: 1, + updatedAt: 1, + }; + + const result = await generateText({ + model: getAIModel({ connection, apiKey: input.apiKey, modelId }), + prompt: 'Call echo with hello.', + providerOptions: buildProviderOptions(connection, modelId), + stopWhen: isStepCount(2), + tools: { + echo: tool({ + description: 'Echo text', + inputSchema: z.object({ text: z.string() }), + execute: async ({ text }) => ({ echoed: text }), + }), + }, + }); + + assert.deepEqual( + requestBodies.map((body) => body.model), + [modelId, modelId], + ); + assert.deepEqual( + requestBodies.map((body) => body.store), + [false, false], + ); + assert.deepEqual( + (requestBodies[1].input as Array>).find( + ({ type }) => type === 'reasoning', + ), + { + type: 'reasoning', + id: 'rs_alibaba_tool', + summary: [{ type: 'summary_text', text: 'Use echo.' }], + }, + ); + assert.deepEqual( + (requestBodies[1].input as Array>).find( + ({ type }) => type === 'function_call_output', + ), + { + type: 'function_call_output', + call_id: 'call_alibaba_echo', + output: '{"echoed":"hello"}', + }, + ); + assert.equal(result.text, 'Echoed hello.'); + } +} + async function runCloudflareDiscovery(): Promise { const server = await startJsonServer((request, response) => { assert.equal(request.method, 'GET'); @@ -883,8 +1049,19 @@ async function runOpenAIResponsesWire(input: { modelId: string; apiKey: string; statelessReasoning?: boolean; + plaintextReasoning?: boolean; }): Promise { - const { providerType, slug, name, basePath, modelId, apiKey, statelessReasoning } = input; + const { + providerType, + slug, + name, + basePath, + modelId, + apiKey, + statelessReasoning, + plaintextReasoning, + } = input; + const hasReasoning = statelessReasoning || plaintextReasoning; const requestBodies: Array> = []; const server = await startJsonServer(async (request, response) => { assert.equal(request.method, 'POST'); @@ -908,7 +1085,16 @@ async function runOpenAIResponsesWire(input: { encrypted_content: 'encrypted-relay-reasoning', }, ] - : []), + : plaintextReasoning + ? [ + { + type: 'reasoning', + id: 'rs_relay_tool', + summary: [], + content: [{ type: 'reasoning_text', text: 'Use echo.' }], + }, + ] + : []), { type: 'function_call', id: 'fc_relay_echo', @@ -954,7 +1140,7 @@ async function runOpenAIResponsesWire(input: { const result = await generateText({ model: getAIModel({ connection, apiKey, modelId }), prompt: 'Call echo with hello.', - ...(statelessReasoning ? { providerOptions: buildProviderOptions(connection, modelId) } : {}), + ...(hasReasoning ? { providerOptions: buildProviderOptions(connection, modelId) } : {}), stopWhen: isStepCount(2), tools: { echo: tool({ @@ -984,6 +1170,19 @@ async function runOpenAIResponsesWire(input: { }, ); } + if (plaintextReasoning) { + assert.deepEqual( + (requestBodies[1].input as Array>).find( + ({ type }) => type === 'reasoning', + ), + { + type: 'reasoning', + id: 'rs_relay_tool', + summary: [], + content: [{ type: 'reasoning_text', text: 'Use echo.' }], + }, + ); + } assert.deepEqual( (requestBodies[1].input as Array>).find( ({ type }) => type === 'function_call_output', diff --git a/packages/runtime/src/__tests__/request-customization-fetch.test.ts b/packages/runtime/src/__tests__/request-customization-fetch.test.ts index cca34784aa..5dda1aa769 100644 --- a/packages/runtime/src/__tests__/request-customization-fetch.test.ts +++ b/packages/runtime/src/__tests__/request-customization-fetch.test.ts @@ -119,4 +119,31 @@ describe('createRequestCustomizationFetch', () => { /Extra request body conflicts/, ); }); + + test('applies a provider finalizer after caller body overlays', async () => { + let captured: Request | undefined; + const fetch = createRequestCustomizationFetch( + async (input, init) => { + captured = new Request(input, init); + return Response.json({ ok: true }); + }, + { + bodyOverlay: { store: true }, + finalizeBody: (body) => ({ ...body, store: false }), + }, + ); + + await fetch('https://provider.invalid/responses', { + method: 'POST', + headers: { 'content-length': '999', 'content-type': 'application/json' }, + body: new TextEncoder().encode(JSON.stringify({ model: 'qwen3.8-max' })).buffer, + }); + + assert.deepEqual(await captured?.json(), { model: 'qwen3.8-max', store: false }); + assert.equal(captured?.headers.get('content-length'), null); + await assert.rejects( + fetch('https://provider.invalid/responses', { method: 'POST', body: 'not-json' }), + /finalizer requires a JSON object request body/, + ); + }); }); diff --git a/packages/runtime/src/__tests__/responses-reasoning-state.test.ts b/packages/runtime/src/__tests__/responses-reasoning-state.test.ts new file mode 100644 index 0000000000..96753ad4aa --- /dev/null +++ b/packages/runtime/src/__tests__/responses-reasoning-state.test.ts @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import assert from 'node:assert/strict'; +import { test } from 'node:test'; +import { + decodePlaintextResponsesReasoningState, + plaintextResponsesReasoningProviderOptions, + replayPlaintextResponsesProviderOptions, + responsesReasoningItemId, +} from '../responses-reasoning-state.js'; + +test('round-trips one bounded versioned plaintext Responses item identity', () => { + const options = plaintextResponsesReasoningProviderOptions( + 'reasoning-item-1', + 'alibaba-token-plan-cn', + ['reasoning summary'], + ); + assert.deepEqual(options, { + makaResponses: { + version: 1, + profile: 'alibaba-token-plan-cn', + itemId: 'reasoning-item-1', + summaryPartLengths: [17], + }, + }); + assert.deepEqual(decodePlaintextResponsesReasoningState(options), { + kind: 'valid', + state: { + version: 1, + profile: 'alibaba-token-plan-cn', + itemId: 'reasoning-item-1', + summaryPartLengths: [17], + }, + }); + assert.equal(responsesReasoningItemId(options), 'reasoning-item-1'); +}); + +test('rejects malformed, widened, and unsafe plaintext Responses state', () => { + for (const makaResponses of [ + { version: '2', profile: 'alibaba-token-plan-cn', itemId: 'item' }, + { version: 1, profile: '', itemId: 'item' }, + { version: 1, profile: 'bad\nprofile', itemId: 'item' }, + { version: 1, profile: 'alibaba-token-plan-cn', itemId: '' }, + { version: 1, profile: 'alibaba-token-plan-cn', itemId: 'bad\nitem' }, + { + version: 1, + profile: 'alibaba-token-plan-cn', + itemId: 'item', + summaryPartLengths: [4], + raw: 'provider-body', + }, + ]) { + assert.equal(decodePlaintextResponsesReasoningState({ makaResponses }).kind, 'malformed'); + } + assert.deepEqual(decodePlaintextResponsesReasoningState(undefined), { kind: 'missing' }); + assert.deepEqual( + decodePlaintextResponsesReasoningState({ + makaResponses: { + version: 2, + profile: 'another-provider', + itemId: 'item', + }, + }), + { kind: 'unsupported-version', version: 2 }, + ); +}); + +test('degrades a well-formed state version that this Runtime cannot replay', () => { + assert.deepEqual( + decodePlaintextResponsesReasoningState({ + makaResponses: { + version: 2, + profile: 'alibaba-token-plan-cn', + itemId: 'item', + summaryPartLengths: [4], + }, + }), + { kind: 'unsupported-version', version: 2 }, + ); +}); + +test('reconstructs provider-native summary parts', () => { + const summary = { + version: 1, + profile: 'alibaba-token-plan-cn', + itemId: 'summary-item', + summaryPartLengths: [10, 7], + } as const; + assert.deepEqual( + replayPlaintextResponsesProviderOptions({ + providerOptionsKey: 'alibaba-token-plan-cn', + state: summary, + text: 'reasoning summary', + }), + { + 'alibaba-token-plan-cn': { + itemId: 'summary-item', + reasoningSummary: [ + { type: 'summary_text', text: 'reasoning ' }, + { type: 'summary_text', text: 'summary' }, + ], + reasoningContent: null, + }, + }, + ); +}); + +test('rejects summary boundaries that disagree with canonical text', () => { + const state = { + version: 1, + profile: 'alibaba-token-plan-cn', + itemId: 'summary-item', + summaryPartLengths: [8], + } as const; + assert.throws( + () => + replayPlaintextResponsesProviderOptions({ + providerOptionsKey: 'alibaba-token-plan-cn', + state: { ...state, summaryPartLengths: [3] }, + text: 'expected', + }), + /summary boundaries do not match text/, + ); +}); + +test('keeps encrypted OpenAI item identity readable for shared step grouping', () => { + assert.equal( + responsesReasoningItemId({ + openai: { itemId: 'openai-item', reasoningEncryptedContent: 'encrypted' }, + }), + 'openai-item', + ); +}); diff --git a/packages/runtime/src/__tests__/responses-wire-contract.test.ts b/packages/runtime/src/__tests__/responses-wire-contract.test.ts index 99b1bc93cf..5f2086d9c1 100644 --- a/packages/runtime/src/__tests__/responses-wire-contract.test.ts +++ b/packages/runtime/src/__tests__/responses-wire-contract.test.ts @@ -19,6 +19,7 @@ import assert from 'node:assert/strict'; import { describe, test } from 'node:test'; +import { streamText, type ToolSet } from 'ai'; import type { LlmConnection } from '@maka/core/llm-connections'; import type { RuntimeEvent } from '@maka/core/runtime-event'; import { modelMetadataIdsForProvider } from '@maka/core/model-metadata'; @@ -26,6 +27,10 @@ import { PROVIDER_REGISTRY } from '@maka/core/llm-connections'; import { thinkingVariantsForModel } from '@maka/core/model-thinking'; import { buildProviderOptions, getAIModel } from '../model-factory.js'; import { resolveModelRuntime } from '../model-runtime.js'; +import { + RUNTIME_PROVIDER_PROFILE_IDS, + resolveRuntimeProviderAdapter, +} from '../provider-runtime-policy.js'; import { lowerModelTools } from '../model-adapter.js'; import { openAiCodexCompactionMessages } from '../openai-codex-history-compactor.js'; import { openAiResponsesBaseUrl, openResponsesUrl } from '../provider-urls.js'; @@ -51,13 +56,21 @@ function openAiNamespace(options: Record): Record { - test('keeps Qwen3.8 Max on Token Plan Chat until the provider adapter supports Responses', () => { + test('routes only Qwen3.8 Max through Token Plan Responses', () => { for (const providerType of ['alibaba-token-plan-cn', 'alibaba-token-plan'] as const) { assert.equal( resolveModelRuntime({ providerType }, 'qwen3.8-max').wire, - 'openai-chat', + 'openai-responses', providerType, ); + assert.equal(resolveModelRuntime({ providerType }, 'qwen3.7-max').wire, 'openai-chat'); + assert.equal( + resolveModelRuntime( + { providerType, models: [{ id: 'qwen3.8-max', apiProtocol: 'openai-chat' }] }, + 'qwen3.8-max', + ).wire, + 'openai-chat', + ); } }); @@ -147,6 +160,26 @@ describe('responses wire contract', () => { kind: 'responses', contract: { adapter: 'open-responses', reasoningReplay: 'plaintext-content' }, }); + assert.equal(deepseek.responsesProviderOptionsKey, undefined); + assert.equal(deepseek.responsesReplayProfile, undefined); + + const alibaba = resolveModelRuntime({ providerType: 'alibaba-token-plan-cn' }, 'qwen3.8-max'); + assert.deepEqual(alibaba.reasoningReplay, { + kind: 'responses', + contract: { + adapter: 'open-responses', + reasoningReplay: 'plaintext-summary', + compatibility: 'alibaba-token-plan', + }, + }); + assert.equal(alibaba.responsesProviderOptionsKey, 'alibaba-token-plan-cn'); + assert.equal(alibaba.responsesReplayProfile, 'alibaba-token-plan-cn'); + const accountScopedAlibaba = resolveModelRuntime( + { providerType: 'alibaba-token-plan-cn', slug: 'token-plan-account-a' }, + 'qwen3.8-max', + ); + assert.equal(accountScopedAlibaba.responsesProviderOptionsKey, 'alibaba-token-plan-cn'); + assert.equal(accountScopedAlibaba.responsesReplayProfile, 'token-plan-account-a'); const xai = resolveModelRuntime({ providerType: 'xai' }, 'grok-4.5'); assert.deepEqual(xai.reasoningReplay, { @@ -195,9 +228,19 @@ describe('responses wire contract', () => { ); }); + test('Core Runtime-profile references exactly match Runtime implementations', () => { + const referenced = Object.values(PROVIDER_REGISTRY).flatMap((definition) => { + const adapter = definition.runtimeAdapter; + return adapter.kind === 'openai-compatible' && adapter.runtimeProfile + ? [adapter.runtimeProfile] + : []; + }); + assert.deepEqual([...new Set(referenced)].sort(), [...RUNTIME_PROVIDER_PROFILE_IDS].sort()); + }); + test('enables Responses only through an explicit supported contract', () => { const configured = Object.entries(PROVIDER_REGISTRY).flatMap(([providerType, definition]) => { - const adapter = definition.runtimeAdapter; + const adapter = resolveRuntimeProviderAdapter(definition.runtimeAdapter); return adapter.kind === 'openai-compatible' && adapter.responses ? [{ providerType, contract: adapter.responses }] : []; @@ -216,6 +259,22 @@ describe('responses wire contract', () => { providerType: 'xai-oauth', contract: { adapter: 'openai', reasoningReplay: 'encrypted-content' }, }, + { + providerType: 'alibaba-token-plan-cn', + contract: { + adapter: 'open-responses', + reasoningReplay: 'plaintext-summary', + compatibility: 'alibaba-token-plan', + }, + }, + { + providerType: 'alibaba-token-plan', + contract: { + adapter: 'open-responses', + reasoningReplay: 'plaintext-summary', + compatibility: 'alibaba-token-plan', + }, + }, ]); const relay = PROVIDER_REGISTRY['openai-responses-compatible'].runtimeAdapter; @@ -246,7 +305,7 @@ describe('responses wire contract', () => { if ( runtime.wire !== 'openai-responses' || (runtime.reasoningReplay.kind === 'responses' && - runtime.reasoningReplay.contract.reasoningReplay === 'plaintext-content') + runtime.reasoningReplay.contract.adapter === 'open-responses') ) { continue; } @@ -270,6 +329,340 @@ describe('responses wire contract', () => { }); describe('responses wire request body', () => { + test('Alibaba compatibility owns store:false after request overlays', async () => { + const bodies: Array> = []; + const urls: string[] = []; + const fetch = (async (url: string | URL | Request, init?: RequestInit) => { + urls.push(String(url)); + bodies.push(JSON.parse(String(init?.body))); + return Response.json({ + id: 'response-1', + object: 'response', + created_at: 1, + model: 'qwen3.8-max', + status: 'completed', + output: [], + usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 }, + }); + }) as unknown as typeof globalThis.fetch; + const connection = { + ...conn('alibaba-token-plan-cn'), + baseUrl: 'https://token-plan.example/compatible-mode/v1', + requestBodyOverlay: { store: true }, + }; + const model = getAIModel({ + connection, + apiKey: 'token-plan-key', + modelId: 'qwen3.8-max', + fetch, + }); + + await model.doGenerate({ + prompt: [{ role: 'user', content: [{ type: 'text', text: 'ping' }] }], + providerOptions: buildProviderOptions(connection, 'qwen3.8-max', 'medium'), + }); + + assert.deepEqual(urls, ['https://token-plan.example/compatible-mode/v1/responses']); + assert.equal(bodies[0]?.store, false); + assert.deepEqual(bodies[0]?.reasoning, { effort: 'medium' }); + }); + + test('Alibaba compatibility survives header-only request customization', async () => { + let body: Record | undefined; + let headers: Headers | undefined; + const fetch = (async (_url: string | URL | Request, init?: RequestInit) => { + body = JSON.parse(String(init?.body)); + headers = new Headers(init?.headers); + return Response.json({ + id: 'response-header-customization', + object: 'response', + created_at: 1, + model: 'qwen3.8-max', + status: 'completed', + output: [], + usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 }, + }); + }) as unknown as typeof globalThis.fetch; + const connection = { + ...conn('alibaba-token-plan-cn'), + baseUrl: 'https://token-plan.example/compatible-mode/v1', + }; + const model = getAIModel({ + connection, + apiKey: 'token-plan-key', + modelId: 'qwen3.8-max', + requestHeaders: { 'x-token-plan-routing': 'custom' }, + fetch, + }); + + await model.doGenerate({ + prompt: [{ role: 'user', content: [{ type: 'text', text: 'ping' }] }], + }); + + assert.equal(body?.store, false); + assert.equal(headers?.get('x-token-plan-routing'), 'custom'); + }); + + test('Alibaba Token Plan lowers and decodes provider-executed web search', async () => { + let body: Record | undefined; + const webSearchCall = { + type: 'web_search_call', + id: 'web-search-1', + status: 'completed', + action: { + type: 'search', + query: 'latest Maka', + sources: [{ type: 'url', url: 'https://maka.example/releases' }], + }, + }; + const message = { + type: 'message', + id: 'message-1', + status: 'completed', + role: 'assistant', + content: [{ type: 'output_text', text: 'Maka shipped.', annotations: [] }], + }; + const completed = { + id: 'response-web-search', + object: 'response', + created_at: 1, + model: 'qwen3.8-max', + status: 'completed', + output: [webSearchCall, message], + usage: { + input_tokens: 10, + output_tokens: 4, + total_tokens: 14, + x_tools: { web_search: { count: 1 } }, + }, + }; + const events = [ + { type: 'response.created', response: { id: completed.id } }, + { + type: 'response.output_item.added', + output_index: 0, + item: { ...webSearchCall, status: 'in_progress', action: undefined }, + }, + { type: 'response.output_item.done', output_index: 0, item: webSearchCall }, + { type: 'response.output_item.added', output_index: 1, item: message }, + { + type: 'response.output_text.delta', + item_id: message.id, + output_index: 1, + content_index: 0, + delta: 'Maka shipped.', + }, + { type: 'response.output_item.done', output_index: 1, item: message }, + { type: 'response.completed', response: completed }, + ]; + const fetch = (async (_url: string | URL | Request, init?: RequestInit) => { + body = JSON.parse(String(init?.body)); + return new Response( + `${events.map((event) => `data: ${JSON.stringify(event)}`).join('\n\n')}\n\ndata: [DONE]\n\n`, + { status: 200, headers: { 'content-type': 'text/event-stream' } }, + ); + }) as unknown as typeof globalThis.fetch; + const connection = { + ...conn('alibaba-token-plan-cn'), + baseUrl: 'https://token-plan.example/compatible-mode/v1', + }; + const model = getAIModel({ + connection, + apiKey: 'token-plan-key', + modelId: 'qwen3.8-max', + fetch, + }); + const tools = lowerModelTools({ + WebSearch: { + kind: 'provider', + providerTool: { kind: 'openai-web-search', searchContextSize: 'medium' }, + }, + }); + const result = streamText({ + model, + messages: [{ role: 'user', content: 'What shipped?' }], + tools: tools as ToolSet, + maxRetries: 0, + }); + const parts = []; + for await (const part of result.fullStream) parts.push(part); + + assert.deepEqual(body?.tools, [{ type: 'web_search' }]); + assert.equal(body?.store, false); + const call = parts.find((part) => part.type === 'tool-call'); + assert.ok(call && call.type === 'tool-call'); + assert.equal(call.toolCallId, 'web-search-1'); + assert.equal(call.toolName, 'WebSearch'); + assert.equal(call.providerExecuted, true); + const toolResult = parts.find((part) => part.type === 'tool-result'); + assert.ok(toolResult && toolResult.type === 'tool-result'); + assert.equal(toolResult.providerExecuted, true); + assert.deepEqual(toolResult.output, { + action: { type: 'search', query: 'latest Maka' }, + sources: [{ type: 'url', url: 'https://maka.example/releases' }], + }); + assert.equal(await result.text, 'Maka shipped.'); + const finishStep = parts.find((part) => part.type === 'finish-step'); + assert.ok(finishStep && finishStep.type === 'finish-step'); + assert.deepEqual(finishStep.usage.raw?.x_tools, { web_search: { count: 1 } }); + }); + + test('Open Responses preserves search, open-page, and find-in-page actions', async () => { + let body: Record | undefined; + const calls = [ + { + type: 'web_search_call', + id: 'search-1', + status: 'completed', + action: { type: 'search', queries: ['latest Maka'] }, + }, + { + type: 'web_search_call', + id: 'open-1', + status: 'completed', + action: { type: 'open_page', url: 'https://maka.example/releases' }, + }, + { + type: 'web_search_call', + id: 'find-1', + status: 'completed', + action: { + type: 'find_in_page', + url: 'https://maka.example/releases', + pattern: 'WebSearch', + }, + }, + ]; + const completed = { + id: 'response-page-actions', + object: 'response', + created_at: 1, + model: 'deepseek-v4-flash', + status: 'completed', + output: calls, + usage: { input_tokens: 5, output_tokens: 2, total_tokens: 7 }, + }; + const events = [ + { type: 'response.created', response: { id: completed.id } }, + ...calls.flatMap((call, output_index) => [ + { + type: 'response.output_item.added', + output_index, + item: { ...call, status: 'in_progress', action: undefined }, + }, + { type: 'response.output_item.done', output_index, item: call }, + ]), + { type: 'response.completed', response: completed }, + ]; + const fetch = (async (_url: string | URL | Request, init?: RequestInit) => { + body = JSON.parse(String(init?.body)); + return new Response( + `${events.map((event) => `data: ${JSON.stringify(event)}`).join('\n\n')}\n\ndata: [DONE]\n\n`, + { status: 200, headers: { 'content-type': 'text/event-stream' } }, + ); + }) as unknown as typeof globalThis.fetch; + const model = getAIModel({ + connection: { ...conn('deepseek'), baseUrl: 'https://deepseek.example' }, + apiKey: 'deepseek-key', + modelId: 'deepseek-v4-flash', + fetch, + }); + const tools = lowerModelTools({ + WebSearch: { + kind: 'provider', + providerTool: { kind: 'openai-web-search', searchContextSize: 'medium' }, + }, + }); + const result = streamText({ + model, + messages: [{ role: 'user', content: 'Search, open, and find.' }], + tools: tools as ToolSet, + maxRetries: 0, + }); + const parts = []; + for await (const part of result.fullStream) parts.push(part); + + assert.deepEqual(body?.tools, [{ type: 'web_search' }]); + assert.deepEqual( + parts + .filter((part) => part.type === 'tool-result') + .map((part) => (part.type === 'tool-result' ? part.output : undefined)), + [ + { action: { type: 'search', queries: ['latest Maka'] } }, + { action: { type: 'openPage', url: 'https://maka.example/releases' } }, + { + action: { + type: 'findInPage', + url: 'https://maka.example/releases', + pattern: 'WebSearch', + }, + }, + ], + ); + }); + + test('Alibaba stateless request carries the reconstructed summary item', async () => { + let body: Record | undefined; + const fetch = (async (_url: string | URL | Request, init?: RequestInit) => { + body = JSON.parse(String(init?.body)); + return Response.json({ + id: 'response-2', + object: 'response', + created_at: 2, + model: 'qwen3.8-max', + status: 'completed', + output: [], + usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 }, + }); + }) as unknown as typeof globalThis.fetch; + const connection = { + ...conn('alibaba-token-plan-cn'), + baseUrl: 'https://token-plan.example/compatible-mode/v1', + }; + const model = getAIModel({ + connection, + apiKey: 'token-plan-key', + modelId: 'qwen3.8-max', + fetch, + }); + + await model.doGenerate({ + prompt: [ + { role: 'user', content: [{ type: 'text', text: 'first question' }] }, + { + role: 'assistant', + content: [ + { + type: 'reasoning', + text: 'durable reasoning summary', + providerOptions: { + 'alibaba-token-plan-cn': { + itemId: 'reasoning-item-1', + reasoningSummary: [{ type: 'summary_text', text: 'durable reasoning summary' }], + reasoningContent: null, + }, + }, + }, + { type: 'text', text: 'first answer' }, + ], + }, + { role: 'user', content: [{ type: 'text', text: 'follow up' }] }, + ], + }); + + assert.equal(body?.store, false); + assert.deepEqual( + (body?.input as Array> | undefined)?.find( + (item) => item.type === 'reasoning', + ), + { + type: 'reasoning', + id: 'reasoning-item-1', + summary: [{ type: 'summary_text', text: 'durable reasoning summary' }], + }, + ); + }); + test('adds the V2 trigger only through the explicit OpenAI provider option', async () => { const bodies: Array> = []; const fetch = (async (_url: string | URL | Request, init?: RequestInit) => { diff --git a/packages/runtime/src/ai-sdk-backend.ts b/packages/runtime/src/ai-sdk-backend.ts index 48bb09af68..d8032d6159 100644 --- a/packages/runtime/src/ai-sdk-backend.ts +++ b/packages/runtime/src/ai-sdk-backend.ts @@ -172,6 +172,11 @@ import { type RequestProjectionContext, type RequestProjectionStage, } from './request-projection.js'; +import { + decodePlaintextResponsesReasoningState, + replayPlaintextResponsesProviderOptions, + responsesReasoningItemId, +} from './responses-reasoning-state.js'; import type { ActiveToolResultPruneDiagnosticPatch } from './active-tool-result-prune.js'; import { toolResultOutput } from './tool-result-output.js'; import { compactionDecisionDiagnosticPatch } from './compaction-boundary.js'; @@ -1456,6 +1461,7 @@ export class AiSdkBackend implements AgentBackend { let sawStepThinking = false; let stepThinkingProviderOptions: NonNullable | undefined; let stepResponsesThinkingParts: AssistantThinkingPart[] = []; + let stepResponsesThinkingPartsByItemId = new Map(); let stepSignature: string | undefined; const startedAt = this.now(); @@ -1551,6 +1557,7 @@ export class AiSdkBackend implements AgentBackend { sawStepThinking = false; stepThinkingProviderOptions = undefined; stepResponsesThinkingParts = []; + stepResponsesThinkingPartsByItemId = new Map(); stepSignature = undefined; }; let tokenUsage: NormalizedAiSdkUsage | undefined; @@ -2301,32 +2308,58 @@ export class AiSdkBackend implements AgentBackend { } stepThinkingProviderOptions = event.providerOptions; } - const openai = event.providerOptions?.openai; const itemId = - openai && typeof openai === 'object' && !Array.isArray(openai) - ? (openai as { itemId?: unknown }).itemId - : undefined; + event.reasoningItemId ?? responsesReasoningItemId(event.providerOptions); if (typeof itemId === 'string' && itemId.length > 0) { - let part = stepResponsesThinkingParts.find( - (candidate) => - (candidate.providerOptions?.openai as { itemId?: unknown } | undefined) - ?.itemId === itemId, - ); + let part = stepResponsesThinkingPartsByItemId.get(itemId); + if ( + part && + event.providerOptions === undefined && + decodePlaintextResponsesReasoningState(part.providerOptions).kind === 'valid' + ) { + // The SDK does not suppress a stray delta after + // output_item.done. Keep it out of the finalized item or + // its durable summary boundaries will no longer match. + part = { text: '' }; + stepResponsesThinkingParts.push(part); + stepResponsesThinkingPartsByItemId.set(itemId, part); + } if (!part) { part = { text: stepResponsesThinkingParts.length === 0 && event.text.length === 0 ? stepThinking : '', - providerOptions: event.providerOptions, }; stepResponsesThinkingParts.push(part); - } else { + stepResponsesThinkingPartsByItemId.set(itemId, part); + } + const nextPartText = part.text + event.text; + if ( + event.reasoningSummaryText !== undefined && + event.reasoningSummaryText !== nextPartText + ) { + throw new Error( + 'Streamed plaintext Responses reasoning does not match final provider summary', + ); + } + part.text = nextPartText; + if (event.providerOptions !== undefined) { part.providerOptions = event.providerOptions; } - part.text += event.text; } else if (stepResponsesThinkingParts.length > 0) { - stepResponsesThinkingParts.at(-1)!.text += event.text; + const lastPart = stepResponsesThinkingParts.at(-1)!; + const lastState = decodePlaintextResponsesReasoningState( + lastPart.providerOptions, + ); + if (lastState.kind === 'valid') { + // An invalid next item has no usable stream id. Do not + // append its deltas to the finalized item: partial-error + // flush must keep that item's durable boundaries valid. + stepResponsesThinkingParts.push({ text: event.text }); + } else { + lastPart.text += event.text; + } } queue.push({ type: 'thinking_delta', @@ -3751,15 +3784,42 @@ export class AiSdkBackend implements AgentBackend { } : undefined; } - if (replaySupport.responsesReasoning === 'plaintext-content') { - if (item.text.length === 0) return undefined; + if ( + typeof replaySupport.responsesReasoning === 'object' && + replaySupport.responsesReasoning.kind === 'plaintext-item' + ) { + const decoded = decodePlaintextResponsesReasoningState(item.providerOptions); + if (decoded.kind === 'missing') return undefined; + if (decoded.kind === 'unsupported-version') return undefined; + if (decoded.kind === 'malformed') { + if ( + decoded.profile !== undefined && + decoded.profile !== replaySupport.responsesReasoning.profile + ) { + return undefined; + } + throw new Error('Malformed durable plaintext Responses reasoning state'); + } + const state = decoded.state; + if (state.profile !== replaySupport.responsesReasoning.profile) { + return undefined; + } return { part: { type: 'reasoning' as const, text: item.text, + providerOptions: replayPlaintextResponsesProviderOptions({ + providerOptionsKey: replaySupport.responsesReasoning.providerOptionsKey, + state, + text: item.text, + }), }, }; } + if (replaySupport.responsesReasoning === 'plaintext-content') { + if (item.text.length === 0) return undefined; + return { part: { type: 'reasoning' as const, text: item.text } }; + } if (replaySupport.responsesReasoning === 'encrypted-content') { const openai = item.providerOptions?.openai; if (openai && typeof openai === 'object' && !Array.isArray(openai)) { diff --git a/packages/runtime/src/model-adapter.ts b/packages/runtime/src/model-adapter.ts index a21f89f3e1..b080ea4947 100644 --- a/packages/runtime/src/model-adapter.ts +++ b/packages/runtime/src/model-adapter.ts @@ -56,6 +56,10 @@ export type { } from './model-protocol.js'; import { resolveModelRuntime, type ResolvedModelRuntime } from './model-runtime.js'; +import { + plaintextResponsesReasoningProviderOptions, + safePlaintextResponsesReasoningItemId, +} from './responses-reasoning-state.js'; import { classifyError, errorPresentationFromClass, @@ -186,9 +190,17 @@ export class ModelAdapter { // recorded to the event log and rendered regardless. unsignedThinking: this.runtime.reasoningReplay.kind === 'openai-chat-plaintext', responsesReasoning: - this.runtime.reasoningReplay.kind === 'responses' - ? this.runtime.reasoningReplay.contract.reasoningReplay - : 'none', + this.runtime.reasoningReplay.kind !== 'responses' + ? 'none' + : this.runtime.reasoningReplay.contract.reasoningReplay === 'encrypted-content' + ? 'encrypted-content' + : this.runtime.reasoningReplay.contract.reasoningReplay === 'plaintext-content' + ? 'plaintext-content' + : { + kind: 'plaintext-item', + profile: requireResponsesReplayProfile(this.runtime), + providerOptionsKey: requireResponsesProviderOptionsKey(this.runtime), + }, }; } @@ -327,6 +339,7 @@ export class ModelAdapter { ? this.openAiChatReasoningTransportState : undefined; const openAiResponsesTransportState = this.openAiResponsesTransportState; + const resolvedRuntime = this.runtime; let settleOutcome!: (outcome: ModelStepOutcome) => void; const outcome = new Promise((resolve) => { settleOutcome = resolve; @@ -337,10 +350,33 @@ export class ModelAdapter { let failure: ModelFailure | undefined; let sawFinish = false; let streamedFinishReason: string | undefined; + let streamedRawFinishReason: string | undefined; + let sawUnfinalizedPlaintextSummary = false; try { for await (const chunk of sdk.stream as AsyncIterable) { onStreamActivity(); - for (const event of translateChunk(chunk, openAiChatReasoningTransportState)) { + if ( + chunk.type === 'finish' || + chunk.type === 'finish-step' || + chunk.type === 'step-finish' + ) { + streamedRawFinishReason = + rawFinishReasonString(chunk.rawFinishReason) ?? streamedRawFinishReason; + } + if (isUnfinalizedPlaintextSummaryReasoningEnd(chunk, resolvedRuntime)) { + // The SDK emits this trailer from flush() when no + // response.output_item.done finalized the active item. Defer the + // decision until the terminal outcome is known: an existing + // provider failure must win, while a successful stream must + // still fail closed instead of losing replay state silently. + sawUnfinalizedPlaintextSummary = true; + continue; + } + for (const event of translateChunk( + chunk, + openAiChatReasoningTransportState, + resolvedRuntime, + )) { if (event.kind === 'error') failure = event.failure; if (event.kind === 'finish') sawFinish = true; if (event.kind === 'finish' || event.kind === 'step-finish') { @@ -350,24 +386,56 @@ export class ModelAdapter { } } } catch (error) { - failure = normalizeProviderFailure(error); - yield { kind: 'error', failure }; + if (!failure) { + failure = normalizeProviderFailure(error); + yield { kind: 'error', failure }; + } } finally { const [sdkUsage, sdkFinishReason] = await Promise.all([ sdk.usage.catch(() => undefined), sdk.finishReason.catch(() => undefined), ]); + // An early-stopping consumer (a provider-mismatch throw, a user + // stop) ends this stream before any finish chunk exists, so the SDK + // rejects every result promise during teardown. `usage` and + // `finishReason` are consumed above; `response` is only read on the + // completed continuation path below. Sink it unconditionally so the + // error path can never surface an unhandled rejection after the + // turn unwinds — the timing of that settlement is scheduler-owned + // (observed post-test on Windows), and Node's default makes an + // unhandled rejection a crash. + void Promise.resolve(sdk.response).catch(() => undefined); const finishReason = streamedFinishReason ?? rawFinishReasonString(sdkFinishReason) ?? 'unknown'; - const usage = normalizeAiSdkUsage(sdkUsage, { rawFinishReason: finishReason }); + const rawFinishReason = + streamedRawFinishReason ?? rawFinishReasonString(sdkFinishReason) ?? finishReason; + const usage = normalizeAiSdkUsage(sdkUsage, { rawFinishReason }); let settled = settleModelStepOutcome({ aborted: continuation.abortSignal.aborted, failure, sawFinish, finishReason, + rawFinishReason, usage, request, }); + let deferredFailure: ModelFailure | undefined; + + if (sawUnfinalizedPlaintextSummary && settled.kind === 'completed') { + failure = normalizeProviderFailure( + new Error('Plaintext Responses reasoning item is missing final summary metadata'), + ); + deferredFailure = failure; + settled = settleModelStepOutcome({ + aborted: continuation.abortSignal.aborted, + failure, + sawFinish, + finishReason, + rawFinishReason, + usage, + request, + }); + } try { if (continuation.lane) { @@ -392,6 +460,12 @@ export class ModelAdapter { } finally { settleOutcome(settled); } + if (deferredFailure) { + // Consumers may stop iterating at the first error. The outcome and + // continuation lane must already be settled before this yield so a + // generator return cannot strand the caller awaiting result.outcome. + yield { kind: 'error', failure: deferredFailure }; + } } }, }; @@ -427,6 +501,7 @@ export class ModelAdapter { this.runtime.reasoningReplay.kind === 'openai-chat-plaintext' ? this.openAiChatReasoningTransportState : undefined, + this.runtime, ); } @@ -473,12 +548,13 @@ interface ModelStepSettlementEvidence { failure?: ModelFailure; sawFinish: boolean; finishReason: ModelFinishReason; + rawFinishReason?: string; usage?: NormalizedUsage; request: ModelRequestMetadata; } export function settleModelStepOutcome(evidence: ModelStepSettlementEvidence): ModelStepOutcome { - const { aborted, failure, sawFinish, finishReason, usage, request } = evidence; + const { aborted, failure, sawFinish, finishReason, rawFinishReason, usage, request } = evidence; if (aborted || failure?.kind === 'abort') { return failedStepOutcome( 'aborted', @@ -507,14 +583,13 @@ export function settleModelStepOutcome(evidence: ModelStepSettlementEvidence): M ); } if (finishReason === 'content-filter' || finishReason === 'error') { + const terminalFailure = + finishReason === 'error' + ? providerFinishFailure(rawFinishReason) + : modelStepFailure('unknown', 'Provider stopped the stream on a content filter'); return failedStepOutcome( - 'terminal-failure', - modelStepFailure( - finishReason === 'content-filter' ? 'unknown' : 'provider_unavailable', - finishReason === 'content-filter' - ? 'Provider stopped the stream on a content filter' - : 'Provider stopped the stream with an error', - ), + terminalFailure.retryable ? 'retryable-failure' : 'terminal-failure', + terminalFailure, request, usage, ); @@ -532,6 +607,24 @@ function modelStepFailure(kind: ModelFailureKind, message: string): ModelFailure return { type: 'model_failure', kind, message, retryable: false }; } +function providerFinishFailure(rawFinishReason: string | undefined): ModelFailure { + if (rawFinishReason && rawFinishReason !== 'error') { + const normalized = normalizeProviderFailure({ + code: rawFinishReason, + message: 'Provider stopped the stream with an error', + }); + // A finish reason carries no request-level Retry-After or transport + // evidence. Preserve its classification for diagnostics without widening + // the pre-existing retry policy for every provider. + if (normalized.kind !== 'unknown') return { ...normalized, retryable: false }; + return { + ...modelStepFailure('provider_unavailable', 'Provider stopped the stream with an error'), + ...(normalized.code ? { code: normalized.code } : {}), + }; + } + return modelStepFailure('provider_unavailable', 'Provider stopped the stream with an error'); +} + function failedStepOutcome( kind: Exclude, failure: ModelFailure, @@ -590,7 +683,29 @@ export interface ModelAdapterRuntimeEventReplaySupport { providerExecutedTools: boolean; signedThinking: boolean; unsignedThinking: boolean; - responsesReasoning: 'none' | 'encrypted-content' | 'plaintext-content'; + responsesReasoning: + | 'none' + | 'encrypted-content' + | 'plaintext-content' + | { + kind: 'plaintext-item'; + profile: string; + providerOptionsKey: string; + }; +} + +function requireResponsesProviderOptionsKey(runtime: ResolvedModelRuntime): string { + if (!runtime.responsesProviderOptionsKey) { + throw new Error('Plaintext Responses replay requires a provider-options key'); + } + return runtime.responsesProviderOptionsKey; +} + +function requireResponsesReplayProfile(runtime: ResolvedModelRuntime): string { + if (!runtime.responsesReplayProfile) { + throw new Error('Plaintext Responses replay requires a source profile'); + } + return runtime.responsesReplayProfile; } /** @@ -669,8 +784,48 @@ function reasoningSignatureFromChunk(chunk: AiSdkStreamChunk): string | undefine function openAiResponsesReasoningProviderOptionsFromChunk( chunk: AiSdkStreamChunk, + runtime: ResolvedModelRuntime, ): NonNullable | undefined { const meta = chunk.providerMetadata; + if ( + runtime.reasoningReplay.kind === 'responses' && + runtime.reasoningReplay.contract.reasoningReplay === 'plaintext-summary' + ) { + if (chunk.type !== 'reasoning' && chunk.type !== 'reasoning-end') return undefined; + const providerOptionsKey = runtime.responsesProviderOptionsKey; + const provider = + providerOptionsKey && meta && typeof meta === 'object' + ? (meta as Record)[providerOptionsKey] + : undefined; + const metadataItemIdValue = + provider && typeof provider === 'object' && !Array.isArray(provider) + ? (provider as { itemId?: unknown }).itemId + : undefined; + const streamItemIdValue = (chunk as { id?: unknown }).id; + if ( + typeof metadataItemIdValue === 'string' && + typeof streamItemIdValue === 'string' && + metadataItemIdValue !== streamItemIdValue + ) { + throw new Error('Plaintext Responses reasoning item id changed within one stream item'); + } + const itemId = + safePlaintextResponsesReasoningItemId(metadataItemIdValue) ?? + safePlaintextResponsesReasoningItemId(streamItemIdValue); + const summaryParts = plaintextSummaryParts(provider); + if (!itemId || !summaryParts) { + throw new Error('Plaintext Responses reasoning item is missing final summary metadata'); + } + const providerOptions = plaintextResponsesReasoningProviderOptions( + itemId, + requireResponsesReplayProfile(runtime), + summaryParts, + ); + if (!providerOptions) { + throw new Error('Plaintext Responses reasoning summary exceeds durable state bounds'); + } + return providerOptions; + } if (!meta || typeof meta !== 'object') return undefined; const openai = (meta as { openai?: unknown }).openai; if (!openai || typeof openai !== 'object' || Array.isArray(openai)) return undefined; @@ -689,6 +844,65 @@ function openAiResponsesReasoningProviderOptionsFromChunk( }; } +function isUnfinalizedPlaintextSummaryReasoningEnd( + chunk: AiSdkStreamChunk, + runtime: ResolvedModelRuntime, +): boolean { + return ( + runtime.reasoningReplay.kind === 'responses' && + runtime.reasoningReplay.contract.reasoningReplay === 'plaintext-summary' && + chunk.type === 'reasoning-end' && + (chunk.providerMetadata === undefined || chunk.providerMetadata === null) + ); +} + +function plaintextSummaryParts(provider: unknown): string[] | undefined { + if (!provider || typeof provider !== 'object' || Array.isArray(provider)) return undefined; + const summary = (provider as { reasoningSummary?: unknown }).reasoningSummary; + if (!Array.isArray(summary)) return undefined; + const parts: string[] = []; + for (const part of summary) { + if (!part || typeof part !== 'object' || Array.isArray(part)) return undefined; + const { type, text } = part as { type?: unknown; text?: unknown }; + if (type !== 'summary_text' || typeof text !== 'string') return undefined; + parts.push(text); + } + return parts; +} + +function plaintextSummaryTextFromChunk( + chunk: AiSdkStreamChunk, + runtime: ResolvedModelRuntime | undefined, +): string | undefined { + if ( + runtime?.reasoningReplay.kind !== 'responses' || + runtime.reasoningReplay.contract.reasoningReplay !== 'plaintext-summary' || + (chunk.type !== 'reasoning' && chunk.type !== 'reasoning-end') + ) { + return undefined; + } + const providerOptionsKey = runtime.responsesProviderOptionsKey; + const meta = chunk.providerMetadata; + const provider = + providerOptionsKey && meta && typeof meta === 'object' + ? (meta as Record)[providerOptionsKey] + : undefined; + return plaintextSummaryParts(provider)?.join(''); +} + +function plaintextSummaryItemIdFromChunk( + chunk: AiSdkStreamChunk, + runtime: ResolvedModelRuntime | undefined, +): string | undefined { + if ( + runtime?.reasoningReplay.kind !== 'responses' || + runtime.reasoningReplay.contract.reasoningReplay !== 'plaintext-summary' + ) { + return undefined; + } + return safePlaintextResponsesReasoningItemId((chunk as { id?: unknown }).id); +} + /** * Translate one raw AI SDK stream chunk into zero or more Maka-owned * `ModelStreamEvent`s. The sole site that parses SDK chunk names; the backend @@ -697,8 +911,13 @@ function openAiResponsesReasoningProviderOptionsFromChunk( function translateChunk( chunk: AiSdkStreamChunk, openAiChatReasoningTransportState?: OpenAiChatReasoningTransportState, + runtime?: ResolvedModelRuntime, ): ModelStreamEvent[] { switch (chunk.type) { + case 'reasoning-start': { + const reasoningItemId = plaintextSummaryItemIdFromChunk(chunk, runtime); + return reasoningItemId ? [{ kind: 'thinking', text: '', reasoningItemId }] : []; + } case 'text-start': return [{ kind: 'text-start' }]; case 'text-delta': { @@ -725,7 +944,11 @@ function translateChunk( ? chunk.delta : undefined; const signature = reasoningSignatureFromChunk(chunk); - const responsesProviderOptions = openAiResponsesReasoningProviderOptionsFromChunk(chunk); + const responsesProviderOptions = runtime + ? openAiResponsesReasoningProviderOptionsFromChunk(chunk, runtime) + : undefined; + const reasoningItemId = plaintextSummaryItemIdFromChunk(chunk, runtime); + const reasoningSummaryText = plaintextSummaryTextFromChunk(chunk, runtime); const events: ModelStreamEvent[] = []; if (signature) events.push({ kind: 'thinking-signature', signature }); // The signed reasoning chunk arrives as a standalone delta with empty @@ -745,17 +968,31 @@ function translateChunk( providerOptionsOrigin: 'maka_transport' as const, } : {}), + ...(reasoningItemId ? { reasoningItemId } : {}), + ...(reasoningSummaryText !== undefined ? { reasoningSummaryText } : {}), }); } return events; } case 'reasoning-end': { const signature = reasoningSignatureFromChunk(chunk); - const responsesProviderOptions = openAiResponsesReasoningProviderOptionsFromChunk(chunk); + const responsesProviderOptions = runtime + ? openAiResponsesReasoningProviderOptionsFromChunk(chunk, runtime) + : undefined; + const reasoningItemId = plaintextSummaryItemIdFromChunk(chunk, runtime); + const reasoningSummaryText = plaintextSummaryTextFromChunk(chunk, runtime); return [ ...(signature ? [{ kind: 'thinking-signature' as const, signature }] : []), ...(responsesProviderOptions - ? [{ kind: 'thinking' as const, text: '', providerOptions: responsesProviderOptions }] + ? [ + { + kind: 'thinking' as const, + text: '', + providerOptions: responsesProviderOptions, + ...(reasoningItemId ? { reasoningItemId } : {}), + ...(reasoningSummaryText !== undefined ? { reasoningSummaryText } : {}), + }, + ] : []), ]; } @@ -771,9 +1008,12 @@ function translateChunk( case 'finish-step': case 'step-finish': { const finishReason = chunkFinishReason(chunk); + const rawFinishReason = rawFinishReasonString(chunk.rawFinishReason); // The same value the turn's outcome is decided from, so the record and // the outcome cannot name different reasons for the same stream. - const usage = normalizeAiSdkUsage(chunk.usage, { rawFinishReason: finishReason }); + const usage = normalizeAiSdkUsage(chunk.usage, { + rawFinishReason: rawFinishReason ?? finishReason, + }); return [ { kind: 'step-finish', @@ -786,7 +1026,6 @@ function translateChunk( const finishReason = chunkFinishReason(chunk); return [{ kind: 'finish', ...(finishReason ? { finishReason } : {}) }]; } - case 'reasoning-start': case 'start-step': case 'tool-result': case 'tool-error': { diff --git a/packages/runtime/src/model-factory.ts b/packages/runtime/src/model-factory.ts index 37dd1216be..4803582e08 100644 --- a/packages/runtime/src/model-factory.ts +++ b/packages/runtime/src/model-factory.ts @@ -32,7 +32,6 @@ import { type SharedV4ProviderOptions, } from '@ai-sdk/provider'; import { type RuntimeExecutionConnection } from '@maka/core/llm-connections'; -import type { ProviderRuntimeAdapter } from '@maka/core/llm-connections'; import type { ThinkingLevel } from '@maka/core/model-thinking'; import { resolveThinkingLevel, @@ -53,7 +52,9 @@ import { openAiResponsesBaseUrl, openResponsesUrl, } from './provider-urls.js'; +import { createOpenResponsesCompatibilityFinalizer } from './open-responses-compatibility.js'; import { resolveModelRuntime, type ResolvedModelRuntime } from './model-runtime.js'; +import { runtimeProviderName, type RuntimeProviderAdapter } from './provider-runtime-policy.js'; import { openAiCodexHeaders } from './subscription-auth.js'; import { createRequestCustomizationFetch } from './request-customization-fetch.js'; @@ -85,10 +86,12 @@ export function getAIModel(input: ModelFactoryInput): LanguageModelV4 { const hasRequestCustomization = Object.keys(requestHeaders ?? {}).length > 0 || Object.keys(connection.requestBodyOverlay ?? {}).length > 0; - const requestFetch = createRequestCustomizationFetch(fetch ?? globalThis.fetch, { + const baseFetch = fetch ?? globalThis.fetch; + const requestCustomization = { headers: requestHeaders, bodyOverlay: connection.requestBodyOverlay, - }); + } as const; + const requestFetch = createRequestCustomizationFetch(baseFetch, requestCustomization); if (adapter.kind === 'google' && adapter.normalizeBaseUrl === false) { return createGoogle({ apiKey, baseURL, fetch: requestFetch }).chat(modelId); @@ -174,11 +177,20 @@ export function getAIModel(input: ModelFactoryInput): LanguageModelV4 { throw new Error('Responses wire requires a Responses continuation contract'); } if (reasoningReplay.contract.adapter === 'open-responses') { + // Request customization is applied first; provider compatibility is + // the final authority before network dispatch, so an overlay cannot + // re-enable storage or force a tool-choice shape the provider rejects. + const responsesFetch = createRequestCustomizationFetch(baseFetch, { + ...requestCustomization, + finalizeBody: createOpenResponsesCompatibilityFinalizer( + reasoningReplay.contract.compatibility, + ), + }); return createOpenResponses({ - name: openAiCompatibleProviderName(adapter, connection), + name: runtimeProviderName(adapter, connection), apiKey, url: openResponsesUrl(baseURL), - fetch: requestFetch, + fetch: responsesFetch, })(modelId); } return createOpenAI({ @@ -205,7 +217,7 @@ export function getAIModel(input: ModelFactoryInput): LanguageModelV4 { ) : reasoningTransport.transformRequestBody; const model = createOpenAICompatible({ - name: openAiCompatibleProviderName(adapter, connection), + name: runtimeProviderName(adapter, connection), apiKey, baseURL, includeUsage: adapter.includeUsage, @@ -622,7 +634,7 @@ function buildFamilyWire( // openai-compatible — so key by the same name getAIModel passes. return reasoningEffort || serviceTier ? { - [openAiCompatibleProviderName(adapter, connection)]: { + [runtimeProviderName(adapter, connection)]: { ...(reasoningEffort ? { reasoningEffort } : {}), ...(serviceTier ? { serviceTier } : {}), }, @@ -684,20 +696,6 @@ function buildFamilyWire( } } -/** - * The provider IDENTITY passed as `name` to `createOpenAICompatible` in - * `getAIModel` — the raw slug for custom relays. Distinct from the - * providerOptions key the SDK wants: see `openAiCompatibleProviderOptionsKey`. - */ -function openAiCompatibleProviderName( - adapter: ProviderRuntimeAdapter, - connection: RuntimeExecutionConnection, -): string { - return adapter.kind === 'openai-compatible' && adapter.name === 'connection' - ? connection.slug - : connection.providerType; -} - // Mirrors @ai-sdk/openai-compatible's own toCamelCase derivation, so the // key we emit always matches the alias the SDK resolves. function toCamelCase(name: string): string { @@ -718,8 +716,8 @@ function toCamelCase(name: string): string { * silently read nothing for dashed providers. */ function openAiCompatibleProviderOptionsKey( - adapter: ProviderRuntimeAdapter, + adapter: RuntimeProviderAdapter, connection: RuntimeExecutionConnection, ): string { - return toCamelCase(openAiCompatibleProviderName(adapter, connection)); + return toCamelCase(runtimeProviderName(adapter, connection)); } diff --git a/packages/runtime/src/model-protocol.ts b/packages/runtime/src/model-protocol.ts index fe1a1db769..be9c0d0c8c 100644 --- a/packages/runtime/src/model-protocol.ts +++ b/packages/runtime/src/model-protocol.ts @@ -393,6 +393,10 @@ export type ModelStreamEvent = kind: 'thinking'; text: string; providerOptions?: ProviderOptions; + /** Bounded item identity used only while grouping one streamed reasoning item. */ + reasoningItemId?: string; + /** Final provider summary, compared before only its part boundaries are persisted. */ + reasoningSummaryText?: string; /** Maka-authored replay hint; absent provider metadata stays fail-closed. */ providerOptionsOrigin?: 'maka_transport'; } diff --git a/packages/runtime/src/model-runtime.ts b/packages/runtime/src/model-runtime.ts index 874c3e206b..bd8d249f13 100644 --- a/packages/runtime/src/model-runtime.ts +++ b/packages/runtime/src/model-runtime.ts @@ -21,7 +21,6 @@ import { PROVIDER_DEFAULTS, effectiveBaseUrl, type ModelInfo, - type ProviderResponsesContract, type ProviderRuntimeAdapter, type ProviderType, } from '@maka/core/llm-connections'; @@ -32,6 +31,12 @@ import { } from '@maka/core/model-metadata'; import { isRetiredProvider } from '@maka/core/provider-registry'; import { resolveApplyPatchProfile, type ApplyPatchProfile } from './apply-patch-profile.js'; +import { + resolveRuntimeProviderAdapter, + runtimeProviderName, + type RuntimeProviderAdapter, + type RuntimeProviderResponsesContract, +} from './provider-runtime-policy.js'; export type ModelRuntimeWire = | 'anthropic-messages' @@ -44,10 +49,10 @@ export type ReasoningReplayContract = | { kind: 'none' } | { kind: 'anthropic-signed' } | { kind: 'openai-chat-plaintext'; requestField: 'observed' | 'reasoning' } - | { kind: 'responses'; contract: ProviderResponsesContract }; + | { kind: 'responses'; contract: RuntimeProviderResponsesContract }; export interface ResolvedModelRuntime { - adapter: ProviderRuntimeAdapter; + adapter: RuntimeProviderAdapter; baseUrl: string; /** Account-advertised request wire for adapters that route per model. */ apiProtocol?: ModelInfo['apiProtocol']; @@ -57,11 +62,16 @@ export interface ResolvedModelRuntime { reasoningReplay: ReasoningReplayContract; /** Effective parallel-tool-call support after model facts and wire defaults are resolved. */ parallelToolCalls?: boolean; + /** Provider-options namespace used by durable plaintext-summary replay. */ + responsesProviderOptionsKey?: string; + /** Stable connection identity that issued a durable plaintext-summary item. */ + responsesReplayProfile?: string; /** Effective ApplyPatch contract after provider, model, and request wire are resolved. */ applyPatchProfile: ApplyPatchProfile | null; } export interface ModelRuntimeConnection { + readonly slug?: string; readonly providerType: ProviderType; readonly baseUrl?: string; readonly models?: readonly ModelInfo[]; @@ -102,7 +112,7 @@ export function resolveModelRuntime( `Kimi Coding Plan protocol must be openai-chat or anthropic-messages, received ${apiProtocol}`, ); } - const adapter: ProviderRuntimeAdapter = + const baseAdapter: ProviderRuntimeAdapter = connection.providerType === 'kimi-coding-plan' && apiProtocol === 'openai-chat' ? ({ kind: 'openai-compatible', @@ -112,12 +122,14 @@ export function resolveModelRuntime( : override ? runtimeAdapterOverride(override.npm) : defaults.runtimeAdapter; + const adapter = resolveRuntimeProviderAdapter(baseAdapter); const configuredBaseUrl = connection.baseUrl?.trim(); const resolvedBaseUrl = configuredBaseUrl ? effectiveBaseUrl(connection) : (override?.api ?? effectiveBaseUrl(connection)); const wire = resolveModelRuntimeWire(connection.providerType, modelId, adapter, apiProtocol); const parallelToolCalls = resolveParallelToolCalls(connection, modelId, adapter); + const replay = reasoningReplayContract(adapter, wire); return { adapter, baseUrl: @@ -126,8 +138,16 @@ export function resolveModelRuntime( : resolvedBaseUrl, ...(apiProtocol ? { apiProtocol } : {}), wire, - reasoningReplay: reasoningReplayContract(adapter, wire), + reasoningReplay: replay, ...(parallelToolCalls === undefined ? {} : { parallelToolCalls }), + ...(replay.kind === 'responses' && + replay.contract.adapter === 'open-responses' && + replay.contract.reasoningReplay === 'plaintext-summary' + ? { + responsesProviderOptionsKey: runtimeProviderName(adapter, connection), + responsesReplayProfile: connection.slug ?? connection.providerType, + } + : {}), applyPatchProfile: resolveApplyPatchProfile( { wire, @@ -141,7 +161,7 @@ export function resolveModelRuntime( function resolveParallelToolCalls( connection: ModelRuntimeConnection, modelId: string, - adapter: ProviderRuntimeAdapter, + adapter: RuntimeProviderAdapter, ): boolean | undefined { const stored = connection.models?.find((model) => model.id === modelId)?.capabilities ?.parallelToolCalls; @@ -177,7 +197,7 @@ export function modelUsesNativeOpenAiResponses( function resolveModelRuntimeWire( providerType: ProviderType, modelId: string, - adapter: ProviderRuntimeAdapter, + adapter: RuntimeProviderAdapter, apiProtocol: ModelInfo['apiProtocol'] | undefined, ): ModelRuntimeWire { switch (adapter.kind) { @@ -215,7 +235,7 @@ function resolveModelRuntimeWire( } function reasoningReplayContract( - adapter: ProviderRuntimeAdapter, + adapter: RuntimeProviderAdapter, wire: ModelRuntimeWire, ): ReasoningReplayContract { switch (wire) { @@ -237,7 +257,7 @@ function reasoningReplayContract( } } -function responsesContract(adapter: ProviderRuntimeAdapter): ProviderResponsesContract { +function responsesContract(adapter: RuntimeProviderAdapter): RuntimeProviderResponsesContract { if (adapter.kind === 'openai-compatible' && adapter.responses) return adapter.responses; return { adapter: 'openai', reasoningReplay: 'encrypted-content' }; } diff --git a/packages/runtime/src/open-responses-compatibility.ts b/packages/runtime/src/open-responses-compatibility.ts new file mode 100644 index 0000000000..e0e1920378 --- /dev/null +++ b/packages/runtime/src/open-responses-compatibility.ts @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import type { OpenResponsesCompatibilityProfile } from './provider-runtime-policy.js'; + +export function createOpenResponsesCompatibilityFinalizer( + profile: OpenResponsesCompatibilityProfile | undefined, +): ((body: Record) => Record) | undefined { + if (!profile) return undefined; + return (body) => { + const choice = body.tool_choice; + if (choice === 'required' || (choice !== null && typeof choice === 'object')) { + throw new Error('Alibaba Token Plan Responses does not support forced tool_choice'); + } + return { ...body, store: false }; + }; +} diff --git a/packages/runtime/src/provider-runtime-policy.ts b/packages/runtime/src/provider-runtime-policy.ts new file mode 100644 index 0000000000..6a24c64340 --- /dev/null +++ b/packages/runtime/src/provider-runtime-policy.ts @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import type { + ProviderResponsesContract, + ProviderRuntimeAdapter, + ProviderRuntimeProfileId, + ProviderType, +} from '@maka/core/llm-connections'; + +export type OpenResponsesCompatibilityProfile = 'alibaba-token-plan'; + +export type RuntimeProviderResponsesContract = + | Extract + | { + readonly adapter: 'open-responses'; + readonly reasoningReplay: 'plaintext-content' | 'plaintext-summary'; + readonly compatibility?: OpenResponsesCompatibilityProfile; + }; + +type OpenAiCompatibleRuntimeAdapter = Omit< + Extract, + 'responses' +> & { + readonly responses?: RuntimeProviderResponsesContract; +}; + +export type RuntimeProviderAdapter = + | Exclude + | OpenAiCompatibleRuntimeAdapter; + +interface RuntimeProviderIdentity { + readonly providerType: ProviderType; + readonly slug?: string; +} + +interface RuntimeProviderProfile { + readonly responses: RuntimeProviderResponsesContract; +} + +const ALIBABA_TOKEN_PLAN_RESPONSES = { + adapter: 'open-responses', + // The pinned SDK streams `response.reasoning_text.delta` as reasoning text, + // then reads durable summary boundaries from `output_item.done.item.summary`. + // Keep the raw-SSE contract test in sync if either channel changes. + reasoningReplay: 'plaintext-summary', + compatibility: 'alibaba-token-plan', +} as const satisfies RuntimeProviderResponsesContract; + +const RUNTIME_PROVIDER_PROFILES = { + 'alibaba-token-plan': { responses: ALIBABA_TOKEN_PLAN_RESPONSES }, +} as const satisfies Record; + +export const RUNTIME_PROVIDER_PROFILE_IDS = Object.freeze( + Object.keys(RUNTIME_PROVIDER_PROFILES) as ProviderRuntimeProfileId[], +); + +export function resolveRuntimeProviderAdapter( + adapter: ProviderRuntimeAdapter, +): RuntimeProviderAdapter { + if (adapter.kind !== 'openai-compatible' || adapter.runtimeProfile === undefined) { + return adapter; + } + return { ...adapter, responses: RUNTIME_PROVIDER_PROFILES[adapter.runtimeProfile].responses }; +} + +/** Raw provider identity passed to open-responses and used as its provider-options key. */ +export function runtimeProviderName( + adapter: RuntimeProviderAdapter, + connection: RuntimeProviderIdentity, +): string { + return adapter.kind === 'openai-compatible' && adapter.name === 'connection' + ? (connection.slug ?? connection.providerType) + : connection.providerType; +} diff --git a/packages/runtime/src/request-customization-fetch.ts b/packages/runtime/src/request-customization-fetch.ts index a7e8dc3de0..9fcddabdac 100644 --- a/packages/runtime/src/request-customization-fetch.ts +++ b/packages/runtime/src/request-customization-fetch.ts @@ -26,6 +26,8 @@ import { export interface RequestCustomization { readonly headers?: Readonly>; readonly bodyOverlay?: JsonObject; + /** Final provider-owned body policy, applied after caller overlays. */ + readonly finalizeBody?: (body: Record) => Record; } export function createRequestCustomizationFetch( @@ -34,7 +36,14 @@ export function createRequestCustomizationFetch( ): typeof globalThis.fetch { const headers = normalizeRequestHeaders(customization.headers ?? {}); const bodyOverlay = normalizeRequestBodyOverlay(customization.bodyOverlay ?? {}); - if (Object.keys(headers).length === 0 && Object.keys(bodyOverlay).length === 0) return upstream; + const bodyOverlayKeys = Object.keys(bodyOverlay); + if ( + Object.keys(headers).length === 0 && + bodyOverlayKeys.length === 0 && + !customization.finalizeBody + ) { + return upstream; + } return async (input, init) => { const request = new Request(input, init); @@ -48,14 +57,28 @@ export function createRequestCustomizationFetch( } let body: BodyInit | null = request.body === null ? null : await request.clone().arrayBuffer(); - if (Object.keys(bodyOverlay).length > 0 && requestHasJsonBody(request)) { - const generatedBody = await parseRequestBody(request); - for (const key of Object.keys(bodyOverlay)) { + if (bodyOverlayKeys.length > 0 || customization.finalizeBody) { + if (!requestHasJsonBody(request)) { + if (customization.finalizeBody) { + throw new Error('Request body finalizer requires a JSON object request body'); + } + return upstream(request.url, requestInit(request, nextHeaders, body)); + } + const generatedBody = await parseRequestBody( + request, + customization.finalizeBody + ? 'Request body finalizer requires a JSON object request body' + : 'Extra request body can only be applied to a JSON object request', + ); + for (const key of bodyOverlayKeys) { if (Object.hasOwn(generatedBody, key)) { throw new Error(`Extra request body conflicts with a generated field: ${key}`); } } - body = JSON.stringify({ ...generatedBody, ...bodyOverlay }); + const customizedBody = { ...generatedBody, ...bodyOverlay }; + body = JSON.stringify( + customization.finalizeBody ? customization.finalizeBody(customizedBody) : customizedBody, + ); nextHeaders.delete('content-length'); } return upstream(request.url, requestInit(request, nextHeaders, body)); @@ -87,15 +110,18 @@ function requestHasJsonBody(request: Request): boolean { ); } -async function parseRequestBody(request: Request): Promise> { +async function parseRequestBody( + request: Request, + invalidBodyMessage: string, +): Promise> { let parsed: unknown; try { parsed = JSON.parse(await request.clone().text()); } catch { - throw new Error('Extra request body can only be applied to a JSON object request'); + throw new Error(invalidBodyMessage); } if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) { - throw new Error('Extra request body can only be applied to a JSON object request'); + throw new Error(invalidBodyMessage); } return parsed as Record; } diff --git a/packages/runtime/src/responses-reasoning-state.ts b/packages/runtime/src/responses-reasoning-state.ts new file mode 100644 index 0000000000..0303686e61 --- /dev/null +++ b/packages/runtime/src/responses-reasoning-state.ts @@ -0,0 +1,189 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import type { ModelMessage } from './model-protocol.js'; + +const STATE_KEY = 'makaResponses'; +const STATE_VERSION = 1; +const MAX_ITEM_ID_LENGTH = 512; +const MAX_PROFILE_LENGTH = 128; +const MAX_SUMMARY_PARTS = 128; +const MAX_SUMMARY_TEXT_LENGTH = 10_000_000; + +export interface PlaintextResponsesReasoningState { + readonly version: 1; + readonly profile: string; + readonly itemId: string; + readonly summaryPartLengths: readonly number[]; +} + +export type PlaintextResponsesReasoningStateDecodeResult = + | { readonly kind: 'missing' } + | { readonly kind: 'unsupported-version'; readonly version: number } + | { readonly kind: 'malformed'; readonly profile?: string } + | { readonly kind: 'valid'; readonly state: PlaintextResponsesReasoningState }; + +export function plaintextResponsesReasoningProviderOptions( + itemId: string, + profile: string, + summaryParts: readonly string[], +): NonNullable | undefined { + if (!isSafeItemId(itemId) || !isSafeProfile(profile) || !isSafeSummaryParts(summaryParts)) { + return undefined; + } + return { + [STATE_KEY]: { + version: STATE_VERSION, + profile, + itemId, + summaryPartLengths: summaryParts.map((part) => part.length), + }, + }; +} + +export function decodePlaintextResponsesReasoningState( + providerOptions: Readonly> | undefined, +): PlaintextResponsesReasoningStateDecodeResult { + const raw = providerOptions?.[STATE_KEY]; + if (raw === undefined) return { kind: 'missing' }; + if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return { kind: 'malformed' }; + const record = raw as Record; + const profile = isSafeProfile(record.profile) ? record.profile : undefined; + const itemId = isSafeItemId(record.itemId) ? record.itemId : undefined; + if (isSafeStateVersion(record.version) && record.version !== STATE_VERSION) { + return { kind: 'unsupported-version', version: record.version }; + } + const baseInvalid = record.version !== STATE_VERSION || !profile || !itemId; + if (baseInvalid) { + return { kind: 'malformed', ...(profile ? { profile } : {}) }; + } + if ( + !isSafeSummaryPartLengths(record.summaryPartLengths) || + Object.keys(record).some( + (key) => !['version', 'profile', 'itemId', 'summaryPartLengths'].includes(key), + ) + ) { + return { kind: 'malformed', ...(profile ? { profile } : {}) }; + } + return { + kind: 'valid', + state: { + version: STATE_VERSION, + profile, + itemId, + summaryPartLengths: record.summaryPartLengths, + }, + }; +} + +export function responsesReasoningItemId( + providerOptions: Readonly> | undefined, +): string | undefined { + const plaintext = decodePlaintextResponsesReasoningState(providerOptions); + if (plaintext.kind === 'valid') return plaintext.state.itemId; + const openai = providerOptions?.openai; + if (!openai || typeof openai !== 'object' || Array.isArray(openai)) return undefined; + const itemId = (openai as { itemId?: unknown }).itemId; + return isSafeItemId(itemId) ? itemId : undefined; +} + +export function replayPlaintextResponsesProviderOptions(input: { + providerOptionsKey: string; + state: PlaintextResponsesReasoningState; + text: string; +}): NonNullable { + return { + [input.providerOptionsKey]: { + itemId: input.state.itemId, + reasoningSummary: reconstructSummaryParts(input.text, input.state), + // Presence is meaningful to @ai-sdk/open-responses: null prevents its + // fallback from copying the canonical text into content when the + // provider replays reasoning through summary instead. + reasoningContent: null, + }, + }; +} + +export function safePlaintextResponsesReasoningItemId(value: unknown): string | undefined { + return isSafeItemId(value) ? value : undefined; +} + +function reconstructSummaryParts( + text: string, + state: PlaintextResponsesReasoningState, +): Array<{ type: 'summary_text'; text: string }> { + let offset = 0; + const parts = state.summaryPartLengths.map((length) => { + const part = { type: 'summary_text' as const, text: text.slice(offset, offset + length) }; + offset += length; + return part; + }); + if (offset !== text.length) { + throw new Error('Durable plaintext Responses reasoning summary boundaries do not match text'); + } + return parts; +} + +function isSafeItemId(value: unknown): value is string { + return ( + typeof value === 'string' && + value.length > 0 && + value.length <= MAX_ITEM_ID_LENGTH && + !/[\u0000-\u001f\u007f]/u.test(value) + ); +} + +function isSafeStateVersion(value: unknown): value is number { + return Number.isSafeInteger(value) && Number(value) > 0; +} + +function isSafeSummaryParts(value: readonly string[] | undefined): value is readonly string[] { + if (!value || value.length > MAX_SUMMARY_PARTS) return false; + let total = 0; + for (let index = 0; index < value.length; index += 1) { + if (!Object.hasOwn(value, index)) return false; + const part = value[index]; + if (typeof part !== 'string') return false; + total += part.length; + if (total > MAX_SUMMARY_TEXT_LENGTH) return false; + } + return true; +} + +function isSafeSummaryPartLengths(value: unknown): value is readonly number[] { + if (!Array.isArray(value) || value.length > MAX_SUMMARY_PARTS) return false; + let total = 0; + for (let index = 0; index < value.length; index += 1) { + if (!Object.hasOwn(value, index)) return false; + const length = value[index]; + if (!Number.isSafeInteger(length) || length < 0) return false; + total += length; + if (total > MAX_SUMMARY_TEXT_LENGTH) return false; + } + return true; +} + +function isSafeProfile(value: unknown): value is string { + return ( + typeof value === 'string' && + value.length > 0 && + value.length <= MAX_PROFILE_LENGTH && + !/[\u0000-\u001f\u007f]/u.test(value) + ); +} diff --git a/patches/@ai-sdk+open-responses+2.0.29.patch b/patches/@ai-sdk+open-responses+2.0.29.patch new file mode 100644 index 0000000000..e4df62a9e2 --- /dev/null +++ b/patches/@ai-sdk+open-responses+2.0.29.patch @@ -0,0 +1,383 @@ +diff --git a/node_modules/@ai-sdk/open-responses/dist/index.js b/node_modules/@ai-sdk/open-responses/dist/index.js +index 72fe7a1..caa2b72 100644 +--- a/node_modules/@ai-sdk/open-responses/dist/index.js ++++ b/node_modules/@ai-sdk/open-responses/dist/index.js +@@ -454,15 +454,19 @@ var OpenResponsesLanguageModel = class _OpenResponsesLanguageModel { + providerOptionsName: this.config.providerOptionsName + }); + warnings.push(...inputWarnings); +- const functionTools = []; ++ const responseTools = []; + for (const tool of tools != null ? tools : []) { + if (tool.type === "provider") { +- warnings.push({ +- type: "unsupported", +- feature: `provider-defined tool ${tool.id}` +- }); ++ if (tool.id === "openai.web_search") { ++ responseTools.push({ type: "web_search" }); ++ } else { ++ warnings.push({ ++ type: "unsupported", ++ feature: `provider-defined tool ${tool.id}` ++ }); ++ } + } else { +- functionTools.push({ ++ responseTools.push({ + type: "function", + name: tool.name, + description: tool.description, +@@ -515,7 +519,7 @@ var OpenResponsesLanguageModel = class _OpenResponsesLanguageModel { + summary: openResponsesOptions.reasoningSummary + } + } : void 0, +- tools: functionTools.length ? functionTools : void 0, ++ tools: responseTools.length ? responseTools : void 0, + tool_choice: convertedToolChoice, + ...textFormat != null && { text: { format: textFormat } } + }, +@@ -571,6 +575,7 @@ var OpenResponsesLanguageModel = class _OpenResponsesLanguageModel { + } + const content = []; + let hasToolCalls = false; ++ const webSearchToolName = providerWebSearchToolName(options.tools); + for (const part of response.output) { + switch (part.type) { + // TODO AI SDK 7 adjust reasoning in the specification to better support the reasoning structure from open responses. +@@ -628,6 +633,22 @@ var OpenResponsesLanguageModel = class _OpenResponsesLanguageModel { + }); + break; + } ++ case "web_search_call": { ++ content.push({ ++ type: "tool-call", ++ toolCallId: part.id, ++ toolName: webSearchToolName != null ? webSearchToolName : "web_search", ++ input: JSON.stringify({}), ++ providerExecuted: true ++ }); ++ content.push({ ++ type: "tool-result", ++ toolCallId: part.id, ++ toolName: webSearchToolName != null ? webSearchToolName : "web_search", ++ result: mapWebSearchOutput(part.action) ++ }); ++ break; ++ } + } + } + const usage = response.usage; +@@ -731,6 +752,7 @@ var OpenResponsesLanguageModel = class _OpenResponsesLanguageModel { + }; + const toolCallsByItemId = /* @__PURE__ */ new Map(); + const providerOptionsName = this.config.providerOptionsName; ++ const webSearchToolName = providerWebSearchToolName(options.tools); + return { + stream: response.pipeThrough( + new TransformStream({ +@@ -794,6 +816,31 @@ var OpenResponsesLanguageModel = class _OpenResponsesLanguageModel { + }); + hasToolCalls = true; + toolCallsByItemId.delete(chunk.item.id); ++ } else if (chunk.type === "response.output_item.added" && chunk.item.type === "web_search_call") { ++ controller.enqueue({ ++ type: "tool-input-start", ++ id: chunk.item.id, ++ toolName: webSearchToolName != null ? webSearchToolName : "web_search", ++ providerExecuted: true ++ }); ++ controller.enqueue({ ++ type: "tool-input-end", ++ id: chunk.item.id ++ }); ++ controller.enqueue({ ++ type: "tool-call", ++ toolCallId: chunk.item.id, ++ toolName: webSearchToolName != null ? webSearchToolName : "web_search", ++ input: JSON.stringify({}), ++ providerExecuted: true ++ }); ++ } else if (chunk.type === "response.output_item.done" && chunk.item.type === "web_search_call") { ++ controller.enqueue({ ++ type: "tool-result", ++ toolCallId: chunk.item.id, ++ toolName: webSearchToolName != null ? webSearchToolName : "web_search", ++ result: mapWebSearchOutput(chunk.item.action) ++ }); + } else if (chunk.type === "response.output_item.added" && chunk.item.type === "reasoning") { + controller.enqueue({ + type: "reasoning-start", +@@ -880,6 +927,36 @@ var OpenResponsesLanguageModel = class _OpenResponsesLanguageModel { + }; + } + }; ++function providerWebSearchToolName(tools) { ++ var _a; ++ return (_a = tools == null ? void 0 : tools.find( ++ (tool) => tool.type === "provider" && tool.id === "openai.web_search" ++ )) == null ? void 0 : _a.name; ++} ++function mapWebSearchOutput(action) { ++ if (action == null) return {}; ++ switch (action.type) { ++ case "search": ++ return { ++ action: { ++ type: "search", ++ ...action.query != null && { query: action.query }, ++ ...action.queries != null && { queries: action.queries } ++ }, ++ ...action.sources != null && { sources: action.sources } ++ }; ++ case "open_page": ++ return { action: { type: "openPage", url: action.url } }; ++ case "find_in_page": ++ return { ++ action: { ++ type: "findInPage", ++ url: action.url, ++ pattern: action.pattern ++ } ++ }; ++ } ++} + function createReasoningProviderMetadata({ + part, + providerOptionsName, +diff --git a/node_modules/@ai-sdk/open-responses/src/responses/open-responses-api.ts b/node_modules/@ai-sdk/open-responses/src/responses/open-responses-api.ts +index 8bc1e20..4c1ca42 100644 +--- a/node_modules/@ai-sdk/open-responses/src/responses/open-responses-api.ts ++++ b/node_modules/@ai-sdk/open-responses/src/responses/open-responses-api.ts +@@ -240,6 +240,13 @@ export type FunctionToolParam = { + type: 'function'; + }; + ++/** ++ * A provider-executed web search tool parameter. ++ */ ++export type WebSearchToolParam = { ++ type: 'web_search'; ++}; ++ + /** + * A specific function tool choice. + */ +@@ -476,6 +483,29 @@ export type ReasoningBody = { + encrypted_content?: string; + }; + ++/** ++ * A provider-executed web search action. ++ */ ++export type WebSearchAction = ++ | { ++ type: 'search'; ++ query?: string; ++ queries?: string[]; ++ sources?: Array<{ type: 'url'; url: string }>; ++ } ++ | { type: 'open_page'; url?: string | null } ++ | { type: 'find_in_page'; url?: string | null; pattern?: string | null }; ++ ++/** ++ * A provider-executed web search call. ++ */ ++export type WebSearchCall = { ++ type: 'web_search_call'; ++ id: string; ++ status: FunctionCallStatus; ++ action?: WebSearchAction; ++}; ++ + /** + * Output item field union type. + */ +@@ -483,7 +513,8 @@ export type OutputItem = + | FunctionCall + | FunctionCallOutput + | Message +- | ReasoningBody; ++ | ReasoningBody ++ | WebSearchCall; + + /** + * Details about why the response was incomplete. +@@ -643,7 +674,7 @@ export type OpenResponsesRequestBody = { + /** + * A list of tools that the model may call while generating the response. + */ +- tools?: FunctionToolParam[]; ++ tools?: Array; + + /** + * Controls which tool the model should use, if any. +diff --git a/node_modules/@ai-sdk/open-responses/src/responses/open-responses-language-model.ts b/node_modules/@ai-sdk/open-responses/src/responses/open-responses-language-model.ts +index 1146ec7..c35ed1e 100644 +--- a/node_modules/@ai-sdk/open-responses/src/responses/open-responses-language-model.ts ++++ b/node_modules/@ai-sdk/open-responses/src/responses/open-responses-language-model.ts +@@ -37,6 +37,7 @@ import { + type OpenResponsesChunk, + type ReasoningBody, + type ToolChoiceParam, ++ type WebSearchAction, + } from './open-responses-api'; + import { mapOpenResponsesFinishReason } from './map-open-responses-finish-reason'; + import type { OpenResponsesConfig } from './open-responses-config'; +@@ -120,17 +121,22 @@ export class OpenResponsesLanguageModel implements LanguageModelV4 { + + warnings.push(...inputWarnings); + +- // Convert function tools to the Open Responses format +- const functionTools: FunctionToolParam[] = []; ++ // Convert function and the Responses-native web search tool to the Open ++ // Responses format. Other provider-defined tools stay fail-closed. ++ const responseTools: Array = []; + + for (const tool of tools ?? []) { + if (tool.type === 'provider') { +- warnings.push({ +- type: 'unsupported', +- feature: `provider-defined tool ${tool.id}`, +- }); ++ if (tool.id === 'openai.web_search') { ++ responseTools.push({ type: 'web_search' }); ++ } else { ++ warnings.push({ ++ type: 'unsupported', ++ feature: `provider-defined tool ${tool.id}`, ++ }); ++ } + } else { +- functionTools.push({ ++ responseTools.push({ + type: 'function', + name: tool.name, + description: tool.description, +@@ -209,7 +215,7 @@ export class OpenResponsesLanguageModel implements LanguageModelV4 { + }), + } + : undefined, +- tools: functionTools.length ? functionTools : undefined, ++ tools: responseTools.length ? responseTools : undefined, + tool_choice: convertedToolChoice, + ...(textFormat != null && { text: { format: textFormat } }), + }, +@@ -273,6 +279,7 @@ export class OpenResponsesLanguageModel implements LanguageModelV4 { + + const content: Array = []; + let hasToolCalls = false; ++ const webSearchToolName = providerWebSearchToolName(options.tools); + + for (const part of response.output) { + switch (part.type) { +@@ -335,6 +342,23 @@ export class OpenResponsesLanguageModel implements LanguageModelV4 { + }); + break; + } ++ ++ case 'web_search_call': { ++ content.push({ ++ type: 'tool-call', ++ toolCallId: part.id, ++ toolName: webSearchToolName ?? 'web_search', ++ input: JSON.stringify({}), ++ providerExecuted: true, ++ }); ++ content.push({ ++ type: 'tool-result', ++ toolCallId: part.id, ++ toolName: webSearchToolName ?? 'web_search', ++ result: mapWebSearchOutput(part.action), ++ }); ++ break; ++ } + } + } + +@@ -454,6 +478,7 @@ export class OpenResponsesLanguageModel implements LanguageModelV4 { + { toolName?: string; toolCallId?: string; arguments?: string } + >(); + const providerOptionsName = this.config.providerOptionsName; ++ const webSearchToolName = providerWebSearchToolName(options.tools); + + return { + stream: response.pipeThrough( +@@ -540,6 +565,37 @@ export class OpenResponsesLanguageModel implements LanguageModelV4 { + hasToolCalls = true; + + toolCallsByItemId.delete(chunk.item.id); ++ } else if ( ++ chunk.type === 'response.output_item.added' && ++ chunk.item.type === 'web_search_call' ++ ) { ++ controller.enqueue({ ++ type: 'tool-input-start', ++ id: chunk.item.id, ++ toolName: webSearchToolName ?? 'web_search', ++ providerExecuted: true, ++ }); ++ controller.enqueue({ ++ type: 'tool-input-end', ++ id: chunk.item.id, ++ }); ++ controller.enqueue({ ++ type: 'tool-call', ++ toolCallId: chunk.item.id, ++ toolName: webSearchToolName ?? 'web_search', ++ input: JSON.stringify({}), ++ providerExecuted: true, ++ }); ++ } else if ( ++ chunk.type === 'response.output_item.done' && ++ chunk.item.type === 'web_search_call' ++ ) { ++ controller.enqueue({ ++ type: 'tool-result', ++ toolCallId: chunk.item.id, ++ toolName: webSearchToolName ?? 'web_search', ++ result: mapWebSearchOutput(chunk.item.action), ++ }); + } + + // Reasoning events (note: response.reasoning_text.delta is an LM Studio extension, not in official spec) +@@ -657,6 +713,39 @@ export class OpenResponsesLanguageModel implements LanguageModelV4 { + } + } + ++function providerWebSearchToolName( ++ tools: LanguageModelV4CallOptions['tools'], ++): string | undefined { ++ return tools?.find( ++ tool => tool.type === 'provider' && tool.id === 'openai.web_search', ++ )?.name; ++} ++ ++function mapWebSearchOutput(action: WebSearchAction | undefined) { ++ if (action == null) return {}; ++ switch (action.type) { ++ case 'search': ++ return { ++ action: { ++ type: 'search' as const, ++ ...(action.query != null && { query: action.query }), ++ ...(action.queries != null && { queries: action.queries }), ++ }, ++ ...(action.sources != null && { sources: action.sources }), ++ }; ++ case 'open_page': ++ return { action: { type: 'openPage' as const, url: action.url } }; ++ case 'find_in_page': ++ return { ++ action: { ++ type: 'findInPage' as const, ++ url: action.url, ++ pattern: action.pattern, ++ }, ++ }; ++ } ++} ++ + function createReasoningProviderMetadata({ + part, + providerOptionsName, diff --git a/patches/README.md b/patches/README.md index f7ef9200ad..dffb7420a4 100644 --- a/patches/README.md +++ b/patches/README.md @@ -45,6 +45,18 @@ Streaming tool-call association for gateways that reuse or omit `index` / `id` Delete when that guard passes against an unpatched package. +## `@ai-sdk/open-responses@2.0.29` + +The adapter drops every provider-defined tool and only decodes function-call +output items. Alibaba Token Plan exposes its provider-executed Harness web +search over the otherwise supported Responses wire, using +`tools: [{ type: "web_search" }]` and `web_search_call` output items. The patch +lowers that one published tool shape and maps its streamed call/result pair into +the AI SDK provider-executed tool protocol; all other provider-defined tools +remain unsupported. Product-level wire tests pin the request and result shape. + +Delete when `@ai-sdk/open-responses` ships equivalent provider-tool support. + ## `@astryxdesign/core@0.4.5` Six published component seams drop host-owned state or semantics: