isWrapped crashes on custom OpenAI-compatible clients when the braintrust SDK is loaded (TypeError: reading 'completions')

[Jamie-Ellis]

Summary

isWrapped in js/oai.ts throws TypeError: Cannot read properties of undefined (reading 'completions') for any custom OpenAI-compatible client that is not an instance of the OpenAI SDK class, whenever the braintrust SDK has been imported in the same process (it sets globalThis.__inherited_braintrust_wrap_openai at module load).

This crashes every LLM classifier (ClosedQA, Factuality, …) for frameworks that pass a plain-object bridge client — e.g. eve bridges autoevals graders to the Vercel AI SDK via a { chat: { completions: { create } } } object, which works perfectly until the Braintrust SDK is also loaded (e.g. by a Braintrust reporter in the same run), at which point every judge call dies.

Affected: 0.0.132 through 0.3.0 (current latest — same isWrapped implementation).

Repro (no braintrust install needed — the global is the trigger)

import { ClosedQA } from "autoevals";

// What `import "braintrust"` does as a side effect:
globalThis.__inherited_braintrust_wrap_openai = (c) => c;

// A minimal OpenAI-compatible bridge client (plain object):
const client = {
  chat: {
    completions: {
      create: async () => ({
        choices: [{ index: 0, finish_reason: "stop", message: { role: "assistant", content: "1" } }],
      }),
    },
  },
};

await ClosedQA({ input: "q", output: "a", criteria: "is helpful", model: "gpt-4o", client });
// TypeError: Cannot read properties of undefined (reading 'completions')

Remove the globalThis line and the same call works.

Mechanism

var isWrapped = (client, dangerouslyAllowBrowser) => {
  const Constructor = Object.getPrototypeOf(client).constructor; // `Object` for a plain object
  const clean = new Constructor({ apiKey: "dummy", dangerouslyAllowBrowser }); // → just { apiKey: "dummy", … }
  return String(client.chat.completions.create) !== String(clean.chat.completions.create);
  //                                                     ^ clean.chat is undefined → throws
};

buildOpenAIClient only consults isWrapped when globalThis.__inherited_braintrust_wrap_openai is set, which is why the crash appears/disappears with the presence of the Braintrust SDK — nasty to debug, since importing the observability SDK is what breaks the evals.

Suggested fix

The wrapper isWrapped guards is already defensive — braintrust's wrapOpenAI duck-types the client ("chat" in oai && … completions … create) and handles unknown shapes gracefully. isWrapped just needs the same courtesy, e.g.:

return String(client.chat?.completions?.create) !== String(clean?.chat?.completions?.create);

(or duck-type first and treat non-reconstructible clients as not-wrapped, letting wrapOpenAI decide). Happy to send a PR if useful.