Skip to content

test(e2e): expect embeddings API to reject empty string input - #723

Closed
christineschen wants to merge 1 commit into
mainfrom
devin/1785362221-embeddings-empty-input
Closed

test(e2e): expect embeddings API to reject empty string input#723
christineschen wants to merge 1 commit into
mainfrom
devin/1785362221-embeddings-empty-input

Conversation

@christineschen

Copy link
Copy Markdown
Contributor

Summary

validate has been red on every PR (mine, and Speakeasy's autogen branches) since openrouter-web#30855 — "fix(embeddings): validate non-empty input…" — tightened the embeddings request schema on Jul 27:

# packages/embedding-interfaces/schemas/request/index.ts
-  input: z.union([z.string(), z.array(z.string()), ...])
+  input: z.union([z.string().min(1), z.array(z.string().min(1)), ...])

Prod now answers input: '' with 400 too_small: expected string to have >=1 characters, so the e2e test that asserted empty input succeeds throws instead. This flips the test to assert the new (intended) contract:

-it('should handle empty string input gracefully', async () => {
-  const response = await client.embeddings.generate({ requestBody: { input: '', ... } });
-  expect(response.data).toBeDefined();
+it('should reject an empty string input', async () => {
+  await expect(client.embeddings.generate({ requestBody: { input: '', ... } }))
+    .rejects.toThrow(BadRequestResponseError);
 });

No SDK/source change — tests/e2e/embeddings.test.ts only. Verified against the live API: vitest run tests/e2e/embeddings.test.ts → 5/5 pass (was 4/5).

Link to Devin session: https://openrouter.devinenterprise.com/sessions/236b5466ec42476c8645101ef5bcd17f
Requested by: @christineschen

@christineschen christineschen self-assigned this Jul 29, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor
Original prompt from christine.chen

SYSTEM:
=== BEGIN THREAD HISTORY ===
<most_recent_message>
Christine Chen (U0B8QM7RKLL): create a pr to fix this

[Slack unfurl — this is an automatic link preview, not a user message]
Quote of conversation (https://openrouter.slack.com/archives/C0BCDN7RHJM/p1785278350731749?thread_ts=1785278350.731749&amp;cid=C0BCDN7RHJM):
> From David Bai
> <@U0B8QM7RKLL> non urgent nit but would it be a problem to update our client sdks to use initialism for BYOK, TTS, STT etc? At first I thought it was just a docs change but looks like need to update speakeasy configs and not sure if theres any implications there
> Posted on July 28, 2026 at 10:39 PM

</most_recent_message>
=== END THREAD HISTORY ===

Thread URL: https://openrouter.slack.com/archives/D0B9SPV4LV9/p1785338154566759?thread_ts=1785338154.566759&amp;cid=D0B9SPV4LV9

The latest message is the one right above that tagged you. The <most_recent_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review July 29, 2026 21:58

@perry-the-pr-reviewer perry-the-pr-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perry's Review

Verdict: 💬 Comments / questions
Risk: 🟢 Low

Details

Clean test-only fix for a red CI test. After openrouter-web#30855 tightened the embeddings request schema to reject empty input with a 400, the e2e test that asserted input: '' succeeds was failing. This PR flips it to assert the new contract instead.

Correctness:

  • The SDK's Embeddings.generate() calls unwrapAsync(), which throws the Result error on a non-OK status. The 400 response is mapped to BadRequestResponseError via M.jsonErr(400, ...), so .rejects.toThrow(BadRequestResponseError) correctly catches it.
  • Import path follows the repo's .js-extension ESM convention.

CI: The validate check passes. All 5 e2e tests pass (was 4/5).

Note: I would have approved this, but the maintainer app isn't authorized on OpenRouterTeam, so this is a COMMENT verdict. LGTM from a review standpoint — safe to merge with a human approval.

No findings.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Superseded by #716, which now carries this exact test change (cherry-picked); #716 is green.

christineschen added a commit that referenced this pull request Jul 30, 2026
…716)

## Summary

The OpenAPI tags are already `BYOK` / `TTS` / `STT`, but Speakeasy
re-cases single all-caps tokens, so generated symbols came out as
`Byok`, `Tts`, `Stt`. The fix is generator config, not a hand edit:
`customCasings` declares terms whose PascalCase form must stay all-caps.

```diff
 typescript:
   version: 1.2.0
+  customCasings:
+    byok:
+      initialism: true
+    stt:
+      initialism: true
+    tts:
+      initialism: true
```

Regenerated with the pinned `speakeasyVersion` (1.787.0), which renames
the sub-SDK classes (`class Byok` → `class BYOK`, `Tts` → `TTS`, `Stt` →
`STT`, modules stay `src/sdk/byok.ts` etc.) and the `ApiType` enum keys.

Unchanged: accessors (`openrouter.byok`, `.tts`, `.stt`), camelCase
properties (`isByok`, `includeByokInLimit`), and every wire value/JSON
name — only `pascal` casing feeds generated symbol names.

### Breaking for source consumers

`ApiType` is re-exported from `@openrouter/sdk/models`, and **two** of
its keys change (the values don't):

```diff
 export const ApiType = {
-  Tts: "tts",
-  Stt: "stt",
+  TTS: "tts",
+  STT: "stt",
 } as const;
```

So `ApiType.Tts` / `ApiType.Stt` and direct imports of the
`Byok`/`Tts`/`Stt` classes stop compiling; `ApiType.TTS` / `ApiType.STT`
replace them. Worth calling out in release notes.

### Also included

`tests/e2e/embeddings.test.ts` asserted that `input: ''` succeeds, but
openrouter-web#30855 tightened the embeddings schema to
`z.string().min(1)`, so the live API now returns 400 and the `validate`
job failed on every typescript-sdk PR (including Speakeasy's own regen
branches). The test now asserts the current contract:

```ts
it('should reject an empty string input', async () => {
  await expect(client.embeddings.generate({ requestBody: { input: '', model: 'openai/text-embedding-3-small' } }))
    .rejects.toThrow(BadRequestResponseError);
});
```

This supersedes #723.

Scope note: `Api` → `API` was deliberately left alone (`APIKeys` already
renders correctly since the tag is `API Keys`; renaming `ApiType` and
friends would be a much wider change). Matching python-sdk PR (merged):
OpenRouterTeam/python-sdk#561 · go-sdk (merged):
OpenRouterTeam/go-sdk#480

Version is hand-set to **1.2.0** in `gen.yaml` (Speakeasy honours a
custom version: `versioning: custom SDK version detected`), since the
generator would otherwise have shipped this source break as a patch.
Local `pnpm run typecheck`, `eslint`, and the unit suite pass.


BREAKING CHANGE: `ApiType.Tts` and `ApiType.Stt` are renamed to
`ApiType.TTS` and `ApiType.STT` (enum values `"tts"`/`"stt"` unchanged),
and the deep-importable sub-SDK classes `Byok`/`Tts`/`Stt` are renamed
to `BYOK`/`TTS`/`STT`. Runtime behaviour, JSON field names, and the
`openrouter.byok`/`.tts`/`.stt` accessors are unchanged.


Link to Devin session:
https://openrouter.devinenterprise.com/sessions/236b5466ec42476c8645101ef5bcd17f
Requested by: @christineschen

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant