test: harden the suite against brittle, flaky, and shallow tests

[cody-dot-js]

An adversarial audit of every test file in the repo, the fixes it produced, and the conventions that should
keep the problems from regrowing.

The method was to read each test file next to its implementation and ask two questions per test: what
one-line change to the implementation would make this fail, and what change that is not a bug would
make this fail. A second, skeptical pass then tried to refute each finding — it killed 48 of 219, which is
roughly the refutation rate you want from a review that isn't just agreeing with itself.

1,636 → 2,450 tests. All green, plus lint, fmt, typecheck, and the @ngrok/mantle build.

The honest headline

The suite was not especially flaky or brittle — it was thin, and thin in a way the test count hid.

Of 171 verified findings, only 1 was a genuine flake and 8 were brittleness. 131 were missing coverage. The
dominant problem: 40 of 84 component test files dispatched no user event at all, and that set included the
components whose entire value is interaction — Select was never opened, Tabs never clicked, SplitButton's menu
never opened, AlertDialog hardcoded open and never closed, DataTable's sort button never clicked, readOnly
had zero references anywhere in the repo. 1,636 green tests in 8 seconds was fast because most of the tests
were cheap.

The pure-logic layers are genuinely good and were left largely alone — the chart math, the code-block parsers,
and the ~3,000-case cx parity fixture are the house standard. The gap was everything between a pure function
and a rendered component.

Three real bugs, found by asking what no test covered

These are fixed here, each with a regression test that fails before the fix:

• Open redirect. isSafeLocalPath rejected //evil.com but accepted /\evil.com. The WHATWG URL parser
  treats \ as a synonym for / under special schemes, so a browser resolves that to https://evil.com/.
  The predicate's negative cases covered //, absolute URLs and control characters — never a backslash.
• Toast.Icon silently discarded a custom svg when intent="info". That one branch rendered InfoIcon
  unconditionally while all three sibling intents honored svg ?? …. The branches are now two module-scope
  maps keyed by ToastIntent, so the fallback can't be forgotten again without a type error.
• MediaObject dropped almost every prop. All three parts destructured five props with no rest spread, so
  id, onClick, aria-* and data-* were accepted by ComponentProps<"div"> and thrown away.

Plus a duplicate "016" in countryCodes (739 entries, 738 distinct).

An order-dependent test, reproduced and fixed

code-block's "fires onCopyError when clipboard write fails" patched navigator.clipboard.writeText before
userEvent.setup() — which replaces navigator.clipboard wholesale. The patch survived only when an earlier
test had already installed userEvent's stub. It failed on 2 of 6 shuffled seeds.

The general fix is config-level: restoreMocks, unstubEnvs, unstubGlobals on both projects. The suite had
~30 bare spy.mockRestore() calls at the end of test bodies, every one of which leaks when the test throws
before reaching it — turning one failure into a cascade. Now no inline teardown is load-bearing.

Verified with 14 shuffled seeds across both file and test order.

Tests that could not fail

A sample of what the audit found, all now fixed:

Where	Problem
use-breakpoint	six of eight tests asserted properties of literals declared in the test body
use-debounced-callback	neither test asserted a call count, so deleting the timer reset that is the debounce left both green
renderer	the equal-area test re-declared the implementation's shape constants and compared its own arithmetic
sidebar	the account-swatch hash test compared one id against itself
otp-input, data-table	assertions whose selectors could never match
accordion.browser	asserted offsetHeight === 0 for a reason unrelated to the line it existed to guard
field-context, mantle-style-sheets	byte-for-byte duplicate tests

Also tightened 34 call-count-blind spies to toHaveBeenCalledTimes + toHaveBeenLastCalledWith, and 29
assertions whose only claim was toBeDefined() / toBeTruthy() / not.toThrow().

Tailwind class assertions

32 files asserted utility strings as a proxy for behavior. Neither project loads a stylesheet, so those compare
a string against the source literal — they verify nothing and false-fail on token renames. Worse, they pass
while the behavior is broken: code-block asserted overflow-x-auto/shrink-0 to guard "TabList scrolls
instead of wrapping", yet adding flex-wrap — the exact regression a previous PR fixed — satisfies every
assertion.

All 256 were adjudicated individually rather than swept:

• 52 deleted, or replaced with a data attribute or a real behavioral assertion
• 121 kept — they pin a consumer's className beating a default, which is the tailwind-merge contract and
  is worth testing
• 83 kept with a comment naming what consumes the class (mostly Button's size/appearance variants, where
  the class genuinely is the only observable)

field.test.tsx alone went 52 → 14. The raw repo-wide count barely moved because the suite grew 50%; what
changed is that every remaining one is now either a real contract or annotated with why it exists.

Test placement

code-block-fold and mantle-style-sheets move out of the Playwright project — neither used a real-browser
API, so they were paying Chromium startup and browser-launch flake for assertions that are deterministic in
happy-dom.

New browser tests were added only where one is genuinely required, each injecting its load-bearing CSS inline
(browser tests load no Tailwind): canvas paint geometry, Element.moveBefore, real overflow observers.

New coverage for shared primitives

Untested component directories: 20 → 13. Deliberately not all 20 — the ones written are the primitives
other, already-tested components depend on:

dialog (including the primitive.tsx guards Dialog, Sheet and AlertDialog share), table's
ResizeObserver/MutationObserver overflow engine, dropdown-menu, separator, sandboxed-on-click,
media-object and flag. The remaining 13 are leaf presentation (badge, card, kbd, skeleton, …) and
warrant a few assertions each, tracked for later.

Conventions

CONVENTIONS.md § Testing, COMPONENT_SPEC.md §8, and the AGENTS.md diff-audit checklist now state the
rules this audit showed were missing:

• A test must be able to fail — name the one-line change it catches. Bans toBeDefined() /
  not.toThrow() as a test's only assertion, tautologies, and unmatchable queries.
• Drive the interaction — a component with an event handler, controlled prop, or keyboard contract needs a
  real event, not an assertion about its initial markup.
• Assert behavior, not styling internals — with the two legitimate carve-outs spelled out.
• Determinism — no arbitrary sleeps, spies after userEvent.setup(), no order dependence.
• Pin cross-file contracts — when a selector in one file depends on an attribute emitted in another,
  assert both sides in one test.
• Cover the server render — renderToString for SSR branches; evaluate stringified inline scripts.

oxlint's vitest(expect-expect) already catches the assertion-free case; the rest is reviewer-enforced.

Not done, on purpose

• 61 findings need an implementation change to become testable — nearly all "emit a documented data
  attribute so this variant is observable" (e.g. data-appearance on TextArea, data-size on Flag,
  data-slot on the Button spinner). Left for a follow-up rather than changing public API inside a test PR.
• 17 newly found source bugs are recorded but unfixed, including: DataTable.EmptyRow computes colSpan
  from all columns instead of visible leaf columns; goToLastPage doesn't clamp to 1, so an empty list reports
  a negative offset; the date comparators return NaN for an Invalid Date, which Array#sort reads as
  "equal"; MultiSelect needs two ArrowDown presses from a focused tag; DataTable.Root clobbers
  data-slot="table" instead of joining it. Happy to file these as issues.
• No mutation testing or coverage tool ran. Agents hand-verified ~40 mutants; the true count of surviving
  mutants is higher. Line coverage was deliberately skipped — it's the wrong instrument here, since a
  render() with zero assertions scores 100%.
• One unreproduced observation: a shuffled full-suite run once reported an unhandled
  ReferenceError: window is not defined from an input-otp internal timer firing after teardown. Exit code
  0, no test failed, and it did not reproduce in 11 further attempts. Flagged rather than speculatively
  patched.

Verification

lint          0 errors
fmt:check     clean (727 files)
typecheck     7/7 tasks, 0 errors
build         @ngrok/mantle builds clean
mantle        144 files, 2450 tests passed
apps/www       18 files,  170 tests passed
shuffle       14 seeds green across file + test order
browser        4 consecutive full runs green

[changeset-bot]

🦋 Changeset detected

Latest commit: dd78af5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@ngrok/mantle	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[ngrok-ship]

🚀 Deploy Previews

Updated 2026-08-05 17:26 UTC

App	URL	Commit	Status
ngrok-mantle	https://mantle-1367.ngrok-previews.ngrok.app	dd78af5	🟢 Running