SEA: connector should request INLINE_OR_EXTERNAL_LINKS, not EXTERNAL_LINKS (small results forced through CloudFetch)

[eric-wang-1990]

Summary

On the SEA backend, when CloudFetch is enabled (the default, use_cloud_fetch=True), the connector requests disposition=EXTERNAL_LINKS — forcing every result, including tiny ones, through a presigned-URL + cloud-storage download. It should request INLINE_OR_EXTERNAL_LINKS (the HYBRID disposition), so the server can return small results inline and only use external links when the result is large.

Location

src/databricks/sql/backend/sea/backend.py, in the ExecuteStatement request build:

disposition = (
    (
        ResultDisposition.HYBRID            # "INLINE_OR_EXTERNAL_LINKS"
        if self.use_hybrid_disposition
        else ResultDisposition.EXTERNAL_LINKS   # "EXTERNAL_LINKS"  <-- default
    )
    if use_cloud_fetch
    else ResultDisposition.INLINE
).value

Defaults:

• use_cloud_fetch defaults to True (client.py, backend.py)
• use_hybrid_disposition defaults to False (backend.py)

So out of the box, SEA sends EXTERNAL_LINKS, never INLINE_OR_EXTERNAL_LINKS, unless the caller explicitly opts into use_hybrid_disposition=True.

ResultDisposition.HYBRID maps to the wire value "INLINE_OR_EXTERNAL_LINKS" (see sea/utils/constants.py).

Impact

Every SEA query — including a 1-row SELECT 1 — performs a full CloudFetch round-trip (presigned URL fetch + blob download) instead of receiving the small result inline. That is:

• A latency/robustness regression for the common small-result case: an extra network hop to cloud storage, plus exposure to presigned-URL expiry / storage flakiness, with no benefit for small results.
• A divergence from Thrift, which delivers small results inline (direct results) by default.

Observed directly (proxy capture, native SEA, connector 4.3.0):

ExecuteStatement: stmt='SELECT 1 AS value'
  REQUEST  disposition=EXTERNAL_LINKS  format=ARROW_STREAM  wait_timeout=10s
  RESPONSE result.external_links present (total_chunk_count=1)  → 1 cloud download

A one-row literal produced a real external-link download.

Suggested fix

When use_cloud_fetch is enabled, request INLINE_OR_EXTERNAL_LINKS (HYBRID) rather than EXTERNAL_LINKS, so the server chooses inline for small results and external links for large ones. In other words, make HYBRID the default disposition (or, equivalently, always send INLINE_OR_EXTERNAL_LINKS when CloudFetch is on), rather than gating it behind an opt-in use_hybrid_disposition flag. EXTERNAL_LINKS-always should not be the default.

Note

The same captured request also shows wait_timeout=10s (a positive value with direct results), which is the separate ES-2034600 / #860-adjacent concern.