Skip to content

Sync with upstream appium/WebDriverAgent master (v16.8.0) - #28

Open
Timo972 wants to merge 28 commits into
masterfrom
timo/sync-appium-master-16.8
Open

Sync with upstream appium/WebDriverAgent master (v16.8.0)#28
Timo972 wants to merge 28 commits into
masterfrom
timo/sync-appium-master-16.8

Conversation

@Timo972

@Timo972 Timo972 commented Aug 25, 2026

Copy link
Copy Markdown

Merges appium/WebDriverAgent master (v16.4.0 → v16.8.0) into our fork.

What upstream changed

Conflict resolutions

  • standalone vs onControlQueue: both sides independently invented queue-bypassing routes. Adopted upstream's standalone API and migrated our call sites (/status, screencapture status/stop/keyframe routes, unknown-command catch-alls). Coalescing is safe for all of them (idempotent reads/stops).
  • Automation funnel preserved: our serial funnel is now the server's routeQueue; non-standalone handlers hop to main via dispatch_sync from there. Concurrent automation requests still cannot execute reentrantly inside a handler that spins the main run loop. FBWebServerDispatchTests ported to FBHTTPServer — all 4 pass.
  • CocoaAsyncSocket restored: our audio/video streaming stack and the broadcast appex use GCDAsyncSocket, so the vendor lib and its project wiring are kept (fork-maintained from now on). The CocoaHTTPServer/RoutingHTTPServer hardening patches we carried are dropped — obsolete with the new server.
  • FBSession: took upstream's new teardown-condition machinery, kept our @synchronized guards on _activeSession reads/writes (standalone routes read it off-main; upstream assigns it unlocked).
  • FBConfiguration: kept mobilerun no-quiescence defaults (waitForIdleTimeout=0, animationCoolOffTimeout=0), added upstream's accessibilityDeadline=0.
  • FBXCodeCompatibility: dropped our bounded testmanagerd version-exchange wait in favor of upstream's semaphore-based fix for the same hang.
  • CI: kept wda-package.yml deleted (hand-published releases); took upstream's env bumps in wda-tests.yml (watch vars unused by our matrix).

Verification

  • WebDriverAgentLib, WebDriverAgentRunner (with broadcast appex), and WebDriverAgentLib_watchOS build clean for simulator.
  • Full UnitTests suite passes, including the ported FBWebServerDispatchTests (standalone-responsiveness + no-reentrant-nesting) and FBRouteTests.
  • pbxproj validated: plutil parse OK, zero dangling refs, zero orphan build files.
  • tvOS lib not buildable locally (tvOS simulator platform not installed) — covered by CI.

🤖 Generated with Claude Code

mykola-mokhnach and others added 28 commits August 19, 2026 22:04
## [16.5.0](appium/WebDriverAgent@v16.4.0...v16.5.0) (2026-08-20)

### Features

* Add watchOS support to the TS driver, functional tests, and release pipeline ([appium#1217](appium#1217)) ([b53bb8f](appium@b53bb8f))
## [16.5.1](appium/WebDriverAgent@v16.5.0...v16.5.1) (2026-08-20)

### Bug Fixes

* add watchOS assets to GitHub release artifacts ([appium#1219](appium#1219)) ([ce5a9e8](appium@ce5a9e8))
## [16.6.0](appium/WebDriverAgent@v16.5.1...v16.6.0) (2026-08-22)

### Features

* Add MJPEG screenshot streaming support to watchOS ([appium#1220](appium#1220)) ([c6dcf03](appium@c6dcf03))
## [16.7.0](appium/WebDriverAgent@v16.6.0...v16.7.0) (2026-08-22)

### Features

* unify HTTP server across iOS/tvOS/watchOS on Network.framework ([appium#1221](appium#1221)) ([cd741c5](appium@cd741c5))
## [16.7.1](appium/WebDriverAgent@v16.7.0...v16.7.1) (2026-08-23)

### Bug Fixes

* return W3C-compliant JSON error for unmatched routes ([appium#1223](appium#1223)) ([c951a91](appium@c951a91))
## [16.7.2](appium/WebDriverAgent@v16.7.1...v16.7.2) (2026-08-24)

### Bug Fixes

* harden FBHTTPServer/FBTCPSocket against races and protocol gaps ([appium#1224](appium#1224)) ([cf4bb2b](appium@cf4bb2b))
## [16.7.3](appium/WebDriverAgent@v16.7.2...v16.7.3) (2026-08-24)

### Bug Fixes

* let /status, /screenshot, and DELETE /session API methods to bypass the dispatch queue ([appium#1222](appium#1222)) ([f99b011](appium@f99b011))
## [16.8.0](appium/WebDriverAgent@v16.7.3...v16.8.0) (2026-08-24)

### Features

* bound accessibility snapshot requests to avoid indefinite hangs ([appium#1214](appium#1214)) ([cd829eb](appium@cd829eb))
Upstream replaced the vendored CocoaHTTPServer/RoutingHTTPServer stack
with FBHTTPServer on Network.framework (appium#1221) and added standalone
routes that bypass the dispatch queue (appium#1222). Resolutions:

- Adopt upstream's standalone/isStandalone route API and migrate the
  fork's equivalent onControlQueue call sites (status, screencapture,
  unknown-command catch-alls) onto it.
- Keep the fork's automation funnel by using it as FBHTTPServer's
  routeQueue and hopping to the main queue via dispatch_sync inside the
  handler, so concurrent automation requests still cannot nest inside a
  spinning run loop. FBWebServerDispatchTests ported to FBHTTPServer and
  passing.
- Drop the fork's CocoaHTTPServer/RoutingHTTPServer hardening patches
  (obsolete; upstream hardened FBHTTPServer itself in 16.7.2).
- Restore the CocoaAsyncSocket vendor library and its project wiring:
  droidrun's audio/video streaming and the broadcast appex still use
  GCDAsyncSocket. Fork-maintained from now on.
- Keep FBSession's @synchronized reads/writes of _activeSession on top
  of upstream's new teardown-condition machinery.
- Keep mobilerun no-quiescence defaults (waitForIdleTimeout=0,
  animationCoolOffTimeout=0) and add upstream's accessibilityDeadline.
- Drop the fork's bounded testmanagerd version-exchange wait in favor of
  upstream's semaphore-based fix for the same hang.
- Keep wda-package.yml deleted (fork releases are hand-published).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Terminate the tested app inline when -kill already runs on the main
  queue (every POST /session replacement does): the bounded dispatch to
  main would block main on its own semaphore, stall five seconds, and
  then give up without terminating the app.
- Publish _isTeardownInProgress in the same critical section that
  clears _activeSession, so a concurrent
  +killActiveSessionAndWaitForTeardown can no longer observe a nil
  session with no teardown to wait for and launch a replacement whose
  app the still-running teardown then kills.
- Raise the FBWebServerDispatchTests wait deadlines (5s -> 15s): the
  first run after a fresh test-runner install can exceed 5s and flaked
  once locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Parse Content-Length strictly instead of via -integerValue, which
  silently maps garbage ("bogus" -> 0, "12abc" -> 12) to a wrong body
  length and desyncs the connection's request framing, letting body
  bytes be re-parsed as smuggled pipelined requests. An unparseable
  value now gets a 400 and the connection is closed.
- Cap the buffered size of an incomplete header block (64 KiB): a
  client that never sends the terminating CRLFCRLF could previously
  grow the per-connection buffer without bound.

Both defenses existed in the vendored CocoaHTTPServer that upstream's
FBHTTPServer rewrite replaced (strict parseString:intoUInt64: and
MAX_HEADER_LINE_LENGTH/MAX_HEADER_LINES) and were lost in the rewrite.
Covered by the new FBHTTPServerFramingTests raw-socket unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 64 KiB cap only fired while the terminating CRLFCRLF was still
missing; nw_connection_receive delivers up to UINT32_MAX bytes per
callback, so a single large receive containing the terminator skipped
the check and the oversized block was copied and parsed anyway.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Reap connections that never deliver a complete request: a periodic
  sweep closes connections whose current request (first byte through
  declared body end) has not completed within 30 seconds, including
  peers that connect and send nothing. Idle keep-alive connections and
  requests already executing are exempt. The old CocoaHTTPServer stack
  bounded this with 30-second header read timeouts.
- Bound MJPEG frame writes per client: nw_connection_send buffers
  without backpressure, so a viewer that stopped reading retained every
  generated frame until WDA ran out of memory. Frames for a client with
  MAX_PENDING_FRAMES_PER_CLIENT sends still outstanding are dropped
  instead of queued (the old GCDAsyncSocket path disconnected slow
  clients via a 1-second write timeout).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…m sync

Only FBRouteTests.m conflicted: master's timing tweaks to the spinning
probe test (0.3 s gap, 1.0 s spin, expanded comment) are kept, with the
comment's dispatch wording updated to the funnel-as-routeQueue
architecture this branch introduces. FBSessionCommands' new
cachedDeviceInfo pre-warm and the video-stream session hardening merged
cleanly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… body phase

A valid declared body can legitimately take longer than 30 seconds to
arrive (e.g. a large base64 payload over a slow USB tunnel); the reaper
was closing such healthy uploads because the clock started at request
arrival and was never refreshed. Body-phase progress now refreshes the
deadline - the buffered size stays bounded by the already-validated
Content-Length - while the header phase keeps the hard, non-refreshing
deadline (drip-feeding there is additionally bounded by the 64 KiB cap).
The replaced CocoaHTTPServer stack behaved the same way: 30-second
header timeouts, unbounded body reads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…back caching

- Unblock the next pipelined request only from the response send's
  completion (nw_connection_send is FIFO per connection, so ordering is
  unchanged): a client that pipelined requests without reading responses
  could previously accumulate unbounded fully-rendered response buffers
  inside Network.framework. Also cap everything a connection may have
  buffered-but-unconsumed (body limit + 2x header cap) - the per-request
  checks don't run while a request is executing, so a client could pump
  data unboundedly for as long as its previous request took.
- Reject whitespace between a header field name and its colon with 400,
  as RFC 7230 (3.2.4) requires: "Content-Length : 5" was stored under a
  "content-length " key, dispatching the request with a zero-length body
  and desyncing the connection's framing.
- Cache the testmanagerd protocol version timeout fallback: retrying
  meant every /status against a degraded legacy daemon stalled for the
  full 20 seconds, making a bound WDA look permanently unavailable to
  health checks. The value is diagnostic-only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… pipelining

- Reject duplicate Content-Length/Transfer-Encoding headers with a 400
  (RFC 7230 3.3.3) instead of collapsing them last-wins, and reject
  Transfer-Encoding on presence rather than on a non-empty value: a
  "chunked" header followed by an empty one used to look absent, so a
  chunked body was parsed as empty and its bytes re-read as smuggled
  requests.
- Reject requests for a session that was already abandoned. The kill
  notification only reaches requests tracked at that moment, so one
  parsed afterwards queued on a possibly-wedged route queue with no
  abandonment ever coming; the abandoning response is now recorded
  (capped, keyed by the session's UUID) and returned immediately.
  Recorded under the same lock the abandonment takes, so nothing can
  slip in between.
- Propagate nw_connection_send errors to the write completion. A failed
  response send was treated as success, unblocking the next pipelined
  request - so a mutating command could run for a connection that could
  no longer be answered. The connection and its buffered requests are
  now dropped instead.
- Add accessibilityDeadline to the exported WDASettings/WDACapabilities
  TypeScript interfaces; the v16.8.0 setting was otherwise unusable from
  TS without a cast.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants