Skip to content

wpt: raise testharness timeouts to deflake heavy interpolation tests - #815

Merged
nicoburns merged 1 commit into
mainfrom
devin/1788109890-wpt-harness-timeout
Aug 30, 2026
Merged

wpt: raise testharness timeouts to deflake heavy interpolation tests#815
nicoburns merged 1 commit into
mainfrom
devin/1788109890-wpt-harness-timeout

Conversation

@nicoburns

@nicoburns nicoburns commented Aug 30, 2026

Copy link
Copy Markdown
Member

Summary

WPT runs have been producing flaky Timeout ⇄ Fail flips (e.g. ~35 spurious status changes on #814), almost all in the heavy interpolation/parsing testharness suites (css/*/animations/*-interpolation.html, color-mix parsing, test_font_family_parsing.html, has-complexity.html).

Root cause: the injected testharnessreport.js sets timeout_multiplier: 0.1, scaling testharness's internal 10s timeout down to 1s of wall-clock time — and these tests take ~0.6–1.2s of real script time even on an idle machine. Since timers are pumped in real time, the internal timeout races the test's own execution; on loaded CI machines the same tests fall on either side of 1s from run to run. They are not hangs: given budget, every one of them completes (as Fail, mostly — Blitz lacks animation support).

Reproduced locally: with 8 busy-loop processes saturating the cores, the same test flips FAILED/TIMED OUT across runs at multiplier 0.1, and is deterministically FAILED (3/3 runs × 5 tests) with this change.

Change:

  • timeout_multiplier: 0.10.5 (internal timeout 1s → 5s)
  • HARNESS_TIMEOUT backstop 5s → 10s (so the scaled internal timeout still fires first and hangs get a proper harness TIMEOUT status)

Cost is negligible: genuine hangers are already covered by the 259-entry timeout quarantine; the latest full CI run had only 17 TIMEOUT results, nearly all of which are these borderline tests that will now complete instead of waiting out the timeout.

Link to Devin session: https://dioxus.staging.devinenterprise.com/sessions/dbb484350ccd420bb69efaf2f0f462ce
Open in Devin Desktop: https://dioxus.staging.devinenterprise.com/desktop/session/dbb484350ccd420bb69efaf2f0f462ce?variant=devin-insiders
Requested by: @nicoburns

WPT results

4 newly passing, 0 newly failing (net +4), 50 other status changes.

Full diff (54 changed tests)
! Timeout => Fail css/css-anchor-position/anchor-parse-valid.html
! Timeout => Fail css/css-anchor-position/anchor-position-writing-modes-001.html
+ Timeout => Pass css/css-anchor-position/position-area-parsing.html
! Timeout => Fail css/css-anchor-position/property-interpolations.html
! Timeout => Fail css/css-backgrounds/animations/background-position-origin-interpolation.html
! Timeout => Fail css/css-backgrounds/animations/background-size-interpolation.html
! Timeout => Fail css/css-backgrounds/animations/border-image-slice-interpolation.html
! Timeout => Fail css/css-backgrounds/animations/border-image-source-interpolation.html
! Timeout => Fail css/css-backgrounds/animations/border-image-width-interpolation.html
! Timeout => Fail css/css-borders/corner-shape/corner-shape-interpolation.html
! Timeout => Fail css/css-color/parsing/color-computed-color-mix-function.html
! Timeout => Fail css/css-color/parsing/color-computed-relative-color.html
! Timeout => Fail css/css-color/parsing/color-valid-color-mix-function.html
+ Timeout => Pass css/css-color/parsing/color-valid-relative-color.html
! Timeout => Fail css/css-fonts/animations/font-size-adjust-composition.html
! Timeout => Fail css/css-fonts/animations/font-size-adjust-interpolation.html
! Timeout => Fail css/css-fonts/test_font_family_parsing.html
! Timeout => Fail css/css-gaps/animation/column-rule-inset-interpolation.html
! Timeout => Fail css/css-gaps/animation/row-rule-inset-interpolation.html
! Timeout => Fail css/css-grid/animation/grid-no-interpolation.html
+ Timeout => Pass css/css-images/parsing/gradient-interpolation-method-computed.html
+ Timeout => Pass css/css-images/parsing/gradient-interpolation-method-valid.html
! Timeout => Fail css/css-lists/css-lists-no-interpolation.html
! Timeout => Fail css/css-masking/animations/clip-interpolation.html
! Timeout => Fail css/css-masking/animations/clip-path-interpolation-shape-control-points.html
! Timeout => Fail css/css-masking/animations/clip-path-interpolation-xywh-rect.html
! Timeout => Fail css/css-masking/animations/mask-border-slice-interpolation.html
! Timeout => Fail css/css-masking/animations/mask-border-width-interpolation.html
! Timeout => Fail css/css-masking/animations/mask-image-interpolation.html
! Timeout => Fail css/css-shapes/animation/shape-outside-interpolation.html
! Timeout => Fail css/css-shapes/animation/shape-outside-path-interpolation.html
! Timeout => Fail css/css-shapes/animation/shape-outside-shape-interpolation.html
! Timeout => Fail css/css-shapes/shape-outside/values/shape-outside-inset-001.html
! Timeout => Fail css/css-sizing/animation/height-interpolation.html
! Timeout => Fail css/css-sizing/animation/max-height-interpolation.html
! Timeout => Fail css/css-sizing/animation/max-width-interpolation.html
! Timeout => Fail css/css-sizing/animation/min-height-interpolation.html
! Timeout => Fail css/css-sizing/animation/min-width-interpolation.html
! Timeout => Fail css/css-sizing/animation/width-interpolation.html
! Timeout => Fail css/css-transforms/animation/rotate-interpolation.html
! Timeout => Fail css/css-transforms/animation/scale-interpolation.html
! Timeout => Fail css/css-transforms/animation/transform-interpolation-001.html
! Timeout => Fail css/css-transforms/animation/transform-interpolation-004.html
! Timeout => Fail css/css-transforms/animation/transform-interpolation-005.html
! Timeout => Fail css/css-transforms/animation/translate-interpolation.html
! Timeout => Fail css/css-values/calc-size/animation/calc-size-width-interpolation.html
! Timeout => Fail css/css-values/calc-size/animation/interpolate-size-max-width-interpolation.html
! Timeout => Fail css/filter-effects/animation/backdrop-filter-composition-001.html
! Timeout => Fail css/filter-effects/animation/filter-interpolation-003.html
! Timeout => Fail css/motion/animation/offset-path-interpolation-001.html
! Timeout => Fail css/motion/animation/offset-path-interpolation-005.html
! Timeout => Fail css/motion/animation/offset-path-interpolation-006.html
! Timeout => Fail css/motion/animation/offset-rotate-interpolation.html
! Timeout => Fail css/selectors/invalidation/has-complexity.html

Generated by the WPT workflow.

@staging-devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@nicoburns
nicoburns merged commit c6dec88 into main Aug 30, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant