Skip to content

Make test synchronization deterministic - #1346

Merged
quinnj merged 2 commits into
masterfrom
codex/deterministic-test-synchronization
Aug 7, 2026
Merged

Make test synchronization deterministic#1346
quinnj merged 2 commits into
masterfrom
codex/deterministic-test-synchronization

Conversation

@quinnj

@quinnj quinnj commented Aug 6, 2026

Copy link
Copy Markdown
Member

Summary

  • replace test-owned sleeps, polling deadlines, and elapsed-time assertions with explicit lifecycle signals
  • use exact protocol reads and injected clock and wait functions for timeout behavior
  • add contributor guidance and an enforcement test for deterministic synchronization
  • remove the timing watchdog and Windows timing warmup while retaining the workflow job timeout as the final deadlock guard

Root cause

GitHub Actions can pause Julia tasks for unpredictable periods. The test suite used short sleeps, polling deadlines, and elapsed-time limits as proxies for protocol state. Runner scheduling delays could therefore fail correct HTTP behavior. Some timing guards could also hide a real deadlock instead of identifying the missing lifecycle signal.

The first CI run also exposed a coverage regression. Some rewritten deadline tests used already-expired deadlines and no longer executed active wait and wake paths. The follow-up adds deterministic lifecycle barriers and private clock and wait seams. It restores these paths without wall-clock coordination.

Validation

  • full Julia 1.12.6 package test with coverage, including all 63 trim checks
  • Julia 1.10.11 policy, HTTP core, HTTP/1 transport and server, HTTP/2 client and server, and WebSocket suites
  • local source coverage increased from 92.3576% at the base commit to 92.3865% at this head
  • git diff --check
  • exact source scan for prohibited test-owned clock waits

Co-authored by Codex

@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.52%. Comparing base (4b6c835) to head (eca220c).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1346   +/-   ##
=======================================
  Coverage   88.51%   88.52%           
=======================================
  Files          31       31           
  Lines       11955    11961    +6     
=======================================
+ Hits        10582    10588    +6     
  Misses       1373     1373           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@quinnj quinnj closed this Aug 7, 2026
@quinnj quinnj reopened this Aug 7, 2026
@quinnj
quinnj marked this pull request as ready for review August 7, 2026 05:23
quinnj added 2 commits August 7, 2026 14:26
Replace sleeps, polling deadlines, and elapsed-time assertions with lifecycle signals, exact protocol reads, and expired deadlines.

Add a policy test and contributor guidance to keep the suite deterministic across variable CI runner load.
Add lifecycle barriers and private clock and wait seams so deterministic tests still execute active wait, wake, handoff, and shutdown paths.

Restore project coverage without reintroducing elapsed-time coordination.
@quinnj
quinnj force-pushed the codex/deterministic-test-synchronization branch from 57ed420 to eca220c Compare August 7, 2026 20:35
@quinnj
quinnj merged commit bf44f8c into master Aug 7, 2026
8 checks passed
@quinnj
quinnj deleted the codex/deterministic-test-synchronization branch August 7, 2026 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant