test(storage): cap macOS isolate worker churn to avoid Bun segfault - #849
Open
Wibias wants to merge 28 commits into
Open
test(storage): cap macOS isolate worker churn to avoid Bun segfault#849Wibias wants to merge 28 commits into
Wibias wants to merge 28 commits into
Conversation
Two spawn/reset cycles still tripped Bun 1.3.14 on macOS Silicon CI after a green suite with balanced worker counts (exit 133). Keep one-cycle proof on darwin, drain between cycles, and settle briefly before the isolate file boundary.
github-actions
Bot
added
the
chore
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughWorker termination now handles Windows and macOS OS-join delays. Storage worker tests add draining, platform-specific churn limits, macOS skips, and regression coverage. Timer handling, filesystem synchronization, OAuth queue tests, ACL tests, and slow-environment timeouts also change. ChangesWorker teardown and platform test behavior
Process lifetime and storage robustness
Test environment timing and authentication setup
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Comment |
Churn caps and afterAll settle still left macos-latest red with exit 133 after storage-worker-teardown passed (balanced worker counts). Run the same suite without --isolate on macOS; keep isolate on Linux/Windows.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/ci-workflows.test.ts`:
- Around line 45-49: Update the workflow assertions in the test around the
workflow string so each platform condition is matched together with its
corresponding run command within the same step block. Verify the macOS step
contains runner.os == 'macOS' and bun test tests, while the non-macOS step
contains runner.os != 'macOS' and bun test --isolate tests; replace the
independent contains/regex checks rather than adding separate assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 03bc93fc-e97a-4105-953e-8b666404657a
📒 Files selected for processing (3)
.github/workflows/ci.ymltests/ci-workflows.test.tstests/storage-worker-teardown-isolate.test.ts
Wibias
marked this pull request as draft
August 1, 2026 10:23
afterAll settle never ran when Bun 1.3.14 segfaulted mid-file on macOS CI. Apply the same post-close OS-join gap as Windows, and drain+settle in afterEach.
3 tasks
Wibias
marked this pull request as ready for review
August 1, 2026 10:37
Independent contains/regex checks could pass a malformed workflow that splits the OS guard from the run command.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be8110ea1e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Windows GHA hit the default 5s budget at ~5.4s on bun --eval restore; give the file's spawnSync cases 15s so load variance is not a fail.
Non-isolate macOS CI hung past the 20-minute job ceiling. Restore isolate for all platforms, skip Worker-spawning teardown cases on darwin (Bun 1.3.14 balanced-count segfault), and keep the platform-cap meta-test.
A close that wins near timeoutMs could still throw after the 250ms settle because the timer stayed armed across the sleep. Clear it first; add a regression for the close-wins case.
Credential install in beforeEach can exceed the default 5s hook budget under isolate on a loaded windows-latest runner (NTFS ACL harden stalls).
Retrigger Cross-platform CI after the suite-budget bump landed without a pull_request workflow run on the fork tip.
Keep darwin OS-join settle regression plus zero-leak reservation coverage from tip.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/storage-worker-teardown-isolate.test.ts (1)
140-151: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMake the platform-cap assertion independent.
The supplied context omits Lines [140-151]. If the test computes its expected value with the same platform conditional as
workerChurnCyclesForIsolate, it can pass while the helper is wrong.Assert the explicit 8, 1, and 2 cases through an injectable platform input or a table-driven helper.
This request is based on the selected context omitting the new assertion at Lines [140-151].
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/storage-worker-teardown-isolate.test.ts` around lines 140 - 151, The isolate worker churn test currently derives expected values from the same platform conditional as workerChurnCyclesForIsolate, so it cannot detect helper errors. Refactor the assertion around workerChurnCyclesForIsolate to accept an injectable platform or use a table-driven helper, then explicitly verify win32 returns 8, darwin returns 1, and other platforms return 2.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/storage-worker-lifecycle.test.ts`:
- Around line 127-129: In the test setup around registerStorageWorker, assert
before calling terminateStorageWorker that the worker was registered and that a
close listener was attached. Keep these precondition checks ahead of termination
so the test cannot pass through the unregistered-worker fast path and continues
covering the timeout and OS-join settlement behavior.
---
Outside diff comments:
In `@tests/storage-worker-teardown-isolate.test.ts`:
- Around line 140-151: The isolate worker churn test currently derives expected
values from the same platform conditional as workerChurnCyclesForIsolate, so it
cannot detect helper errors. Refactor the assertion around
workerChurnCyclesForIsolate to accept an injectable platform or use a
table-driven helper, then explicitly verify win32 returns 8, darwin returns 1,
and other platforms return 2.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9db173d2-148e-41e3-982d-305aa10a53c3
📒 Files selected for processing (5)
src/storage/worker-lifecycle.tstests/codex-catalog-restore.test.tstests/storage-worker-lifecycle.test.tstests/storage-worker-teardown-isolate.test.tstests/subagent-model-fallback.test.ts
…gate The 128-slot busy-reject case timed out at 5s on Windows isolate and left the first gate closed, hanging the file until the 20-minute job ceiling.
Without live-count and listener preconditions, an unregistered fast path could pass the case without exercising the OS-join settle timer race.
tests-only pushes after the merge stopped scheduling pull_request CI; touch the workflow path filter so the oauth admission timeout harden is verified.
Combine tip's try/finally drain with the shared Windows store case budget.
unref'd queue wait timeouts stalled oauth-store-multi after the 128-slot admission case on windows-latest (no further output until the 20m job kill).
Directory fsync throws on Windows before the test recorder ran, so ordering assertions missed the sync step even though open and the attempt succeeded.
Spawn cases are skipIf(darwin), so afterEach/afterAll Bun.sleep(250) added latency without covering worker churn. OS-join settle stays in worker-lifecycle.
Windows CI hung in shutdown-drain after the oauth wait-timer fix: unref'd grace waiters never fire under bun test --isolate, so drainAndShutdown stalls until the 20m job ceiling.
Preserve ref'd serializeMutation wait timers and STORE_BUDGET_MS oauth admission budgets over upstream's test-only keepalive workaround.
On windows-latest, a required management-dir timeout memo blocked config mutation harden in startServer. Isolate the config-mutation timeout key and keep the directory-timeout case as an init probe (HTTP 503 covered elsewhere).
saveConfig now passes a distinct timeoutMemoKey so management-dir timeouts cannot poison config mutation; update the win32 spy assertion to match.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/responses/spill-store.ts`:
- Around line 87-97: Add a focused Bun regression test near the existing
spill-store subsystem tests covering the directory-fsync failure path: configure
spillIoForTest.fsync to throw, verify "dir-fsync" is still recorded, and assert
the opened descriptor is closed. Also retain a separate assertion that an
openSync failure does not record "dir-fsync".
In `@tests/oauth-store-multi.test.ts`:
- Around line 315-327: Replace the pending Bun.sleep deadline in the
Promise.race around timedOut with a setTimeout-backed promise, retain its timer
handle, and clear it in a nested finally that always runs when the race settles.
Preserve the existing OAuthMutationBusyError assertion and deadline failure
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 7d7b5625-092d-4238-89a9-849bd376cbbb
📒 Files selected for processing (11)
.github/workflows/ci.ymlsrc/adapters/cursor/native-exec-shell.tssrc/config.tssrc/oauth/store.tssrc/responses/spill-store.tstests/config.test.tstests/oauth-store-multi.test.tstests/server-management-auth.test.tstests/shutdown-drain.test.tstests/storage-worker-lifecycle.test.tstests/storage-worker-teardown-isolate.test.ts
lidge-jun
added a commit
that referenced
this pull request
Aug 1, 2026
…e churn Cherry-picked from #849 (Wibias) onto current dev. That PR carries 21 commits against a base four commits behind and conflicts in four files that dev has since fixed independently — the two files here are the part dev still lacks, so they land directly rather than dragging a stale merge through. Bun 1.3.14 has a second Worker-reclaim failure mode beyond the Windows unbalanced-join panic: a segfault with a *balanced* workers_spawned === workers_terminated count, seen as exit 132/133 on both ubuntu and macOS Silicon under `bun test --isolate`. It crashes the whole run after green assertions, so every PR touching nothing related goes red. Two changes. `worker-lifecycle` now runs the post-close OS-join settle on darwin as well as win32, and disarms the terminate timeout before that settle so a late timer cannot report a false timeout after close already won. The isolate test caps spawn churn per platform (win32 8, linux 2, darwin 1) and skips the Worker-spawning cases on darwin, where even one cycle tripped the runtime; a meta-test keeps the caps themselves covered so the skip cannot silently rot. Co-authored-by: Wibias <Wibias@users.noreply.github.com>
Add focused spill-store seams/tests for throwing directory fsync and open failure, and clear the oauth wait-timeout race deadline so Bun.sleep cannot keep an isolate alive after settle.
Recent tip pushes only ran pull_request_target checks; touch the workflow comment so the path filter definitely re-evaluates.
Conflicted tip blocked GitHub's pull_request merge ref, so only pull_request_target checks ran. Keep config-mutation ACL timeoutMemoKey, directory fsync seams, and ref'd shell-grace timers on top of upstream's Windows runner/ACL harden.
Keep upstream's scoped ACL timeout fake for env management auth; config-mutation directory harden soft-fails that timeout so startServer still boots.
Collaborator
Author
[shipping-github] Verdict: approve-commentPR: UsefulnessYes. Bun 1.3.14 under Bugs / correctness
Security
Spec / standards
Reviews
Base / CI
Gatenone Bottom lineShip-ready. Tip keeps Cross-platform |
Collaborator
Author
[shipping-github] Merge readyPR: Reviews
Tip freshness
Checks
Residualnone for merge bar Ready to merge. |
Resolve oauth wait-timer, oauth-store-multi deadline, and management-auth ACL stub conflicts on top of the landed storage-worker isolate beforeEach work.
The failBeforeStateCommit + failSatelliteRestore path measured ~10s on windows-latest and tripped Bun's default 5s harness timeout.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bun test --isolatestill segfaults on macOS Silicon CI afterstorage-worker-teardown-isolatepasses with a balancedworkers_spawned === workers_terminatedcount (exit 133 / Trace/BPT). Seen on fix(update): recover Windows dashboard proxy after self-update #838 and earlier with heavier churn on ubuntu.afterAllbefore the isolate file boundary.beforeEachawait) — do not fold into that draft.Test plan
bun test ./tests/storage-worker-teardown-isolate.test.ts(4 pass on Windows)macos-latestgreen withbun test --isolate testsSummary by CodeRabbit
Bug Fixes
Tests