Skip to content

test: rare intermittent failure in the full bun test suite (~3 in 25 runs) #175

Description

@ramarivera

The full bun test suite intermittently reports a failure that does not reproduce on re-run. Observed three times in roughly 25 runs during the #170 / #173 / #174 work:

  • one run reporting 2 fail, 1 error
  • two runs reporting 1 fail

Every attempt to catch it failed. Immediately after each occurrence the suite was re-run and came back 0 fail, and two deliberate capture loops (5 and 6 consecutive runs each, stdout tee'd to files) produced zero failures, so no test name was ever recorded.

What is known:

  • It predates feat(version): keep every version field in sync with package.json #174 — the first occurrence was at 471 tests, before sync-version existed.
  • Per-file runs of the suspected files are always clean.
  • It is not the achievement-banner TTL tests: those were run in isolation repeatedly and are deterministic apart from a 901s-vs-900s wall-clock margin roughly 900x larger than observed test runtime.

Leading hypothesis (unproven): contention with live companion state. statusline/buddy-status.test.ts spawns the real buddy-status.sh many times, and a running statusline in another session concurrently writes .substatus.* files in the live state dir. A full-suite run with CLAUDE_CONFIG_DIR unset and HOME redirected to a temp dir wrote nothing under the fake home, so it is not a straightforward path leak — but timing-sensitive interaction has not been ruled out.

Next step for whoever picks this up: run bun test in a loop under load with output retained per run (for i in $(seq 30); do bun test > run$i.log 2>&1; done) and grep for (fail). Without a captured test name this cannot be diagnosed further.

Filing rather than dismissing it — a failure that only appears under whole-suite runs is exactly the kind that gets waved off until it lands in CI.

🤖 Created with the help of AI (Claude Opus 5).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions