perf(shard): adaptive busy-poll contention governor — --io-busy-poll-us is now deploy-safe (O3)

[pilotspacex-byte]

The p=1 busy-poll win (GCE pinned c1: ARM c4a 0.95->1.21x, x86 c3
1.06->1.66x vs Redis) inverted into a regression whenever the shard's
core was shared (documented OrbStack/laptop finding), which made the
flag pinned-cores-only operator judgment — the last open item (O3) in
tmp/CPU-CACHE-DIGEST.md.

Each shard thread's 1s chore now samples its own
nonvoluntary_ctxt_switches from /proc/thread-self/status — the kernel's
direct "another runnable thread needed this core" signal — and flips a
per-thread contention gate in the vendored monoio legacy driver
(monoio::set_legacy_spin_contended; the gated park skips the readiness
spin exactly as if the budget were 0, leaving budget and idle-disengage
state untouched).

Policy (src/shard/spin_governor.rs, asymmetric hysteresis):

• one window over 25 preempts/s -> gate immediately (a contended core
  stops burning budget within ~1s);
• 5 consecutive clean windows -> re-enable (no flapping);
• start ungated: pinned deployments see the win from the first request,
  a shared-core host pays at most ~one window of spin.

A sleeping shard cannot be involuntarily preempted, so an idle server
never gates — idle spin cost is already bounded by the driver's 10ms
idle-disengage (unchanged). Non-Linux has no preemption signal: the
governor is inert there (pre-O3 behavior). This also makes
--profile standalone (which presets busy-poll 40) safe on
non-dedicated hosts.

Knobs: MOON_SPIN_ADAPTIVE=0 restores unconditional spinning
(same-binary A/B + escape hatch); MOON_SPIN_MAX_PREEMPTS_PER_SEC
overrides the threshold (diagnostics).

Validation:

• governor state machine unit-tested (fast-off, slow-on, streak reset,
  boundary, /proc parse); clippy clean on both feature matrices
  (module + wiring are monoio-only; tokio unaffected).
• VM (shared cores, the regression environment): CPU hog pinned to the
  shard core trips "spin GATED" within a window; full quiet after the
  hog produces "re-enabled" after 5 windows; MOON_SPIN_ADAPTIVE=0
  control leg logs nothing. Under sustained VM cross-traffic the
  governor stays gated — which is the correct answer on that box.
• shards=4 busy-poll functional smoke (SET/GET p1, MSET, EVAL, DBSIZE)
  green with the governor active.
• GCE t2a pinned win-preservation A/B (shards=1, client tasksetted to
  cores 6-7, 3 leg-order-alternating rounds, medians): adaptive vs
  MOON_SPIN_ADAPTIVE=0 at c1 = GET -3.0% / SET +4.6% (equivalent within
  noise), BOTH ~+40% over --io-busy-poll-us 0 (GET 36.2K/37.3K vs
  25.1K; SET 29.8K/28.5K vs 21.9K) — the win is preserved, and ZERO
  "GATED" lines were logged in any adaptive leg under full load on the
  dedicated core (no false positives). Observed in passing (pre-existing
  flag property, identical in both spin legs, NOT an O3 delta): on t2a
  at c=8 p=1 the spin itself costs SET ~-30% vs no-spin — the flag's
  win profile is connection-count-dependent on this machine type; the
  published c4a/c3 numbers were c1.

Refs tmp/CPU-CACHE-DIGEST.md (O3), task #31

Summary by CodeRabbit

• Performance
  • Improved --io-busy-poll-us with an adaptive, per-shard contention governor that samples involuntary context-switch activity to safely gate spinning and restore it when conditions improve.
  • Added/updated controls via MOON_SPIN_ADAPTIVE and MOON_SPIN_MAX_PREEMPTS_PER_SEC (including MOON_SPIN_ADAPTIVE=0 to return to unconditional spinning).
  • Updated documentation clarifying that behavior is Linux-only (governor is inert on non-Linux) and that --profile standalone is safe on non-dedicated hosts.

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2d8d19ad-b2ff-4b68-ad48-fab8bcf64021

📥 Commits

Reviewing files that changed from the base of the PR and between 36c3e2f and aa923ea.

📒 Files selected for processing (9)

• CHANGELOG.md
• CLAUDE.md
• src/config.rs
• src/shard/event_loop.rs
• src/shard/mod.rs
• src/shard/spin_governor.rs
• vendor/monoio/src/driver/legacy/mod.rs
• vendor/monoio/src/driver/mod.rs
• vendor/monoio/src/lib.rs

🚧 Files skipped from review as they are similar to previous changes (9)

• src/shard/mod.rs
• vendor/monoio/src/lib.rs
• CLAUDE.md
• src/config.rs
• vendor/monoio/src/driver/legacy/mod.rs
• src/shard/event_loop.rs
• vendor/monoio/src/driver/mod.rs
• CHANGELOG.md
• src/shard/spin_governor.rs

---

📝 Walkthrough

Walkthrough

Adds a per-thread adaptive busy-poll governor for monoio. Linux preemption counters drive hysteresis-based spin gating, while non-Linux builds remain inert. Runtime wiring, environment controls, tests, and documentation are included.

Changes

Adaptive busy-poll governance

Layer / File(s)	Summary
Monoio spin gate vendor/monoio/src/driver/legacy/mod.rs, vendor/monoio/src/driver/mod.rs, vendor/monoio/src/lib.rs	Adds a thread-local contention flag and legacy-only APIs that prevent readiness spinning while contention is active.
Preemption sampling and hysteresis src/shard/spin_governor.rs	Adds environment-controlled Linux counter sampling, threshold hysteresis, governor state transitions, non-Linux inert behavior, and unit tests.
Shard integration and operational documentation src/shard/mod.rs, src/shard/event_loop.rs, src/config.rs, CLAUDE.md, CHANGELOG.md	Constructs and samples the governor in monoio shard loops, applies contention state to monoio, and documents the adaptive behavior and controls.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ShardRun
  participant SpinGovernor
  participant ProcStatus
  participant MonoioLegacyDriver
  ShardRun->>SpinGovernor: tick every 1 second
  SpinGovernor->>ProcStatus: read preemption counter
  ProcStatus-->>SpinGovernor: counter delta
  SpinGovernor-->>ShardRun: gated or enabled state
  ShardRun->>MonoioLegacyDriver: apply contention state
  MonoioLegacyDriver-->>ShardRun: spin or block in inner_park

Loading

Possibly related PRs

• pilotspace/moon#216: Both modify legacy-driver spin-poll gating with thread-local or idle-disengage controls.
• pilotspace/moon#378: Both modify monoio shard tick-loop behavior affecting governor sampling cadence.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: an adaptive busy-poll contention governor making --io-busy-poll-us deploy-safe.
Description check	✅ Passed	The PR description is detailed and covers summary, validation, impact, and notes, matching the template intent.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/o3-adaptive-spin-governor

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}