ci: add swap headroom to ShellCheck job (fixes intermittent OOM) - #100
Merged
Conversation
The ShellCheck CI job failed non-deterministically with `##[error]The operation was canceled.` — an OOM kill, not a lint finding. shellcheck's core dataflow analysis on tests/test-skill-contracts.sh (~11,671 lines) peaks at ~15-24 GB by itself (measured; all other test files combined peak at 287 MB, hooks at 80 MB), exceeding the 16 GB ubuntu-latest runner intermittently. The file lints clean (rc=0, 0 findings), so the failure was purely an out-of-memory kill on the agent. Add a dedicated 16 GB swapfile before the shellcheck steps so the kernel can page the cold GC copy-space instead of OOM-killing the agent. Both shellcheck steps and `--severity=warning` are byte-identical; lint coverage is unchanged. A distinct path /mnt/swapfile-ci avoids the runner image's default swap, and fallocate has a dd fallback for filesystems that reject holes. This is a pre-existing fragility (main's version of the file uses even more memory; recent releases passed CI by luck and v10.0.0 was admin-merged past a real OOM). Durable follow-up tracked separately: split test-skill-contracts.sh so no single file exceeds runner RAM (worst half measured ~6 GB). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw2bH4wbEPeebXsvSG6rWe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
ShellCheckCI job fails non-deterministically with##[error]The operation was canceled.— an OOM kill, not a lint finding. (It blocked v10.0.0, which was admin-merged past a real OOM.)Root cause (measured):
shellcheck --severity=warningontests/test-skill-contracts.sh(~11,671 lines) peaks at ~15-24 GB by itself — all othertests/*.sh+tests/helpers/*.shcombined peak at 287 MB andhooks/*.sh hooks/lib/*.shat 80 MB. The 16 GBubuntu-latestrunner OOM-kills the agent intermittently. The file lints clean (rc=0, 0 findings); the failure is purely a memory limit. It is check/severity-independent (--severity=errorand--exclude=SC2086both still ~14-24 GB) — shellcheck's core dataflow analysis, super-linear on the monolith.This is pre-existing:
main's version of the file uses even more memory; recent releases passed by luck.Fix
Add a dedicated 16 GB swapfile before the shellcheck steps (16 GB RAM + ~4 GB default + 16 GB new ≈ 36 GB addressable, covering the worst measured peak). The kernel pages the cold GC copy-space instead of OOM-killing the agent.
shellcheck --severity=warningsteps and their coverage are byte-identical — the lint surface is unchanged./mnt/swapfile-ciavoids the runner image's default swap;fallocatehas addfallback for filesystems that reject holes.swapon --show+free -hlogged for observability.Verification
/usr/bin/timepeak ~15-24 GB, rc=0, 0 findings) and that more addressable memory (ulimit -v 32000000vs13000000) lets it complete.swapon --showlists/mnt/swapfile-ciat 16G) and the job goes green reliably across re-runs (the bug is intermittent).Follow-up (durable, tracked separately)
Split
tests/test-skill-contracts.shso no single file exceeds runner RAM (worst half measured ~6 GB at the category boundary line 6989); then the swap step can be removed.🤖 Generated with Claude Code
https://claude.ai/code/session_01Mw2bH4wbEPeebXsvSG6rWe