ci: run the CUDA compile check in a devel container - #4
Merged
Conversation
Installing the CUDA toolkit onto a bare runner cannot work: the cuda-12-6 meta-package depends on the NVIDIA kernel driver, dkms cannot build it on a GitHub runner, and dpkg then fails the entire install including nvcc and cuBLAS. Switching to the full toolkit made this worse, not better. nvidia/cuda:12.6.2-devel-ubuntu24.04 ships nvcc and cuBLAS and needs no driver, which is the right shape for a compile-only check. This mirrors the upstream ubuntu-latest-cuda job. Also enables LLAMA_FATAL_WARNINGS, since -Werror is what surfaced two of the build errors this workflow exists to catch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q5jMcvdevae8j4T36C9h2z
ggml-cuda is 131 translation units, ~70 of them generated template instances, and nvcc compiles each one several times over (device to PTX, PTX to SASS, then the host pass). On a four-core runner starting from an empty build directory that dominates the entire job. ggml's CMake enables ccache automatically once the binary is present, so the action only has to carry the cache between runs. Saving is restricted to pushes on master so pull requests read from a stable cache instead of each writing its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q5jMcvdevae8j4T36C9h2z
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The TurboQuant CI CUDA job has been failing on master. The cause is the toolkit installation, not the code.
cuda-12-6depends on the NVIDIA kernel driver. dkms cannot build a kernel module on a GitHub runner, dpkg fails, and the failure cascades throughcuda-runtime-12-6to the packages the build actually needs. Installing the full toolkit — my previous attempt at avoiding package-name guesswork — made this certain rather than avoiding it, because the meta-package is exactly what pulls the driver in.A
develcontainer already contains nvcc and cuBLAS and installs no driver at all, which is what a compile-only check needs. This is the same approach as the upstreamubuntu-latest-cudajob.Also enables
LLAMA_FATAL_WARNINGS:-Werroris what exposed two of the build errors this workflow was added to catch, so the check should run with it.🤖 Generated with Claude Code
https://claude.ai/code/session_01Q5jMcvdevae8j4T36C9h2z