install: tell a too-new gcc how to build, instead of letting ninja fail - #67
Merged
Conversation
`--codebook` defaulted to `e8_shell`, the original 8-D lattice, while the README recommended trellis and the checkpoint table marked a trellis build as the fastest GLQ decode. Someone following the docs and running `glq-quantize` got a lattice checkpoint. Trellis (QTIP TCQ) has been the recommended path since v0.7 and wins where bits are scarcest — SmolLM3-3B at 2 bpw is PPL 11.94 against 13.79. Two defaults had to move together. `GLQ_TRELLIS_VARIANT` defaulted to `hyb`, but the fused CUDA kernels consume a **3INST** checkpoint; a hyb one falls back to the pure-torch decode, "correct but materially slower". Defaulting to trellis alone would have shipped checkpoints with no fast path — worse than the lattice default it replaced. Quality is a wash: 3INST-4 matches HYB-4 on SQNR and PPL. Consequences handled rather than discovered later: - **`scripts/batch_quantization.py` omitted `--codebook` when it equalled `e8_shell`**, relying on the CLI default matching its own. It now always passes the flag: a job that declared e8_shell would otherwise have silently become trellis, and any fractional-bpw job would then fail, since trellis takes uniform integers only. - **The resume run-key** now hashes the real variant instead of `""`. This invalidates existing trellis resume stores, which is correct: they hold *hyb* work that must never be resumed into a 3inst run. - **The mixed-precision refusal now names `--codebook e8_shell`.** `--bpw 3.5` used to work with no flag at all; without that pointer the new default turns it into a dead end. - README's "fractional like 2.5 also works" is no longer true by default, and says so. `build_parser()` is split out of main() so CLI defaults can be asserted without running a quantization — there was no test surface for them before. Also documents the Blackwell/FlashInfer caveat next to the `vllm serve` examples: on sm_120 with no CUDA Toolkit, vLLM's sampler compiles at startup and takes the engine down before the first token. glq-chat falls back on its own; `vllm serve` and `LLM(...)` do not. Full local suite: 1085 passed, 882 skipped.
fedora:44 ships gcc 16; CUDA's crt/host_config.h refuses anything past 15. A source install
there dies with "unsupported GNU version" on every .cu file, after the user has already
waited out a PyTorch download. Pre-flight promises the opposite: run this command and you
are ready.
Pre-flight now reports the mismatch and names the fix:
nvcc: host gcc is 16; CUDA supports <= 15
prebuilt wheels are unaffected — they compile nothing
to build from source: sudo dnf install -y gcc15 gcc15-c++
then: export NVCC_CCBIN=/usr/bin/g++-15
Deliberately a note, not a blocker. Since 0.8.6 the prebuilt wheels cover cp310-cp314, so
the common path compiles nothing and works fine on fedora:44 — blocking would refuse an
install that succeeds. Only --glq-source reaches nvcc.
Measured, not inferred:
- The cap is real and not an artifact of the pip-shipped headers. The full CUDA 13.3.1
toolkit (4.1 GB) refuses gcc 16 identically, so "install the toolkit" is not a fix.
- NVCC_CCBIN is: with the default gcc 16 the build produced 6 "unsupported GNU version"
errors; with NVCC_CCBIN=/usr/bin/g++-15, zero.
- End to end on fedora:44 — follow pre-flight's package advice, then its compat-compiler
advice, then `install.sh --glq-source` with NVCC_CCBIN exported: install exit 0,
EXT_OK:True, 0 gcc errors. The extension builds where it previously could not.
Only the Fedora mapping is verified. The apt/pacman/zypper lines follow each distro's usual
naming and are marked `# unverified` in the source rather than presented as tested.
Tests fake `gcc -dumpversion` with a PATH shim rather than an env back door, so the real
detection is exercised: advice appears at gcc 16, stays silent at 15, names both the package
and NVCC_CCBIN, and never changes the exit status.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fedora:44 ships gcc 16; CUDA's
crt/host_config.hrefuses anything past 15. A source install there dies withunsupported GNU versionon every.cufile — after the user has waited out a PyTorch download. Pre-flight promises the opposite: run this command and you are ready.What pre-flight now says
A note, not a blocker. Since 0.8.6 the prebuilt wheels cover cp310–cp314, so the common path compiles nothing and works fine on fedora:44 — blocking would refuse an install that succeeds. Only
--glq-sourcereaches nvcc, and one test asserts the exit status is unchanged.Measured, not inferred
nvidia/cu13headersNVCC_CCBINremoves itunsupported GNU versionerrors;NVCC_CCBIN=/usr/bin/g++-15→ 0install.sh --glq-sourcewithNVCC_CCBINexported → install exit 0,EXT_OK:True, 0 gcc errorsThat last row is the point: the extension builds on fedora:44, which was previously impossible.
Honesty about coverage
Only the Fedora mapping is verified. The apt / pacman / zypper lines follow each distro's usual naming and carry
# unverifiedin the source rather than being presented as tested — same convention aspkg_hint's RHEL notes.Ordering quirk, documented in the source
On a pristine image there is no
gccyet, so this says nothing on the first pre-flight run and appears on the second, once the package advice has been followed. That is the order the installer is used in (pre-flight → install packages → pre-flight), and guessing the version of a compiler that is not installed would be worse than silence.Tests
Four new cases in
tests/test_preflight_sh.py, CPU-only and CI-runnable. They fakegcc -dumpversionwith a PATH shim rather than an env back door, so the real detection path is exercised: advice appears at gcc 16, stays silent at 15, names both the package andNVCC_CCBIN, and never changes the exit status. 57 pre-flight tests pass.