Fix Linux release binaries requiring GLIBC_2.39, bump to 1.9.3 - #139
Fix Linux release binaries requiring GLIBC_2.39, bump to 1.9.3#139Ibrahimrahhal wants to merge 8 commits into
Conversation
Reads the ELF version-need table of a built artifact and fails when a *-gnu binary requires a glibc newer than GLIBC_FLOOR, or when a *-musl binary is not fully static. Also checks the ELF machine matches the target triple. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
Building the *-gnu targets natively stamped the runner's glibc into the ELF version-need table, so every release since v1.7.2 required GLIBC_2.39 and failed to load on Ubuntu 22.04 (glibc 2.35) and older: corgea: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found corgea: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.39' not found Cross-build all Linux targets on ubuntu-latest with cargo-zigbuild, pinning the gnu targets to GLIBC_FLOOR, and add an aarch64 musl target so ARM64 hosts have a libc-independent option. Each artifact is verified before it is archived. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
Linux x64 already resolves to the musl build; arm64 pointed at the gnu build and inherited its glibc requirement. Also ignore the generated vendor/ tree so bundled binaries cannot be committed by accident. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
The local release script built the gnu targets natively, which produces exactly the binaries the pipeline now rejects. Route Linux through cargo-zigbuild with the same floor and verify the result. Drops the x86_64-pc-windows-gnu target, which has not been part of a release since v1.7.2 and was the script's only remaining use of cross. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
The aarch64 artifacts are now cross-built, so no job in CI would otherwise execute them. Running --version catches a binary that passes the ELF header checks but fails to start. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
Ships the Linux glibc floor fix. No tag on the 1.9.x line has a usable aarch64-unknown-linux-gnu binary, so a release is the only way to unblock ARM64 users. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
| uses: mlugg/setup-zig@v2 | ||
| with: | ||
| version: ${{ env.ZIG_VERSION }} | ||
|
|
||
| - name: Install cargo-zigbuild | ||
| if: runner.os == 'Linux' | ||
| uses: taiki-e/install-action@v2 |
There was a problem hiding this comment.
The two newly added third-party actions are referenced by mutable major-version tags, while this workflow has contents: write (line 13) and runs on every push. If either tag is moved or its upstream is compromised, that code receives a write-capable GITHUB_TOKEN and can modify releases or repository contents. Pin both actions to immutable full commit SHAs; the current tag commits are shown below.
| uses: mlugg/setup-zig@v2 | |
| with: | |
| version: ${{ env.ZIG_VERSION }} | |
| - name: Install cargo-zigbuild | |
| if: runner.os == 'Linux' | |
| uses: taiki-e/install-action@v2 | |
| uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2 | |
| with: | |
| version: ${{ env.ZIG_VERSION }} | |
| - name: Install cargo-zigbuild | |
| if: runner.os == 'Linux' | |
| uses: taiki-e/install-action@18b1216eba7f8039b0f8d131d5473787f0edce68 # v2.85.3 |
| ;; | ||
|
|
||
| *-gnu) | ||
| needed=$(readelf -V "$binary" | grep -o 'GLIBC_[0-9.]*' | sed 's/^GLIBC_//' | sort -uV) |
There was a problem hiding this comment.
With set -euo pipefail, grep returning 1 for no matches aborts this assignment before the intended empty-result branch on lines 60–63 can run. I reproduced it by checking a static x86_64 ELF as the GNU triple: the script exited 1 with no diagnostic. That makes the new release gate silently reject a compatible GNU artifact with no versioned glibc symbols. Capture successful readelf output first, then tolerate only grep's no-match result; requiring a digit also avoids treating GLIBC_PRIVATE as an empty version.
| needed=$(readelf -V "$binary" | grep -o 'GLIBC_[0-9.]*' | sed 's/^GLIBC_//' | sort -uV) | |
| version_info=$(readelf -V "$binary") | |
| versions=$(grep -oE 'GLIBC_[0-9]+(\.[0-9]+)*' <<<"$version_info" || true) | |
| needed=$(printf '%s | |
| ' "$versions" | sed 's/^GLIBC_//' | sort -uV) |
set -euo pipefail made grep's no-match exit status abort the assignment, so the branch that accepts a binary with no versioned glibc symbols was unreachable and the script exited 1 with no diagnostic. Run grep outside the pipeline that feeds the assignment, and require a digit so GLIBC_PRIVATE and GLIBC_ABI are not read as versions. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
This workflow holds contents: write, so a moved tag on either action would get a write-capable GITHUB_TOKEN. Co-authored-by: Ibrahim Rahhal <ibrahim.rahhal3636@gmail.com>
Problem
A customer on a Harness Cloud ARM64 delegate (Ubuntu 22.04, glibc 2.35) cannot run
corgea-aarch64-unknown-linux-gnu.zip:They asked which version to pin. The answer is none — every published Linux
-gnubinary is affected, on both architectures, going back to the first aarch64 build inv1.7.2:aarch64-...-gnux86_64-...-gnux86_64 users have had a static musl fallback since
v1.8.3. ARM64 users have had no working option at all, including vianpm install -g @corgea/cli, becausebin/corgea.jsmaps linux/arm64 to the gnu build.Root cause
release-binaries.ymlbuilt the gnu targets natively onubuntu-latest(24.04) andubuntu-24.04-arm, both glibc 2.39. Linking against the runner's glibc records version requirements in the ELF version-need table that an older loader refuses. No floor was pinned anywhere, so the supported minimum silently tracked GitHub's runner images.Exactly three symbols pushed the floor from 2.34 to 2.39:
__isoc23_strtol(GLIBC_2.38) — C code compiled against glibc 2.38+ headers (vendored OpenSSL / bundled libgit2); GCC 13 redirectsstrtolto the C23 variant. Hard undefined, fatal on its own.pidfd_spawnp,pidfd_getpid(GLIBC_2.39) — Rust std's process-spawn path. Weak, but still recorded inverneed, which is why the loader names them too.Changes
release-binaries.yml— all four Linux targets cross-build onubuntu-latestwithcargo-zigbuild, pinning the gnu targets to a workflow-levelGLIBC_FLOORof2.17. Addsaarch64-unknown-linux-musl, so ARM64 gets a fully static option like x86_64 already had. Theubuntu-24.04-armrunner and themusl-toolsstep are no longer needed. Both new actions are pinned to commit SHAs, since this workflow holdscontents: write.scripts/check-linux-binary.sh— new guard, run on every Linux artifact before it is archived. Fails if a gnu binary needs a glibc above the floor, if a musl binary is not static, or if the ELF machine does not match the target triple. This is what stops the floor from silently drifting again.--version, ARM64 underqemu-user-static.bin/corgea.js/scripts/npm/bundle-binaries.js— Linux arm64 now resolves to the musl binary, matching what x64 already does.build_release.sh— routes Linux through the same zigbuild + verification path, so local builds cannot produce artifacts the pipeline would reject. Dropsx86_64-pc-windows-gnu, absent from releases sincev1.7.2and the script's only remaining use ofcross.Cargo.toml→1.9.3— a fix in the pipeline changes nothing for users until a tag is cut, and no tag on the 1.9.x line has a working ARM64 binary. PerRELEASING.mdthis is the only manual version edit; PyPI and npm derive theirs from it and the tag.RELEASING.md/README.md— document the floor, how to check it, and the gnu vs. musl choice for Linux.GLIBC_2.17covers RHEL 7 / Ubuntu 14.04 and newer, and matches themanylinux2014floorrelease.ymlalready asserts for the PyPI wheels. It is also Rust's hard minimum for*-unknown-linux-gnu: building at.2.12fails withundefined symbol: getauxvalfromstd.Verification
All evidence below uses the artifacts CI actually produced, not local builds.
The 1.9.3 candidate on an unmodified Ubuntu 22.04 ARM64 rootfs (glibc 2.35) under qemu:
glibc range, same aarch64 gnu artifact: runs on glibc 2.23 (Ubuntu 16.04) and 2.35 (Ubuntu 22.04); the x86_64 gnu artifact runs on 2.39 (Ubuntu 24.04), so the old floor did not cost forward compatibility.
TLS and DNS, a live authentication request against the Corgea API returning its 401, proving DNS, TLS handshake and CA verification all work — including through musl's resolver:
Real workload,
corgea deps scanandcorgea deps diff --base HEAD~1on the glibc 2.35 rootfs, both producing correct findings.Guard behaviour, including against the artifact that shipped broken, a binary with no versioned glibc references, and one carrying
GLIBC_PRIVATE:Size, all artifacts got slightly smaller: aarch64 gnu 15.4 MB → 14.7 MB, x86_64 musl 17.0 MB → 15.2 MB.
npm path,
bundle-binaries.jsover a real asset set produces the expectedvendor/layout and the vendored arm64 musl binary runs../harness cipasses locally in full — strict clippy, format check, dep audit, 524 tests, coverage gate — which is the pre-tag gate inRELEASING.md.Review feedback addressed
set -euo pipefail,grep's no-match exit status killed the assignment, so the branch handling that case was unreachable and the script exited 1 with no diagnostic. Reproduced by checking a static binary against the gnu triple, fixed by runninggrepoutside the pipeline that feeds the assignment. The pattern now requires a digit, soGLIBC_PRIVATEandGLIBC_ABIare not read as versions.contents: writeworkflow.mlugg/setup-zigandtaiki-e/install-actionare pinned to commit SHAs, verified against the live tags.Releasing
Once merged, follow
RELEASING.md:gh release create v1.9.3 --target main --generate-notes. The release will carry a new asset,corgea-aarch64-unknown-linux-musl.zip.Follow-ups, not in this PR
softprops/action-gh-release@v2andactions/upload-artifact@v4in this same workflow are still on mutable tags. Pre-existing, and out of scope here, but the same argument applies.release.ymlbuilds no aarch64 Linux wheels, sopip install corgea-clion ARM64 still compiles from the sdist.corgea scan's in-process libgit2 paths sit behind authentication and were not exercised here; worth one authenticated run on a real ARM64 machine before announcing.