Skip to content

Fix missing Linux wheels: switch CI to cibuildwheel - #1

Merged
meixide merged 5 commits into
mainfrom
fix/linux-wheels-cibuildwheel
Jun 15, 2026
Merged

Fix missing Linux wheels: switch CI to cibuildwheel#1
meixide merged 5 commits into
mainfrom
fix/linux-wheels-cibuildwheel

Conversation

@meixide

@meixide meixide commented Jun 15, 2026

Copy link
Copy Markdown
Owner

Problem

PyPI hapc 2.3.0 ships wheels for macOS and Windows (cp38–cp312) but no Linux wheel — only the sdist. So on any Linux box, including every HPC cluster, pip install hapc falls back to compiling from source (the GLIBC_2.29 / sysroot / timespec_get / LIBRARY_PATH pain).

Root cause

The v2.3.0 workflow run reported success yet the Linux jobs produced zero wheels:

  1. The docker://quay.io/pypa/manylinux2014_x86_64 step started; yum install eigen3-devel succeeded.
  2. The hand-rolled loop iterated /opt/python/*/bin, but the [[ $PYBIN == *cp38* || ... ]] match never fired → pip wheel never ran → dist/ stayed empty.
  3. upload-artifact hit an empty dist/ and only warned (No files were found with the provided path: dist/) — a warning, not an error — so the job went green.
  4. publish-to-pypi then published whatever artifacts existed: mac + win wheels + sdist. No Linux wheel.

Fix

  • Replace the redundant 5×-per-OS matrix + docker loop with pypa/cibuildwheel@v2.23.3 — one job per OS, builds + repairs (auditwheel/delocate/delvewheel) all CPython ABIs (cp38–cp312).
  • manylinux2014 baseline (glibc 2.17 + bundled libstdc++) → wheels install and run unmodified on HPC clusters with no compiler / conda toolchain / sysroot.
  • macOS split into Intel (macos-13) + Apple Silicon (macos-14) native builds (no universal2 cross-compile gotcha, saner deployment target than the current macosx_15_0).
  • Every wheel is smoke-tested (import hapc) before publish — catches wrong-ABI / unresolved-symbol wheels.
  • Guardrails so this can't silently recur: if-no-files-found: error on the wheel upload, and a publish-time step that aborts unless a *manylinux*_x86_64.whl is present.
  • Build config moved into [tool.cibuildwheel] in pyproject.toml → reproducible locally via pipx run cibuildwheel.
  • README: add Linux/HPC install notes (wheel Just Works; source-build path with conda compilers / --no-binary for air-gapped nodes); fix yourusername placeholder URLs → meixide.

Validation

  • pyproject.toml + workflow YAML parse cleanly.
  • cibuildwheel --print-build-identifiers --platform linux → cp3X-manylinux_x86_64.
  • Full isolated build (pip wheel . --no-deps, build isolation pulling pybind11/cmake) produced a working wheel — the exact per-interpreter step cibuildwheel drives.

How to ship

  1. Dry run: trigger via workflow_dispatch — builds + smoke-tests wheels on all OSes; the publish job is gated on refs/tags/, so nothing uploads.
  2. Release: tag v2.3.1 to publish the full wheel set including Linux. (2.3.0 files already exist on PyPI and twine --skip-existing won't overwrite.)

Deferred (optional follow-ups)

  • pybind11 upper-bound pin — left uncapped; 3.0.4 builds fine in CI and now that wheels ship the old-sysroot source-build path is rare.
  • scikit-build-core backend and Eigen vendoring — bigger changes; setuptools+CMake works on all platforms and the README documents the offline build.
  • Drop the numpy<2.3 ceiling in a later release.

🤖 Generated with Claude Code

variacle and others added 4 commits June 15, 2026 12:00
@variacle @claude
Fix missing Linux wheels: switch CI to cibuildwheel
c9b5856 
The v2.3.0 release shipped macOS and Windows wheels but no Linux wheel —
only the sdist — so every Linux/HPC `pip install hapc` fell back to
compiling from source (GLIBC / timespec_get / sysroot pain).

Root cause: the hand-rolled manylinux job ran a `docker://manylinux2014`
container and looped over `/opt/python/*/bin`, but the
`[[ $PYBIN == *cp38* || ... ]]` match never fired, so `pip wheel` never
ran and dist/ stayed empty. upload-artifact only *warned* on the empty
dir, the job went green, and publish shipped only the mac/win artifacts.

Changes:
- Replace the redundant matrix + docker loop with pypa/cibuildwheel; one
  job per OS builds + repairs all CPython ABIs (cp38-cp312).
- manylinux2014 baseline (glibc 2.17 + bundled libstdc++) so wheels run
  unmodified on HPC clusters with no toolchain.
- Split macOS into Intel (macos-13) + Apple Silicon (macos-14) native
  builds; smoke-test every wheel with `import hapc` before publish.
- Guardrails so this can't silently recur: if-no-files-found: error on
  upload, and a publish-time check that aborts unless a manylinux wheel
  is present.
- Move build config into [tool.cibuildwheel] in pyproject.toml so it's
  reproducible locally via `pipx run cibuildwheel`.
- README: add Linux/HPC install notes; fix yourusername placeholder URLs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@variacle @claude
Fix manylinux + macOS builds exposed by the cibuildwheel dry run
05cb562 
The first dry run surfaced two latent bugs that the old workflow never hit
because it never actually compiled on Linux:

1. Linux: CMakeLists required the full Python3 `Development` component, which
   pulls in `Development.Embed` -> libpython. manylinux images deliberately
   ship no libpython, so configure failed with
   "Could NOT find Python3 (missing: Python3_LIBRARIES Development.Embed)".
   Use `Development.Module` (headers only) — the correct component for building
   an extension module. Requires CMake >= 3.18 (bumped minimum + build dep).

2. macOS: cibuildwheel's cp38 interpreter on the arm64 runner is x86_64-only,
   so CMake emitted an x86_64 .so while the wheel was tagged arm64, and
   delocate rejected the arch mismatch. Build proper fat universal2 wheels from
   one runner by forcing CMAKE_OSX_ARCHITECTURES. setup.py now appends the
   conventional $CMAKE_ARGS env var to the cmake invocation; pyproject sets it
   for macOS. (This also catches the old single-arch-but-universal2-tagged
   wheels — delocate now verifies both slices are present.)

Drop the redundant macos-13 runner: universal2 is built fat from macos-14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@variacle @claude
Make the Linux smoke test resolve scipy from a manylinux2014 wheel
7dcdaed 
The cibuildwheel dry run built and auditwheel-repaired all Linux wheels
(cp38-cp312) successfully, but the per-wheel `import hapc` smoke test failed:
installing the wheel pulls scikit-learn -> scipy, and the latest scipy no
longer ships a manylinux2014 wheel, so pip tried to build it from source inside
the glibc-2.17 test container and failed with "OpenBLAS not found".

Pre-install `scipy<1.14` via test-requires so the resolver stays on a scipy
version that has a 2014 wheel. The wheel under test is fine on real systems
(modern pip installs scipy's manylinux_2_28 wheel); this only fixes the
in-container smoke test.

(test-environment / PIP_ONLY_BINARY was tried first but that option only exists
in cibuildwheel 3.x; we pin 2.23.3 for cp38 support.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@variacle @claude
Force scipy to a binary wheel during the smoke test (fix Linux/Windows)
4f13745 
The previous attempt used test-requires=["scipy<1.14"], but cibuildwheel
installs the wheel-under-test (resolving its deps) *before* test-requires runs.
So on Linux the wheel install still pulled the latest scipy (no manylinux2014
wheel) and compiled it from source -> "OpenBLAS not found"; and on Windows cp38
the late `pip install scipy<1.14` tried to build scipy 1.13 (no cp38 support).

Set PIP_ONLY_BINARY=scipy in the cibuildwheel step env and forward it into the
manylinux container via CIBW_ENVIRONMENT_PASS_LINUX, so scipy resolves from a
(slightly older) binary wheel during the wheel install on every platform. Scoped
to scipy so it never blocks building hapc itself from source. macOS universal2
already passed and is unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@meixide

meixide commented Jun 15, 2026

Copy link
Copy Markdown
Owner Author

✅ Dry run green on all platforms — run 27556644116 (workflow_dispatch on this branch; publish job correctly skipped since it's not a tag).

Linux wheels now build, auditwheel-repair, and pass the import hapc smoke test:

hapc-2.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
hapc-2.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
hapc-2.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
hapc-2.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
hapc-2.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Plus macOS universal2 (x86_64+arm64), Windows AMD64, and the sdist.

Issues found and fixed during the dry runs (beyond the original CI swap):

  1. find_package(Python3 ... Development) required libpython, which manylinux doesn't ship → switched to Development.Module.
  2. macOS universal2 needed an explicit CMAKE_OSX_ARCHITECTURES (CMake ignores the interpreter arch flags) → set via CMAKE_ARGS.
  3. Latest scipy (transitive via scikit-learn) has no manylinux2014 wheel, so the in-container smoke test compiled it from source → force PIP_ONLY_BINARY=scipy during the test.

Ready to merge; tag v2.3.1 to publish the full wheel set incl. Linux.

@variacle @claude
Release v2.3.1
5591833 
Patch release shipping Linux manylinux2014 wheels (cp38-cp312) for the first
time, alongside the existing macOS (universal2) and Windows wheels, via the
cibuildwheel-based CI. No library code changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@meixide
meixide merged commit a760658 into main Jun 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@meixide @variacle