Skip to content

Build the Python binding with nanobind + scikit-build-core and ship bundled wheels - #136

Open
grzanka wants to merge 5 commits into
mainfrom
claude/issue-131-C9Tuo
Open

Build the Python binding with nanobind + scikit-build-core and ship bundled wheels#136
grzanka wants to merge 5 commits into
mainfrom
claude/issue-131-C9Tuo

Conversation

@grzanka

@grzanka grzanka commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Closes #131. Implements the binding-technology decision: the Python binding is now a nanobind extension (libdedx._core) built by scikit-build-core that statically links the libdedx C library. Since libdedx embeds its data and needs no GSL, the result is a single self-contained extension and trivially self-contained wheels — pip install libdedx works with no local C build.

What changed

Binding — python/src/dedx_core.cpp

Exposes the full C API the old ctypes wrapper omitted:

Build

  • Top-level pyproject.tomlbuild-backend = "scikit_build_core.build"; build-requires nanobind, scikit-build-core>0.10, setuptools-scm>=8; runtime dep numpy.
  • The repo root is the CMake source tree; the extension is added under SKBUILD, which also skips examples/tests and the C-library install/CPack rules so the wheel contains only the module (verified: wheel ships just libdedx/{__init__.py,_core.so,_core.pyi,py.typed}).
  • Version lockstep: setuptools_scm resolves the version and feeds it into the C library (SKBUILD_PROJECT_VERSION_FULL), so dedx_get_version_string() and the wheel version match.
  • License metadata fixed to GPL-3.0-or-later (bug: license metadata mismatch — Python binding declares LGPL-2.0 but library is GPL-3.0 #115) — matches COPYING/the source headers (was incorrectly LGPL-2.0-or-later).

Note / deviation from the issue text: the issue listed python/pyproject.toml, but pyproject.toml is at the repo root here. With the C sources in src//include/ (one level above python/), the build context that pip/cibuildwheel copy must contain them — only a root-level pyproject.toml makes that work for non-editable wheels and sdists. The Python package itself still lives under python/libdedx/. Happy to revisit if you'd prefer the vendoring route instead.

Docs

  • README.md, index.rst (rendered in the Sphinx site), and CONTRIBUTING.md now document the Python binding (install + quick start, API surface, and the developer build/test workflow, array-ownership rule, version lockstep, and inherited thread-safety limitation).

CI — .github/workflows/wheels.yml

  • cibuildwheel, CPython 3.9–3.14, skip PyPy/musllinux, on Linux + Windows + macOS, macOS deployment target 11.0.
  • Install-and-import smoke test + full test suite run against every built wheel.
  • Builds an sdist and verifies it builds a wheel from that sdist (self-containment check).
  • A top-level least-privilege permissions: contents: read block; publish jobs opt into id-token: write.
  • TestPyPI dry-run → PyPI, both via OIDC Trusted Publishing on v* tags (testpypi/pypi environments).
  • ci.yml python_tests now builds the extension via pip install -e .[dev] and drops the old LIBDEDX_SO flow.

Open decisions (from the issue) — choices made

  • macOS: separate x86_64 + arm64 wheels (not universal2), both built on the arm64 macos-14 runner (arm64 natively, x86_64 cross-compiled). The Intel macos-13 runner is deliberately avoided — its capacity is scarce and jobs were queuing for 25+ minutes.
  • aarch64 Linux: deferred (musllinux also skipped) to keep build times down — easy to add later via cibuildwheel archs.
  • ctypes fallback: dropped — nanobind-only, as recommended.

Verification

  • Editable + isolated pip install build the extension; Python tests pass, ruff clean.
  • python -m build builds an sdist and then a wheel from that sdist (self-contained); installing the wheel in a clean venv imports and computes with no source tree present.
  • Plain C build unaffected: cmake configure/build OK, 27/27 ctest pass.
  • CI green across Linux/Windows/macOS × CPython 3.9–3.14 (wheels, sdist, C build/tests, valgrind, CodeQL, coverage, docs).

Acceptance criteria

  • pip install from a wheel works with no local C build (CI covers Linux/macOS/Windows × 3.9–3.14).
  • Low-level module exposes workspace/config, custom compounds, inverse STP/CSDA, unit conversion, accessors, lists/names — covered by python/tests/test_core.py.
  • Wheels install-smoke-tested in CI before publish; TestPyPI dry-run wired before PyPI.
  • Version lockstep with the C library via setuptools_scm.
  • License metadata corrected (bug: license metadata mismatch — Python binding declares LGPL-2.0 but library is GPL-3.0 #115); ruff + pytest green.

Refines #117 · relates to #112, #115, #119, #111.

https://claude.ai/code/session_01LH8M1RBJBdd3x69Fgu76No


Generated by Claude Code

@codecov

codecov Bot commented Jun 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.86%. Comparing base (60d05f0) to head (8094912).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #136   +/-   ##
=======================================
  Coverage   72.86%   72.86%           
=======================================
  Files          12       12           
  Lines        1640     1640           
  Branches      299      299           
=======================================
  Hits         1195     1195           
  Misses        445      445           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread .github/workflows/wheels.yml Fixed
Comment thread .github/workflows/wheels.yml Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the previous ctypes Python wrapper with a compiled nanobind extension (libdedx._core) built via scikit-build-core, and adds wheel-building/publishing automation so pip install libdedx can work from prebuilt, self-contained wheels that statically link the C library.

Changes:

  • Add a new nanobind C++ extension implementing the workspace/config object model plus vectorized NumPy entry points and accessors, and reimplement the legacy high-level helpers on top of _core.
  • Switch Python packaging to a root-level pyproject.toml using scikit-build-core + setuptools_scm, with CI for wheel/sdist builds and Trusted Publishing to TestPyPI/PyPI.
  • Adjust CMake install/build logic under SKBUILD to avoid installing the C library artifacts into wheels; update tests/docs accordingly.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
CMakeLists.txt Adds DEDX_BUILD_PYTHON and SKBUILD-specific configuration (Python-only builds, version lockstep, skip packaging).
src/CMakeLists.txt Skips dedx_shared and C-library install rules under SKBUILD for wheel builds.
python/CMakeLists.txt Adds CMake build for the _core nanobind module linking the static dedx target.
python/src/dedx_core.cpp Implements the new libdedx._core nanobind binding (Config/Workspace + module-level APIs).
pyproject.toml Root-level scikit-build-core packaging config + cibuildwheel settings and metadata updates.
.github/workflows/wheels.yml Adds multi-OS wheel builds, sdist build/check, and OIDC Trusted Publishing (TestPyPI then PyPI).
.github/workflows/ci.yml Updates Python job to build/install the extension via pip install -e ".[dev]" and run tests/ruff.
python/libdedx/__init__.py Re-exports _core APIs and reimplements legacy convenience helpers on top of _core.
python/libdedx/_core.pyi Adds type stubs for the compiled _core module.
python/tests/test_core.py Adds tests for the new low-level _core API surface.
python/tests/test_libdedx.py Updates high-level smoke tests to use _core constants and adds version-string checks.
python/tests/conftest.py Removes shared-library discovery/env-var setup (no longer needed with static linking).
python/README.md Updates Python binding documentation for nanobind/scikit-build-core usage and workflow.
python/libdedx/_api.py Removes the old ctypes-backed implementation.
python/libdedx/__init__.pyi Removes stubs for the old ctypes-based surface (now typed via inline annotations / _core.pyi).
python/pyproject.toml Removes the old Python-subdir setuptools-based packaging config.
.gitignore Ignores new packaging artifacts (dist/, wheelhouse/, etc.).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread python/src/dedx_core.cpp
Comment thread python/src/dedx_core.cpp
Comment thread python/CMakeLists.txt Outdated
claude added 5 commits June 16, 2026 16:30
Implements the binding-technology decision from #131: the Python binding is
now a nanobind extension (`libdedx._core`) built by scikit-build-core that
statically links the libdedx C library. Because libdedx embeds its data and
needs no GSL, the result is a single self-contained extension and trivially
self-contained wheels — `pip install libdedx` works with no local C build.

Binding (`python/src/dedx_core.cpp`):
- Workspace/Config object model (allocate -> load -> evaluate) with safe
  ownership of the C config's malloc'd element arrays.
- Custom compounds (Z + atoms or mass-fractions, rho, per-element I-values).
- stp, CSDA, inverse STP and inverse CSDA (ion_a auto-filled on load so the
  object model works without the C convenience wrappers).
- Unit conversion (incl. keV/um), composition / I-value / density accessors,
  min/max energy, program/ion/material lists and names, Bragg provenance.
- numpy in/out for the vectorised entry points.

Build:
- Top-level pyproject.toml uses build-backend scikit_build_core.build; the repo
  root is the CMake source tree and the extension is added under SKBUILD, which
  also skips the examples/tests and the C-library install/CPack rules so the
  wheel only contains the module. The version is taken from setuptools_scm and
  fed into the C library for lockstep versioning.
- pyproject.toml moved to the repo root (was python/pyproject.toml) so the C
  sources are inside the build context, which is required for self-contained
  wheels via cibuildwheel and source installs.

CI:
- wheels.yml builds CPython 3.9-3.14 wheels with cibuildwheel (auto64, skip
  PyPy/musllinux) on Linux/Windows/macOS x86_64+arm64, smoke-tests each wheel,
  builds an sdist, and publishes to TestPyPI then PyPI via OIDC trusted
  publishing on v* tags.
- ci.yml python_tests now builds the extension via pip and drops LIBDEDX_SO.

Also corrects the license metadata to GPL-3.0-or-later (#115).
- The sdist job's wheel-from-sdist check passed the tarball to `build --wheel`,
  which only accepts a directory ("is not a directory"). Use plain `build`,
  which builds the sdist and then a wheel from that sdist, proving the sdist is
  self-contained. Only the sdist is uploaded.
- Add a top-level `permissions: contents: read` block so the build jobs no
  longer run with the default broad GITHUB_TOKEN scope (CodeQL alerts 10/11).
  The publish jobs keep their explicit id-token: write.
… dest

Copilot review feedback on the nanobind binding:
- Config.set_elements_id() now drops the per-element arrays (atoms /
  mass_fraction / i_value) when the compound length changes, so stale buffers
  can no longer be shorter than elements_length and read out of bounds in
  dedx_load_config().
- check_length() now requires elements_id to be set first and enforces an exact
  length match for the dependent arrays, closing the "set before id" gap.
- python/CMakeLists.txt lists RUNTIME alongside LIBRARY for the _core install so
  the destination is correct regardless of how a generator classifies the
  Python extension module.

Adds a test covering the new ordering/length guards.
The `Wheels on macos-13` job sat queued for 25+ minutes (Intel macOS runners
are being wound down and have very limited capacity), while `macos-14` (arm64)
started immediately. Build both macOS architectures on the arm64 runner instead
— arm64 natively and x86_64 via cross-compilation (cibuildwheel archs) — and
drop macos-13 from the matrix. Intel-Mac wheels are still produced.
The PR ships a Python package, but the top-level docs were still C-only. Add:
- README.md: a "Python binding" section (pip install + quick start).
- index.rst (rendered in the Sphinx docs): a "Python binding" section covering
  the nanobind/scikit-build-core extension and the API surface.
- CONTRIBUTING.md: developer notes on building/testing the binding, the C++
  conventions for the binding source, dedx_config array ownership, version
  lockstep, and the inherited thread-safety limitation.
@grzanka
grzanka force-pushed the claude/issue-131-C9Tuo branch from f398e80 to 8094912 Compare June 16, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build the Python binding with nanobind + scikit-build-core and ship bundled wheels (binding-tech decision; refines #117)

4 participants