Skip to content

Add basic slicing support to pylibcudf's gpumemoryview - #23541

Merged
rapids-bot[bot] merged 5 commits into
NVIDIA:mainfrom
Matt711:fea/pylibcudf/gpumemoryview-slicing
Aug 5, 2026
Merged

Add basic slicing support to pylibcudf's gpumemoryview#23541
rapids-bot[bot] merged 5 commits into
NVIDIA:mainfrom
Matt711:fea/pylibcudf/gpumemoryview-slicing

Conversation

@Matt711

@Matt711 Matt711 commented Aug 4, 2026

Copy link
Copy Markdown
Member

Description

Adds support for byte slicing pylibcudf gpumemoryviews. Used in #23317 where we make a single device allocation for all byte ranges and then slice it into sub-views when passing them to hybrid scan APIs.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@Matt711 Matt711 added feature request New feature or request non-breaking Non-breaking change labels Aug 4, 2026
@Matt711
Matt711 marked this pull request as ready for review August 4, 2026 20:58
@Matt711
Matt711 requested a review from a team as a code owner August 4, 2026 20:58
@NVIDIA NVIDIA deleted a comment from copy-pr-bot Bot Aug 4, 2026
@github-actions github-actions Bot added Python Affects Python cuDF API. pylibcudf Issues specific to the pylibcudf package labels Aug 4, 2026
@Matt711 Matt711 changed the title Add basic slicing support to gpumemoryview Add basic slicing support to pylibcudf's gpumemoryview Aug 4, 2026
@GPUtester GPUtester moved this to In Progress in cuDF Python Aug 4, 2026
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9ccfd6ad-f341-43ea-9620-8ff56dcee400

📥 Commits

Reviewing files that changed from the base of the PR and between 2f06c8e and 9e65664.

📒 Files selected for processing (4)
  • python/pylibcudf/pylibcudf/column.pxd
  • python/pylibcudf/pylibcudf/gpumemoryview.pxd
  • python/pylibcudf/pylibcudf/gpumemoryview.pyx
  • python/pylibcudf/tests/test_gpumemoryview.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • python/pylibcudf/tests/test_gpumemoryview.py
  • python/pylibcudf/pylibcudf/gpumemoryview.pyx

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added byte-range slicing for GPU memory views, returning bounded child views.
    • Supports valid ranges, clamped boundaries, and empty slices.
    • Sliced views remain usable after the original view is released.
  • Bug Fixes

    • Improved length reporting for derived GPU memory views.
    • Added validation for unsupported indexing and non-unit slice steps.
  • Tests

    • Added coverage for valid slices, invalid inputs, empty results, and view lifetime.

Walkthrough

Adds byte_slice to gpumemoryview. Slices create parent-backed views with adjusted CUDA pointers and lengths. The implementation validates slice inputs, handles empty ranges, and preserves parent lifetime. The declaration and tests cover the new behavior.

Changes

GPU memory view slicing

Layer / File(s) Summary
Slice contract and view construction
python/pylibcudf/pylibcudf/gpumemoryview.pyi, python/pylibcudf/pylibcudf/gpumemoryview.pyx, python/pylibcudf/pylibcudf/gpumemoryview.pxd, python/pylibcudf/pylibcudf/column.pxd
The public type declares byte_slice. _slice creates parent-retaining raw-byte views, and __len__ reads the stored CUDA array interface shape. Extension types support weak references.
Slice validation and lifetime coverage
python/pylibcudf/pylibcudf/gpumemoryview.pyx, python/pylibcudf/tests/test_gpumemoryview.py
byte_slice validates slice inputs and unit steps, clamps ranges, and handles empty views. Tests cover valid slices, invalid inputs, and parent retention.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: brandon-b-miller, mroeschke, vyasr

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the addition of basic slicing support for pylibcudf's gpumemoryview.
Description check ✅ Passed The description explains the byte-slicing change and its intended use for creating sub-views from a single device allocation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
python/pylibcudf/tests/test_gpumemoryview.py (1)

77-80: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Test the parent-lifetime contract.

test_slice keeps gv alive until the child view is consumed. It cannot detect a regression where _slice stops assigning v.obj = parent. Create a child from a temporary parent, release the parent references, force collection, and then consume the child.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/pylibcudf/tests/test_gpumemoryview.py` around lines 77 - 80, Update
test_slice to create the sliced child from a temporary parent, remove all parent
references, and force garbage collection before calling to_pylist. Ensure the
test still compares the child contents with the expected np_array byte slice,
thereby validating that the child retains its parent through the _slice lifetime
contract.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/pylibcudf/pylibcudf/gpumemoryview.pyx`:
- Around line 98-109: Add a complete public-method docstring to
gpumemoryview.__getitem__ describing the slice parameter and byte-based bounds,
the step=1 requirement, the returned |u1 view and parent retention, and the
TypeError and ValueError cases. Keep the existing slicing behavior unchanged.
- Around line 96-109: Update __len__ to return self.nbytes so its unit matches
the byte offsets used by __getitem__ and _slice. Preserve the existing slice
validation and byte-view behavior, and add a regression test using a non-u1
dtype that verifies length and boundary slicing are byte-based.

---

Nitpick comments:
In `@python/pylibcudf/tests/test_gpumemoryview.py`:
- Around line 77-80: Update test_slice to create the sliced child from a
temporary parent, remove all parent references, and force garbage collection
before calling to_pylist. Ensure the test still compares the child contents with
the expected np_array byte slice, thereby validating that the child retains its
parent through the _slice lifetime contract.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2793d433-4e09-4474-9d21-891928080960

📥 Commits

Reviewing files that changed from the base of the PR and between 969575e and e5698d4.

📒 Files selected for processing (3)
  • python/pylibcudf/pylibcudf/gpumemoryview.pyi
  • python/pylibcudf/pylibcudf/gpumemoryview.pyx
  • python/pylibcudf/tests/test_gpumemoryview.py

Comment thread python/pylibcudf/pylibcudf/gpumemoryview.pyx
Comment thread python/pylibcudf/pylibcudf/gpumemoryview.pyx Outdated
@Matt711
Matt711 marked this pull request as draft August 4, 2026 21:55
@copy-pr-bot

copy-pr-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@Matt711
Matt711 force-pushed the fea/pylibcudf/gpumemoryview-slicing branch from e5698d4 to e29dffb Compare August 4, 2026 22:19
@Matt711

Matt711 commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

/ok to test 2f06c8e

@Matt711
Matt711 marked this pull request as ready for review August 5, 2026 01:27
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
python/pylibcudf/pylibcudf/gpumemoryview.pyx (2)

99-117: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Complete the byte_slice docstring.

The Raises section documents only TypeError. Document that only unit steps are supported, that non-unit or zero steps raise ValueError, and that out-of-range or reversed ranges produce zero-length views.

Proposed documentation update
         gpumemoryview
             A ``|u1`` view of the requested byte range. The returned view
             holds a reference to the parent buffer, keeping it alive.
 
         Raises
         ------
         TypeError
             If ``s`` is not a slice.
+        ValueError
+            If the slice step is not 1.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/pylibcudf/pylibcudf/gpumemoryview.pyx` around lines 99 - 117, Complete
the docstring for byte_slice by documenting that only unit slice steps are
supported, with non-unit or zero steps raising ValueError, and that out-of-range
or reversed ranges return zero-length views.

Source: Coding guidelines


12-19: 📐 Maintainability & Code Quality | 🔵 Trivial

Rerun the Cython build and focused tests.

This .pyx file changed. Rerun the repository's standard Python/Cython build and python/pylibcudf/tests/test_gpumemoryview.py.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/pylibcudf/pylibcudf/gpumemoryview.pyx` around lines 12 - 19, Rerun the
repository’s standard Python/Cython build after the `_slice` change, then
execute the focused `python/pylibcudf/tests/test_gpumemoryview.py` test suite
and address any failures.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/pylibcudf/pylibcudf/gpumemoryview.pyx`:
- Around line 12-18: Validate that the parent passed to _slice is C-contiguous
before constructing and publishing the raw byte view. Reject byte slicing for
non-contiguous gpumemoryview instances, using the existing CUDA Array Interface
strides/layout information, while preserving the current contiguous |u1 view
behavior.
- Line 18: Update the CUDA Array Interface construction in gpumemoryview to
preserve the parent interface’s stream metadata: when the parent interface
includes stream, copy that value into the sliced interface, while leaving it
absent when the parent has no stream.
- Line 18: Update the `v.cai` construction in `gpumemoryview` to set the data
read-only flag from `parent.cai["data"][1]` instead of hard-coding `False`,
preserving the parent’s mutability for child views.

---

Nitpick comments:
In `@python/pylibcudf/pylibcudf/gpumemoryview.pyx`:
- Around line 99-117: Complete the docstring for byte_slice by documenting that
only unit slice steps are supported, with non-unit or zero steps raising
ValueError, and that out-of-range or reversed ranges return zero-length views.
- Around line 12-19: Rerun the repository’s standard Python/Cython build after
the `_slice` change, then execute the focused
`python/pylibcudf/tests/test_gpumemoryview.py` test suite and address any
failures.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0ea183dc-0944-4878-8176-ea54964c1b23

📥 Commits

Reviewing files that changed from the base of the PR and between 969575e and 2f06c8e.

📒 Files selected for processing (3)
  • python/pylibcudf/pylibcudf/gpumemoryview.pyi
  • python/pylibcudf/pylibcudf/gpumemoryview.pyx
  • python/pylibcudf/tests/test_gpumemoryview.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • python/pylibcudf/tests/test_gpumemoryview.py

Comment thread python/pylibcudf/pylibcudf/gpumemoryview.pyx Outdated
Comment thread python/pylibcudf/pylibcudf/gpumemoryview.pyx Outdated
Comment thread python/pylibcudf/tests/test_gpumemoryview.py
Comment thread python/pylibcudf/tests/test_gpumemoryview.py Outdated
@Matt711

Matt711 commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

/merge

@rapids-bot
rapids-bot Bot merged commit 1b2e313 into NVIDIA:main Aug 5, 2026
132 of 133 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in cuDF Python Aug 5, 2026
@Matt711
Matt711 deleted the fea/pylibcudf/gpumemoryview-slicing branch August 5, 2026 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Non-breaking change pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants