Skip to content

Add float, boolean and array indexing support to Java Variant extraction - #23711

Open
nartal1 wants to merge 1 commit into
NVIDIA:mainfrom
nartal1:support-float-bool
Open

Add float, boolean and array indexing support to Java Variant extraction#23711
nartal1 wants to merge 1 commit into
NVIDIA:mainfrom
nartal1:support-float-bool

Conversation

@nartal1

@nartal1 nartal1 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Description

This PR contributes to NVIDIA/cudf-spark#15222 and NVIDIA/cudf-spark#15223.

It brings VariantUtils in line with Variant functionality subsequently merged into libcudf:

In this PR:

  • added DType.FLOAT32, DType.FLOAT64, and DType.BOOL8 to the supported Variant target types
  • documented the exact physical-type matching behavior for Variant decoding
  • documented array-index path syntax such as $[0] and $.a[1].b
  • documented null behavior for missing fields, out-of-bounds indices, container mismatches, encoded Variant nulls, and physical-type mismatches
  • expanded VariantUtilsTest coverage for the newly supported types and path behavior

Variant decoding requires the encoded physical type to exactly match the requested target type. Numeric conversions are not performed. For example, a FLOAT32 Variant value cannot be extracted as FLOAT64; the result is null.

Array Indexing

Variant extraction paths now document and test zero-based array indexing:

  • root array indexing: $[0]
  • mixed object and array traversal: $.a[1].b
  • raw extraction followed by casting
  • combined extraction and casting

Testing

Expanded VariantUtilsTest coverage includes:

  • direct and combined extraction of FLOAT32 and FLOAT64
  • float width mismatches and truncated payloads
  • direct and combined extraction of boolean true and false values
  • encoded Variant nulls, input nulls, and parent struct nulls
  • empty float and boolean inputs
  • root and nested array indexing
  • out-of-bounds array indices and container mismatches
  • sliced input columns
  • null argument and malformed input validation

Validation results:

  • VariantUtilsTest: 32 tests, 0 failures, 0 errors

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@nartal1
nartal1 requested a review from a team August 18, 2026 20:20
@nartal1 nartal1 self-assigned this Aug 18, 2026
@nartal1
nartal1 requested review from tgravescs and removed request for a team August 18, 2026 20:20
@nartal1 nartal1 added the 2 - In Progress Currently a work in progress label Aug 18, 2026
@copy-pr-bot

copy-pr-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the Java Affects Java cuDF API. label Aug 18, 2026
@nartal1 nartal1 added non-breaking Non-breaking change Spark Functionality that helps Spark RAPIDS feature request New feature or request labels Aug 18, 2026
@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b767e1a4-e483-49d9-96eb-0515cc450705

📥 Commits

Reviewing files that changed from the base of the PR and between c563e26 and e7645f8.

📒 Files selected for processing (2)
  • java/src/main/java/ai/rapids/cudf/VariantUtils.java
  • java/src/test/java/ai/rapids/cudf/VariantUtilsTest.java

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added support for extracting FLOAT32, FLOAT64, and BOOL8 values from variants.
    • Expanded JSONPath-style extraction to support array indices and nested object-array paths.
    • Added clearer handling for null, empty, invalid, and unsupported values.
  • Bug Fixes

    • Prevented incorrect numeric conversions by requiring exact physical-type matches.
    • Improved handling of invalid array indexes, truncated data, mismatched types, and sliced inputs.

Walkthrough

VariantUtils now documents and tests FLOAT32, FLOAT64, BOOL8, array-index paths, exact physical-type matching, null handling, and invalid inputs.

Changes

Variant extraction support

Layer / File(s) Summary
Target types, paths, and encoded fixtures
java/src/main/java/ai/rapids/cudf/VariantUtils.java, java/src/test/java/ai/rapids/cudf/VariantUtilsTest.java
The supported target types and extraction paths are documented. Tests add encoders and fixtures for arrays, floating-point values, booleans, and nulls.
Extraction and casting validation
java/src/test/java/ai/rapids/cudf/VariantUtilsTest.java
Tests cover nested array paths, scalar casting, sliced and empty inputs, type mismatches, invalid arguments, and truncated float payloads.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to e7645

This PR adds float, boolean, and array-path Variant extraction with expanded tests; no actionable merge-blocking risk remains.

Suggested reviewers: tgravescs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 7.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main changes: float, boolean, and array-index support for Java Variant extraction.
Description check ✅ Passed The description directly explains the supported types, array indexing, documentation updates, and expanded tests.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@nartal1
nartal1 requested a review from a team August 18, 2026 20:24
@nartal1

nartal1 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test e7645f8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2 - In Progress Currently a work in progress feature request New feature or request Java Affects Java cuDF API. non-breaking Non-breaking change Spark Functionality that helps Spark RAPIDS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant