Add hybrid scan multifile two-step materializers - #22833
Conversation
Co-authored-by: Yunsong Wang <12716979+PointKernel@users.noreply.github.com>
Co-authored-by: Yunsong Wang <12716979+PointKernel@users.noreply.github.com>
…id-scan-multifile-row-mask
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp`:
- Around line 174-212: The construct_row_group_passes function is a public API
wrapper that is missing the CUDF_FUNC_RANGE() macro required by coding
guidelines. Add CUDF_FUNC_RANGE() as the first statement in the function body
before the line calculating total_row_groups using std::accumulate, ensuring it
comes before any other function logic to properly instrument the public API
delegation to _impl.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a18d206e-75d7-4707-807f-53f1e46f8a52
📒 Files selected for processing (2)
cpp/include/cudf/io/experimental/hybrid_scan_multifile.hppcpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp`:
- Around line 174-212: The construct_row_group_passes function is a public API
wrapper that is missing the CUDF_FUNC_RANGE() macro required by coding
guidelines. Add CUDF_FUNC_RANGE() as the first statement in the function body
before the line calculating total_row_groups using std::accumulate, ensuring it
comes before any other function logic to properly instrument the public API
delegation to _impl.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a18d206e-75d7-4707-807f-53f1e46f8a52
📒 Files selected for processing (2)
cpp/include/cudf/io/experimental/hybrid_scan_multifile.hppcpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp
🛑 Comments failed to post (1)
cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp (1)
174-212: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Add
CUDF_FUNC_RANGE()to the new public API wrapper.
construct_row_group_passesis a public API incpp/srcthat delegates to_impl, but it skipsCUDF_FUNC_RANGE(), unlike adjacent public wrappers.As per coding guidelines, public functions in
cpp/src/**/*.{cu,cpp}should includeCUDF_FUNC_RANGE()before delegating to detail implementations.Suggested patch
std::vector<std::vector<std::vector<size_type>>> hybrid_scan_multifile::construct_row_group_passes( cudf::host_span<std::vector<size_type> const> row_group_indices, std::size_t pass_read_limit) const { + CUDF_FUNC_RANGE(); + auto const total_row_groups = std::accumulate(row_group_indices.begin(), row_group_indices.end(), std::size_t{0}, [](auto sum, auto const& rgs) { return sum + rgs.size(); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cpp/src/io/parquet/experimental/hybrid_scan_multifile.cpp` around lines 174 - 212, The construct_row_group_passes function is a public API wrapper that is missing the CUDF_FUNC_RANGE() macro required by coding guidelines. Add CUDF_FUNC_RANGE() as the first statement in the function body before the line calculating total_row_groups using std::accumulate, ensuring it comes before any other function logic to properly instrument the public API delegation to _impl.Source: Coding guidelines
|
/merge |
| ConfigureTest( | ||
| HYBRID_SCAN_TEST | ||
| io/experimental/hybrid_scan_composer.cpp | ||
| io/experimental/hybrid_scan_multifile_composer.cpp |
There was a problem hiding this comment.
Please sort these files alphabetically.
Description
Contributes to #22583
This PR adds two-step column materialization APIs for the multifile hybrid scan reader
Checklist