Handle empty tensors in CUDA InstanceNormalization - #31647
Open
apsonawane wants to merge 1 commit into
Open
Conversation
Early-return success for empty input tensors in both generic and MLFloat16 CUDA paths to avoid host-side divide-by-zero in N!=1 branch. Add CUDA regression tests for zero-channel and zero-batch inputs.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the CUDA InstanceNormalization kernel to correctly handle valid ONNX cases where the input tensor has zero elements (any dimension is 0), ensuring it returns an empty output without invoking cuDNN paths that would otherwise hit divide-by-zero/invalid descriptor scenarios. It also adds CUDA-targeted unit tests to cover zero-channel and zero-batch edge cases.
Changes:
- Add early-return guards in CUDA
InstanceNorm<T>::ComputeInternal(includingMLFloat16specialization) forx_shape.Size() == 0after input validation. - Add CUDA-only tests to verify empty outputs are produced (and no error occurs) when
N==0orC==0.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| onnxruntime/core/providers/cuda/nn/instance_norm.cc | Early-return on empty input tensors (after ValidateInputs) to avoid invalid cuDNN/statistics paths and produce empty outputs. |
| onnxruntime/test/providers/cpu/nn/instance_norm_op_test.cc | Adds CUDA-scoped tests for empty channel and empty batch cases to validate correct empty-output behavior. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request improves the handling of empty input tensors in the CUDA implementation of the
InstanceNormalizationoperator and adds corresponding unit tests to ensure correct behavior. The main focus is to ensure compliance with the ONNX specification, which allows empty inputs and expects empty outputs.CUDA Implementation Improvements:
InstanceNorm<T>::ComputeInternalandInstanceNorm<MLFloat16>::ComputeInternalininstance_norm.ccto check for empty input tensors and return early, ensuring that empty inputs produce empty outputs as required by ONNX. This prevents unnecessary computation and potential errors when the input size is zero. [1] [2]Unit Test Additions:
instance_norm_op_test.cc:InstanceNormEmptyChannel_Cudaverifies correct handling when the channel dimension is zero.InstanceNormEmptyBatch_Cudaverifies correct handling when the batch dimension is zero.These tests confirm that the operator returns empty outputs without error for these edge cases.