Skip to content

Fix layernorm scale bias 31147 - #31672

Open
Nash6643 wants to merge 2 commits into
microsoft:mainfrom
Nash6643:fix-layernorm-scale-bias-31147
Open

Fix layernorm scale bias 31147#31672
Nash6643 wants to merge 2 commits into
microsoft:mainfrom
Nash6643:fix-layernorm-scale-bias-31147

Conversation

@Nash6643

@Nash6643 Nash6643 commented Aug 5, 2026

Copy link
Copy Markdown

Description

This PR fixes an issue in LayerNormFusion and SimplifiedLayerNormFusion where 1D tensors (or tensors matching the reduction axes rank) caused the scale and bias selection loops to misidentify intermediate node outputs (Div or Mul) as the scale or bias.

Specifically:

  • Added input_def != div_output check when selecting the scale operand.
  • Added input_def != mul_output check when selecting the bias operand.
  • Added a 1D regression unit test (DivAndScaleSameRank1D_RegressionTest) in graph_transform_test.cc.

Motivation and Context

Fixes #31147

When input tensors are rank 1, Div output and Mul output also have rank 1. The previous scale/bias matching loops only checked rank dimension size against axes_values.size(), causing the optimizer to select the Div node output as scale (and Mul output as bias). When the fusion replaced these nodes, the fused graph contained dangling/invalid node input references, failing session creation. This change ensures internal graph node outputs are explicitly skipped during operand resolution.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@Nash6643

Nash6643 commented Aug 5, 2026

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LayerNorm fusion selects the Div output as scale, failing session creation even with keepdims=1

1 participant