Draft
jit: skip expensive XLA graph passes when no clusters are compiled#3
Conversation
…sary overhead when no XLA clusters exist - EncapsulateSubgraphsPass: Skip expensive CPU device/FLR creation and graph copy when no nodes have kXlaClusterAttr (i.e., no XLA clusters were created by MarkForCompilationPass). This eliminates significant per-graph-construction overhead when XLA is enabled but no ops are actually compiled. - MarkForCompilation: After running the analysis, mark the source node with kXlaAlreadyClustered so that subsequent calls on the same graph (e.g., from PartitionedCall re-running the optimization pipeline) return early in O(1) time instead of scanning all O(n) nodes without finding an early exit. - Updated tests to reflect new behavior: kXlaHasReferenceVarsAttr is only set when XLA clusters exist (not on all nodes unconditionally). Added NoXlaClustersEarlyExit test to validate the optimization. Agent-Logs-Url: https://github.com/fenxcc/tensorflow/sessions/24af514f-b99e-4428-a563-f4d9c64241ca Co-authored-by: fenxcc <64964597+fenxcc@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Investigate performance degradation with XLA enabled on armv9.2-a
jit: skip expensive XLA graph passes when no clusters are compiled
Mar 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enabling XLA on CPU causes 2–5% throughput regression even when zero subgraphs are compiled (and >5% with small subgraphs), due to unconditional overhead in graph optimization passes that runs regardless of whether any ops are actually marked for XLA compilation.
Changes
EncapsulateSubgraphsPass: early exit when no XLA clusters existThe pass previously performed expensive setup work unconditionally:
Devicecreation viaDeviceFactoryProcessFunctionLibraryRuntimeconstructionEncapsulateSubgraphsInFunctionsNow the pass checks for
kXlaClusterAttrduring the existing TPU-skip scan and returns early if no nodes were marked for compilation:MarkForCompilation: O(1) early-exit on repeat invocationsAfter
MarkForCompilationPassImpl::Run()completes, the source node (ID 0, always first ingraph->nodes()iteration) is marked withkXlaAlreadyClustered. Previously, for graphs where no clusters were created, the existing early-exit scan found nothing and forced a full O(n) re-analysis on every subsequent invocation (e.g. fromPartitionedCallre-running the optimization pipeline). The source-node marker converts that to O(1).Test updates (
encapsulate_subgraphs_pass_test.cc)RefVariablesMarked: updated to assertkXlaHasReferenceVarsAttris not set when no clusters exist (attribute is only meaningful onXlaLaunch/_XlaCompilenodes)NoRefVarsNoAttr: adds explicitkXlaClusterAttrso the full encapsulation + ref-var analysis path is exercisedNoXlaClustersEarlyExit: directly validates the early-exit optimization