From a81e5683cc2e5c259bc1fc04c1dd937a47a34b9e Mon Sep 17 00:00:00 2001 From: Xinyi Cao Date: Mon, 24 Aug 2026 22:25:56 -0700 Subject: [PATCH] docs(troubleshooting): add Blackwell (sm_103) cuDNN SDPA failure and workaround On Blackwell-generation NVIDIA GPUs, PyTorch's default SDPA path can select the cuDNN backend, which has no valid plan for this architecture and fails with "cuDNN Frontend error: No valid execution plans built". With FLUX this hits the first attention block in the UNet, so no image is produced. Documents --use-split-cross-attention as the workaround, the cu128-or-newer build requirement, and that setting enable_cudnn_sdp(False) from a sitecustomize.py does not work. --- troubleshooting/overview.mdx | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/troubleshooting/overview.mdx b/troubleshooting/overview.mdx index 0cd704b92..c63a0c9d8 100644 --- a/troubleshooting/overview.mdx +++ b/troubleshooting/overview.mdx @@ -283,6 +283,31 @@ nvidia-smi nvidia-smi --query-gpu=driver_version --format=csv ``` +**Blackwell GPUs (`sm_103`): generation fails at the first attention layer:** + +On Blackwell-generation NVIDIA GPUs, PyTorch's default scaled dot-product attention (SDPA) can +select the cuDNN backend, which has no valid plan for this architecture: + +``` +cuDNN Frontend error: No valid execution plans built +``` + +With FLUX this hits the very first attention block in the UNet, so no image is ever produced. + +```bash +# Use a hand-written attention implementation that bypasses SDPA and cuDNN +python main.py --use-split-cross-attention + +# Blackwell also requires a CUDA 12.8 or newer PyTorch build; +# builds compiled against cu121 will not run on these cards at all +``` + + +Disabling the cuDNN SDPA backend programmatically before startup — for example calling +`torch.backends.cuda.enable_cudnn_sdp(False)` from a `sitecustomize.py` — does not work, because +the flag is not retained through CUDA initialization. The launch flag is the reliable fix. + + ### AMD GPU Issues **ROCm support (Linux only):**