Skip to content

fix(contrib): call torch.cuda.is_available() in FP16_Optimizer guard - #2021

Open
andrewwhitecdw wants to merge 1 commit into
NVIDIA:masterfrom
andrewwhitecdw:sweep/bugfix-cuda-availability-check
Open

fix(contrib): call torch.cuda.is_available() in FP16_Optimizer guard#2021
andrewwhitecdw wants to merge 1 commit into
NVIDIA:masterfrom
andrewwhitecdw:sweep/bugfix-cuda-availability-check

Conversation

@andrewwhitecdw

Copy link
Copy Markdown

Summary

apex.contrib.optimizers.FP16_Optimizer.__init__ is supposed to fail fast with a clear SystemError("Cannot use fp16 without CUDA.") on machines without CUDA. The guard never fires, so on a CPU-only machine (or a build without the CUDA extensions) construction proceeds and blows up later with a confusing RuntimeError: FP16_Optimizer requires cuda extensions from deep in the initializer.

Root cause

if not torch.cuda.is_available:
    raise SystemError("Cannot use fp16 without CUDA.")

torch.cuda.is_available (without parentheses) is the bound method object, which is always truthy, so not torch.cuda.is_available is always False and the raise is dead code.

Fix

Call the function:

if not torch.cuda.is_available():
    raise SystemError("Cannot use fp16 without CUDA.")

One-line change; behavior on CUDA machines is unchanged (the guard still does not fire there).

Testing

On a CPU-only machine with an ephemeral env (uv run --with torch), using torch.optim.SGD as the wrapped optimizer:

  • With the fix: FP16_Optimizer(opt, static_loss_scale=128.0) raises SystemError: Cannot use fp16 without CUDA. immediately — the intended behavior.
  • With the fix stashed (unmodified master, 6424da3): the same call skips the guard and instead raises RuntimeError: FP16_Optimizer requires cuda extensions from line 64, confirming the guard was dead code.

⚠️ The CUDA-dependent test suites under tests/ (L0/L1) require a GPU and the compiled amp_C extensions, so they could not be run locally; this change does not touch any CUDA code path, but please rely on CI for the full suite.

Why existing tests missed it

The optimizer tests run exclusively on CUDA machines, where the guard is a no-op either way; nothing exercises the CPU-only error path.

@crcrpar crcrpar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good but I'm afraid I don't think the test is necessary

@andrewwhitecdw
andrewwhitecdw force-pushed the sweep/bugfix-cuda-availability-check branch 2 times, most recently from 72d5d57 to acb771b Compare July 31, 2026 14:45
Remove the unnecessary regression test and apply pre-commit auto-fixes.
@andrewwhitecdw
andrewwhitecdw force-pushed the sweep/bugfix-cuda-availability-check branch from c5c6b72 to 40c0f0e Compare July 31, 2026 14:58
@andrewwhitecdw

Copy link
Copy Markdown
Author

Closing per adversarial audit: branch contains off-scope or not-a-bug changes.

@andrewwhitecdw
andrewwhitecdw deleted the sweep/bugfix-cuda-availability-check branch August 2, 2026 14:03
@andrewwhitecdw
andrewwhitecdw restored the sweep/bugfix-cuda-availability-check branch August 17, 2026 20:32
@andrewwhitecdw

Copy link
Copy Markdown
Author

script closed on accident

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants