Skip to content

Replace deprecated torch.cuda.amp API with torch.amp equivalents - #2025

Open
xyf5432 wants to merge 1 commit into
NVIDIA:masterfrom
xyf5432:fix/amp-deprecation
Open

Replace deprecated torch.cuda.amp API with torch.amp equivalents#2025
xyf5432 wants to merge 1 commit into
NVIDIA:masterfrom
xyf5432:fix/amp-deprecation

Conversation

@xyf5432

@xyf5432 xyf5432 commented Aug 20, 2026

Copy link
Copy Markdown

Fixes #2024

Summary

torch.cuda.amp has been deprecated since torch 2.3 (GradScaler) / 2.4 (autocast, custom_fwd, custom_bwd) and is scheduled for removal. This PR migrates the remaining usages to torch.amp, which has provided the same names since torch 2.3, and enforces the existing torch>=2.6.0 floor from requirements.txt inside setup.py.

Changes:

  • apex/_autocast_utils.pytorch.cuda.amp.autocast_mode._cast(args, dtype)torch.amp.autocast_mode._cast(args, "cuda", dtype). The torch.amp variant takes an explicit device_type since torch 2.5; the old two-arg form only forwards and warns "Please use torch.amp.autocast_mode._cast(value, 'cuda', dtype) instead".
  • apex/contrib/cudnn_gbn/batch_norm.pyfrom torch.amp import custom_bwd, custom_fwd; decorators now pass the required keyword-only device_type="cuda" (same style as the existing apex/contrib/conv_bias_relu/conv_bias_relu.py).
  • apex/contrib/optimizers/distributed_fused_adam.py — module-level aliases GradScaler = torch.amp.GradScaler and GradScalerOptState = torch.amp.grad_scaler.OptState (verified to be the identical objects behind the deprecated names), used by the grad_scaler type annotations, docstrings, and the per-method OptState assignments that are now removed.
  • apex/contrib/test/optimizers/test_distributed_fused_lamb.pyfrom torch.amp import GradScaler.
  • tests/L0/run_mlp/test_mlp.pytorch.amp.autocast("cuda", enabled=...).
  • setup.py — compile-time check now rejects torch < 2.6.0 (via packaging.parse, previously a dormant 0.4 check) and install_requires gains torch>=2.6.0, matching requirements.txt.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Validation

Verified on torch 2.11.0 (CPU build):

  • torch.amp.grad_scaler.OptState is torch.cuda.amp.grad_scaler.OptStateTrue, so the is comparisons in unscale_grads/step are unaffected.
  • New _cast path is warning-free under warnings.simplefilter("error", FutureWarning), while the old two-arg _cast raises under the same filter — the migration is effective and the filter is sensitive.
  • New and old _cast produce identical outputs for identical inputs.
  • custom_fwd(device_type="cuda") / custom_bwd(device_type="cuda") forward + backward run cleanly.
  • Module-level aliases resolve to the torch.amp objects; the GradScaler | None annotations evaluate.
  • setup.py version check passes 2.11.0 and rejects 1.13.1 (control group raises RuntimeError).
  • ruff format --check with the pinned 0.16.1: all touched files unchanged.
  • ruff check: identical error set before and after (27 → 27, only line-number shifts from the added module-level aliases) — no new lint issues.
  • py_compile passes on all touched files.

User impact

No API or behavior changes for users; the FutureWarnings emitted through torch.cuda.amp (and the risk of breaking on its removal) are eliminated.

Notes for reviewers

  • torch.amp has provided the same names since torch 2.3, and requirements.txt already declares torch>=2.6.0, so the migration introduces no minimum-version regression.
  • The three-argument _cast(value, device_type, dtype) signature exists since torch 2.5; 2.3/2.4 are outside the declared floor.

torch.cuda.amp autocast/custom_fwd/custom_bwd are deprecated since torch
2.4 and GradScaler since 2.3, all scheduled for removal. Migrate the
remaining uses (autocast_mode._cast, custom_fwd/custom_bwd, GradScaler,
grad_scaler.OptState, autocast context) to torch.amp, which has provided
the same names since torch 2.3. Also enforce the torch>=2.6.0 floor
declared in requirements.txt inside setup.py, replacing the dormant 0.4
check.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

torch.cuda.amp still used in 5 files (incl. private autocast_mode._cast / grad_scaler.OptState) — deprecated since torch 2.3/2.4, no guards

1 participant