Register nvidia-cublas-cu12/nvidia-cudnn-cu12 DLL dirs on Windows import - #2098
Open
si-kui-a wants to merge 2 commits into
Open
Register nvidia-cublas-cu12/nvidia-cudnn-cu12 DLL dirs on Windows import#2098si-kui-a wants to merge 2 commits into
si-kui-a wants to merge 2 commits into
Conversation
The Python wheel does not bundle cuBLAS/cuDNN (see CONTRIBUTING.md,
"CUDA support in Python wheels") -- they are loaded at runtime instead.
cuBLAS specifically is loaded with a plain LoadLibraryA() call
(src/cuda/cublas_stub.cc), which does not consult directories
registered via os.add_dll_directory() -- verified empirically that
add_dll_directory() alone still leaves LoadLibraryA("cublas64_12.dll")
failing; only adding the same directory to PATH (part of the legacy
search order LoadLibraryA does use) makes it resolve.
Users who `pip install nvidia-cublas-cu12 nvidia-cudnn-cu12` for CUDA
execution (the standard pip-only setup also documented by
faster-whisper) hit "Could not locate cublas64_12.dll" because that
package's DLL directory is never registered by either mechanism --
reproduced and fixed here on ctranslate2==4.8.1 with a real GPU.
OpenNMT#1915 and OpenNMT#1826 report the same missing-registration gap for cuDNN
specifically. Both predate PR OpenNMT#1949 (2025-12-30), which stopped
linking cuDNN into the official Windows wheel by default (CMake's
WITH_CUDNN now defaults off) -- so their reports were real against the
wheel current at the time, but I could not reproduce that side against
today's release, which no longer needs cuDNN at all for this workflow.
cuDNN registration is kept anyway: harmless where unreachable, correct
for anyone still on a cuDNN-linked build.
Registers both os.add_dll_directory() (for any other loader in the
process that does honor it, same as the existing ROCm SDK handling
above) and PATH (for cuBLAS's own plain LoadLibraryA() call).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reviewed-by: si-kui-a
si-kui-a
force-pushed
the
fix/windows-nvidia-pip-dll-directory
branch
from
September 9, 2026 14:09
4262ff9 to
06d30b2
Compare
Refactors the nvidia.cublas/nvidia.cudnn DLL-directory registration loop (added earlier in this branch to fix OpenNMT#1915/OpenNMT#1826) out of the inline, platform-guarded module body and into a standalone function, _register_nvidia_pip_dll_directories(). Behavior on Windows is unchanged -- it's still only called once, in the same place, when sys.platform == "win32" -- but the function itself no longer depends on being imported on Windows, so it can be exercised directly from a regular pytest run on any host OS by injecting a fake import_module and monkeypatching os.add_dll_directory/os.environ, without a real Windows+CUDA machine. Adds python/tests/test_nvidia_dll_directories.py (4 cases): both nvidia packages present and registered in order, a missing package is skipped without error, a present package whose bin/ subdirectory doesn't exist is skipped, and PATH still gets updated even when os.add_dll_directory() itself raises OSError. Verified by installing the official ctranslate2 4.8.2 wheel into a clean venv, overwriting its __init__.py with this branch's version, and running the new tests against that real install: all 4 pass. Confirmed the tests are not vacuous by deliberately deleting the PATH-update line and re-running -- 2 of the 4 tests failed as expected, then restored and re-confirmed all 4 pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Reviewed-by: si-kui-a
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
python/ctranslate2/__init__.pydoesn't point Windows at cuBLAS/cuDNNfor users who
pip install nvidia-cublas-cu12 nvidia-cudnn-cu12(thestandard pip-only CUDA setup, also used by faster-whisper):
bindirectories.os.add_dll_directory()(used for the existingROCm handling above) isn't enough for cuBLAS: it's loaded via a
plain
LoadLibraryA()call (src/cuda/cublas_stub.cc), whichignores
AddDllDirectory-registered paths -- only loaders thatrequest
LOAD_LIBRARY_SEARCH_DEFAULT_DIRShonor those (e.g.ctypes.CDLL()).#1915 and #1826 report the same missing-registration problem for
cuDNN (Windows/Linux respectively) -- see Verification for why I
couldn't reproduce that side against the current release.
Reproduction (cuBLAS)
Windows, RTX 4060,
ctranslate2==4.8.1(official wheel),pip install nvidia-cublas-cu12 nvidia-cudnn-cu12, no system CUDA toolkit on PATH:Passes with this patch. An
os.add_dll_directory()-only version (noPATH) still fails the same way.Fix
For each of
nvidia.cublas/nvidia.cudnn, if installed, registerits
bindirectory with bothos.add_dll_directory()(parity withthe ROCm handling) and
PATH(whatLoadLibraryA()actually uses).No change for CPU-only / system-CUDA users.
A C++-level fix -- switching
cublas_stub.cc'sLoadLibraryA()toLoadLibraryExW(..., LOAD_LIBRARY_SEARCH_DEFAULT_DIRS)-- would makeos.add_dll_directory()sufficient on its own and might be the morecorrect long-term fix. I left that out: it's a change to the core
loading path, which is outside what CONTRIBUTING.md asks non-core
contributors to touch, and I have no build environment here to verify
it. This PR's PATH-based workaround is self-contained on the Python
side and doesn't rule that out later.
Verification
master, passes with this patch.encode()andalign()(MedianFilter, per Fix process-killing integer division by zero when Whisper align() gets a window with no frames #2065) still succeeded. Why: CMake'sWITH_CUDNNdefaultsOFF, and off meansconv1d_gpu.cu(pureCUDA) is built instead of
conv1d_cudnn_gpu.cu-- the only file inthe repo still calling
cudnnConvolutionForward. The Windows wheelbuild script passed
-DWITH_CUDNN=ONuntil PR Conv1d pure CUDA implementation #1949 (2025-12-30)flipped it off. Both libcudnn_cnn problem #1826 (2024-12-01) and Could not locate cudnn_cnn_infer64_8.dll. Please make sure it is in your library path! #1915 (2025-08-27)
predate that -- real bugs against the wheel current at the time.
Today's 4.8.x wheel just doesn't link cuDNN, so there's nothing here
for me to reproduce. Kept the registration anyway: harmless for
current users, and correct for anyone still on a cuDNN-linked build.
black/isort/flake8clean (this repo'ssetup.cfg, CI-pinnedversions).
nvidia.cuda_runtime(try to add rpath nvidia python #1811's Linux rpath list alsocovers it): CONTRIBUTING.md says cudart is statically linked, and
the reproduction above passes without it.
either, and I have no
_extbuild environment here.AI assistance disclosure
Written with AI assistance (Claude Code), per CONTRIBUTING.md. I hit
this on my own faster-whisper setup and already carry an equivalent
add_dll_directory()+PATHworkaround, which is what caught myfirst
add_dll_directory()-only attempt not actually working. Everyclaim above was run or traced against the actual source/history. This
diff and description were reviewed by me before submission. I own
this change.
Co-authored with Claude Code (Anthropic); reviewed by @si-kui-a before submission.