Title: LoadLibrary Error 126 When Using ONNX Runtime-GPU with CUDA on Windows Server 2022

Accelerated Computing CUDA CUDA Setup and Installation
1

System Information:

  • Operating System: Windows Server 2022
  • Python Version: 3.10
  • ONNX Runtime Version: 1.12.0
  • CUDA Toolkit Version: 11.4
  • cuDNN Version: Compatible version for CUDA 11.4
  • NVIDIA Driver Version: 470
  • GPU Model: NVIDIA Quadro K6000

Issue Description:

I am facing an issue while trying to use the ONNX Runtime with GPU (onnxruntime-gpu) on my Windows Server 2022 setup. Specifically, I receive the following error when attempting to load the onnxruntime_providers_cuda.dll:

vbnet

Copy code

[E:onnxruntime:Default, provider_bridge_ort.cc:1992 onnxruntime::TryGetProviderInfo_CUDA] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1637 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

The error seems to occur when the ONNX Runtime tries to load the CUDA provider. I’ve made sure that CUDA 11.4 and cuDNN (compatible with CUDA 11.4) are installed, but I cannot pinpoint the exact issue.

Steps Taken to Troubleshoot:

  1. CUDA and cuDNN Installation:
  • Installed CUDA Toolkit 11.4 and the compatible version of cuDNN, ensuring compatibility with ONNX Runtime 1.12.0.
  • Verified that CUDA and cuDNN paths are added to the system’s PATH environment variable.
  1. NVIDIA Driver:
  • Confirmed that the NVIDIA drivers (version 470) are up to date and support CUDA 11.4 using the nvidia-smi command.
  1. ONNX Runtime Reinstallation:
  • Uninstalled and reinstalled the onnxruntime-gpu package using pip to ensure the proper version is installed.
  1. File Verification:
  • Verified that the file onnxruntime_providers_cuda.dll exists in the path mentioned in the error log.
  • Used a tool to check for unresolved dependencies within the onnxruntime_providers_cuda.dll.

Despite these steps, the issue persists, and I am unable to use the GPU (Quadro K6000) with ONNX Runtime 1.12.0.

Questions:

  1. Is there a known issue with LoadLibrary Error 126 when loading CUDA-related DLLs in this environment?
  2. Could there be specific configuration or dependency issues with my current setup?
  3. Are there additional troubleshooting steps that I should follow to resolve this issue?

Any guidance or suggestions would be greatly appreciated.

Thank you for your time and assistance!

1 Like
2

This strongly hints at a missing library. A good way to debug this on linux would be calling ldd on each library and checking if its dependencies exist. For windows I usually recommend using this tool. Just start it, drop in onnxruntime_providers_cuda.dll and it will highlight missing dependencies in red. Hope that gets you closer!