Error "Could not load dynamic library" for Tensorflow 2.1.0 with Cuda 10.1

I am trying to run a GPU test script on a remote Linux server with many Cuda version installed. I’ve recently just installed CUDA 10.1 and I get the following error listed below when I try to run the script.

CUDNN 7.6 has also been installed.

python3.5 GPU\ TEST.py

2020-03-31 12:34:47.677201: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory

2020-03-31 12:34:47.678932: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory

2020-03-31 12:34:47.679031: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

WARNING:tensorflow:From GPU TEST.py:14: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.

Instructions for updating:

Use `tf.config.list_physical_devices('GPU')` instead.

2020-03-31 12:35:27.536858: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA

2020-03-31 12:35:27.556895: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2399750000 Hz

2020-03-31 12:35:27.559980: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5565d66e2060 initialized for platform Host (this does not guarantee that XLA will be used). Devices:

2020-03-31 12:35:27.560029: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version

2020-03-31 12:35:27.568513: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1

2020-03-31 12:35:27.865813: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5565d67a4e10 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:

2020-03-31 12:35:27.865997: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce GTX 980, Compute Capability 5.2

2020-03-31 12:35:27.866032: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (1): GeForce GTX 980, Compute Capability 5.2

2020-03-31 12:35:27.869303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:

pciBusID: 0000:03:00.0 name: GeForce GTX 980 computeCapability: 5.2

coreClock: 1.2785GHz coreCount: 16 deviceMemorySize: 3.95GiB deviceMemoryBandwidth: 208.91GiB/s

2020-03-31 12:35:27.870730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 1 with properties:

pciBusID: 0000:83:00.0 name: GeForce GTX 980 computeCapability: 5.2

coreClock: 1.2785GHz coreCount: 16 deviceMemorySize: 3.95GiB deviceMemoryBandwidth: 208.91GiB/s

2020-03-31 12:35:27.890891: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory

2020-03-31 12:35:27.910644: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10'; dlerror: libcublas.so.10: cannot open shared object file: No such file or directory

2020-03-31 12:35:27.930326: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10'; dlerror: libcufft.so.10: cannot open shared object file: No such file or directory

2020-03-31 12:35:27.950175: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10'; dlerror: libcurand.so.10: cannot open shared object file: No such file or directory

2020-03-31 12:35:27.970323: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10'; dlerror: libcusolver.so.10: cannot open shared object file: No such file or directory

2020-03-31 12:35:28.026738: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10'; dlerror: libcusparse.so.10: cannot open shared object file: No such file or directory

2020-03-31 12:35:28.137395: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7

2020-03-31 12:35:28.137655: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1592] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.

Skipping registering GPU devices...

2020-03-31 12:35:28.137988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:

2020-03-31 12:35:28.138039: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] 0 1

2020-03-31 12:35:28.138075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0: N N

2020-03-31 12:35:28.138120: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 1: N N

Why am I getting the ‘Could not load dynamic library’ for multiple libraries error?

It looks like your libraries are not found. You have problems with TensorRT libraries and CUDA libraries.
I guess you must enable the correct CUDA version on your Linux server.
Make LD_LIBRARY_PATH point to the correct libraries inside your script.

2 Likes