trtexec tries to load libcudart.so.10.1 even if it is installed with cuda10.0

I’m struggling with setting up TensorRT to work with tensorflow-gpu1.14

dpkg -l | grep TensorRT
ii  graphsurgeon-tf                                             5.1.2-1+cuda10.0                                              amd64        GraphSurgeon for TensorRT package
ii  libnvinfer-dev                                              5.1.2-1+cuda10.0                                              amd64        TensorRT development libraries and headers
ii  libnvinfer-samples                                          5.1.2-1+cuda10.0                                              all          TensorRT samples and documentation
ii  libnvinfer5                                                 5.1.2-1+cuda10.0                                              amd64        TensorRT runtime libraries
ii  python-libnvinfer                                           5.1.2-1+cuda10.0                                              amd64        Python bindings for TensorRT
ii  python-libnvinfer-dev                                       5.1.2-1+cuda10.0                                              amd64        Python development package for TensorRT
ii  python3-libnvinfer                                          5.1.2-1+cuda10.0                                              amd64        Python 3 bindings for TensorRT
ii  python3-libnvinfer-dev                                      5.1.2-1+cuda10.0                                              amd64        Python 3 development package for TensorRT
ii  tensorrt                                                    5.1.2.2-1+cuda10.0                                            amd64        Meta package of TensorRT
ii  uff-converter-tf                                            5.1.2-1+cuda10.0                                              amd64        UFF converter for TensorRT package

I have installed version 5.1.2-1+cuda10.0 (also tried from the latest version 6.0.1.5).
But when I run in terminal trtexec, I get the following error:

trtexec: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory

I have installed cuda-10.0 and all necessary dependencies.
When I run my tensorflow script, somewhere in log I get:

Running FP32 graph
INFO:tensorflow:Linked TensorRT version: (0, 0, 0)
INFO:tensorflow:Loaded TensorRT version: (0, 0, 0)
INFO:tensorflow:Running against TensorRT version 0.0.0
2019-10-04 13:25:21.803379: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-10-04 13:25:21.821876: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-10-04 13:25:21.822137: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1
2019-10-04 13:25:21.822180: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session
2019-10-04 13:25:21.822501: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-10-04 13:25:21.903012: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-10-04 13:25:21.903394: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x501dc90 executing computations on platform CUDA. Devices:
2019-10-04 13:25:21.903407: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce GTX 1080 Ti, Compute Capability 6.1
2019-10-04 13:25:21.924564: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3600000000 Hz
2019-10-04 13:25:21.925428: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x508dc70 executing computations on platform Host. Devices:
2019-10-04 13:25:21.925439: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2019-10-04 13:25:21.925590: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-10-04 13:25:21.925831: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: 
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.607
pciBusID: 0000:01:00.0

This really makes me crazy. Why that trtexec wants to link against cudart.so.10.1?

Hi tisma,

Instead of installing from source, please consider using our NVIDIA GPU Cloud (NGC) TensorRT docker container, which has minimal host-side dependencies - https://ngc.nvidia.com/catalog/containers/nvidia:tensorrt

Thanks,
NVIDIA Enterprise Support

I think it doens’t work for me. If I do that, how can I combine it with tensorflow if I want to execute tensorrt programmatically from code?
I see that there is also tensorflow docker image, but is there any image that has tensorflow AND tensorrt, or how can I create one?

Hi,

According to the release notes for the TF container, it also comes with TensorRT 6.0

https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/rel_19.09.html#rel_19.09

Please try using that image.

Thanks,
NVIDIA Enterprise Support

I’ve checked a docker image and when I ran that script it calls TensorRT correctly. Now I am wondering where those pip dependencies with these tags come from?

...
tensorboard==1.14.0+nv
tensorflow-gpu==1.14.0+nv
tensorrt==6.0.1.4
...

Is there any nvidia pip wheel repository for it?