Tensorrt looking for wrong cuda

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I am trying to build tensorrt, but it is looking for a version of Cuda that is not on my machine:
~/TensorRT/build$ make
[ 2%] Built target third_party.protobuf
[ 26%] Built target nvinfer_plugin_static
[ 51%] Built target nvinfer_plugin
[ 51%] Built target caffe_proto
[ 57%] Built target nvcaffeparser_static
[ 63%] Built target nvcaffeparser
[ 64%] Built target gen_onnx_proto
[ 65%] Built target onnx_proto
[ 67%] Built target nvonnxparser_static
[ 67%] Linking CXX executable getSupportedAPITest
/usr/bin/ld: warning: libcublas.so.10, needed by /usr/lib/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libcudart.so.10.2, needed by /usr/lib/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libnvrtc.so.10.2, needed by /usr/lib/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaGetDevice' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaDeviceSynchronize’
/usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaPeekAtLastError' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcCreateProgram@libnvrtc.so.10.2’
etc…

looking at /usr/local, we see that 10.2 is not there:
~/TensorRT/build$ ls /usr/local
bin cuda cuda-10.0 cuda-11.2 doc etc games include lib man sbin share src

Environment

TensorRT Version: 8
GPU Type: titan rtx
Nvidia Driver Version:
CUDA Version: 11.2
CUDNN Version:
Operating System + Version: ubuntu 20.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Steps To Reproduce

mkdir build
cd build
cmake …
make

  • Full traceback of errors encountered
    ~/TensorRT/build$ make
    [ 2%] Built target third_party.protobuf
    [ 26%] Built target nvinfer_plugin_static
    [ 51%] Built target nvinfer_plugin
    [ 51%] Built target caffe_proto
    [ 57%] Built target nvcaffeparser_static
    [ 63%] Built target nvcaffeparser
    [ 64%] Built target gen_onnx_proto
    [ 65%] Built target onnx_proto
    [ 67%] Built target nvonnxparser_static
    [ 67%] Linking CXX executable getSupportedAPITest
    /usr/bin/ld: warning: libcublas.so.10, needed by /usr/lib/libnvinfer.so, not found (try using -rpath or -rpath-link)
    /usr/bin/ld: warning: libcudart.so.10.2, needed by /usr/lib/libnvinfer.so, not found (try using -rpath or -rpath-link)
    /usr/bin/ld: warning: libnvrtc.so.10.2, needed by /usr/lib/libnvinfer.so, not found (try using -rpath or -rpath-link)
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaGetDevice' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaDeviceSynchronize’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaPeekAtLastError' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcCreateProgram@libnvrtc.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasGetMathMode@libcublas.so.10' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to __cudaRegisterFunction@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to __cudaUnregisterFatBinary@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaFreeHost’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaMemcpy2DAsync@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasSetMathMode’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasSetStream_v2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcAddNameExpression@libnvrtc.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to __cudaPushCallConfiguration' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasGetVersion_v2@libcublas.so.10’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaGetErrorName' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaEventDestroy’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaCreateTextureObject@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaDeviceGetAttribute’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaDestroyTextureObject@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaHostAlloc’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaEventCreateWithFlags@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to __cudaUnregisterFatBinary’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaFuncGetAttributes' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaPeekAtLastError@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaEventCreateWithFlags' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to __cudaRegisterFatBinary’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasSetStream_v2@libcublas.so.10' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasGemmEx@libcublas.so.10’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcDestroyProgram@libnvrtc.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasHgemm@libcublas.so.10’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcGetPTXSize@libnvrtc.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaGetErrorString@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to __cudaPopCallConfiguration@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to __cudaRegisterVar@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaRuntimeGetVersion@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaGetDeviceProperties@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaGetLastError' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaStreamCreateWithPriority’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaMalloc' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaMemcpy’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to __cudaRegisterFatBinary@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to __cudaRegisterVar’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaFree@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasHgemmStridedBatched@libcublas.so.10’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcCompileProgram@libnvrtc.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaMemcpyAsync’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaStreamDestroy@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaGetDevice@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaStreamWaitEvent@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaEventDestroy@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaEventSynchronize@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaMallocHost’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaStreamWaitEvent' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaStreamCreate’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaGetDeviceProperties' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasScopy_v2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasSgemm_v2@libcublas.so.10' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaDeviceGetAttribute@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaGetDeviceCount@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcGetProgramLog@libnvrtc.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaStreamCreate@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaEventElapsedTime@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaEventRecord@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaDeviceSynchronize@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasCreate_v2@libcublas.so.10' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to __cudaRegisterFatBinaryEnd’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaOccupancyMaxActiveBlocksPerMultiprocessorWithFlags' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to __cudaPopCallConfiguration’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to __cudaRegisterFatBinaryEnd@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaStreamSynchronize’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasGemmEx' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcGetLoweredName@libnvrtc.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaLaunchKernel@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaMemcpyToSymbol@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasDestroy_v2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaStreamAddCallback@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to nvrtcVersion@libnvrtc.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaLaunchKernel’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaFuncSetAttribute@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaGetErrorString’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaMemsetAsync@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaStreamDestroy’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaStreamQuery@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasDestroy_v2@libcublas.so.10’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaMemsetAsync' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasSgemmEx@libcublas.so.10’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasSscal_v2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaMemGetInfo@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmyelin.so: undefined reference to cublasGetProperty@libcublas.so.10' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaMemcpy@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaDeviceReset' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmyelin.so: undefined reference to cublasLoggerConfigure@libcublas.so.10’
    /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmyelin.so: undefined reference to nvrtcGetErrorString@libnvrtc.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to __cudaRegisterFunction’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasSasum_v2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaMalloc@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcGetPTX@libnvrtc.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cublasCreate_v2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaStreamSynchronize@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to nvrtcGetProgramLogSize@libnvrtc.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaFree' /usr/bin/ld: /usr/lib/libnvinfer_plugin.so: undefined reference to cudaEventRecord’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaCreateChannelDesc@libcudart.so.10.2' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to __cudaPushCallConfiguration@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasSgemmStridedBatched@libcublas.so.10' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaMemcpyAsync@libcudart.so.10.2’
    /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cublasSetMathMode@libcublas.so.10' /usr/bin/ld: /usr/lib/libnvinfer.so: undefined reference to cudaGetLastError@libcudart.so.10.2’
    collect2: error: ld returned 1 exit status
    make[2]: *** [parsers/onnx/CMakeFiles/getSupportedAPITest.dir/build.make:106: parsers/onnx/getSupportedAPITest] Error 1
    make[1]: *** [CMakeFiles/Makefile2:1148: parsers/onnx/CMakeFiles/getSupportedAPITest.dir/all] Error 2
    make: *** [Makefile:152: all] Error 2
5

Hi @iviti,

Looks like you’ve installed different package of TensorRT built for different CUDA version. We recommend you to reinstall correct package suitable for your environment. And also please make sure CUDA is installed correctly.

For your reference, similar old git issue,
https://github.com/NVIDIA/TensorRT/issues/227

Please refer support matrix doc for dependencies and follow the installation guide.

Thank you.

6

So I tried going to
https://developer.nvidia.com/nvidia-tensorrt-8x-download
and downloading tensorrt for cuda 11.1, 11.2,11.3 since I have 11.2 installed. But when I try to install it that way, it complains that I dont have libs that depend on 11.3:

~/Downloads$ sudo apt-get install tensorrt
Reading package lists… Done
Building dependency tree
Reading state information… Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
tensorrt : Depends: libnvinfer8 (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvinfer-plugin8 (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvparsers8 (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvonnxparsers8 (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvinfer-bin (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvinfer-dev (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvinfer-plugin-dev (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvparsers-dev (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvonnxparsers-dev (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvinfer-samples (= 8.0.0-1+cuda11.3) but it is not going to be installed
Depends: libnvinfer-doc (= 8.0.0-1+cuda11.3) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

7

Hi @iviti,

We recommend you to please follow installation guide to get correct installation steps.

Thank you.

8

I was,
I got to step 4.1 where I downloaded the deb file for tensorrt for cuda 11.1,2,3
then I tried to install it as per step 4.2:
os=“ubuntuxx04” tag=“cudax.x-trt8.x.x.x-ea-yyyymmdd”
sudo dpkg -i nv-tensorrt-repo-${os}-${tag}_1-1_amd64.deb
sudo apt-key add /var/nv-tensorrt-repo-${tag}/7fa2af80.pub
sudo apt-get update
sudo apt-get install tensorrt
which was where I got stuck (see error above)

9

@iviti,

Could you please provide more details, which deb package(which CUDA version) particularly are you trying to install. Please make sure your installed CUDA version matches with deb package supported version.

If you find libraries with multiple CUDA versions, we recommend you to install CUDA correctly and try to install TensorRT. If you need assistance in CUDA installation, please post your concern here to get better help.

Thank you.

10

I clicked on :
TensorRT 8.0 EA for Ubuntu 20.04 and CUDA 11.1, 11.2 & 11.3 DEB local repo package

which downloads:
nv-tensorrt-repo-ubuntu2004-cuda11.3-trt8.0.0.3-ea-20210423_1-1_amd64.deb

I have:
$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Feb_14_21:12:58_PST_2021
Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0

11

@iviti,

With CUDA 11.3 we have better backwards/forwards compatibility. We recommend you to please upgrade CUDA to 11.3 and install TensorRT package you’ve downloaded.