wrong install path: TensorRT 5.0 RC

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

The CentOS rpms install into /usr/X, e.g., /usr/lib64, /usr/include
This is wrong, it should be /usr/local/cuda/X when installed together w/ Cuda 10.

Please fix for final release.

2

Hello,

Not sure I understand. Standalone CUDA10 installs to /usr/local/cuda-10.0.

Are you saying the TRT header files in /usr/include to be in /usr/local/cuda-10.0/include ?
and TRT libraries in /usr/lib64 should be in /usr/local/cuda-10.0/lib64 ?

Are you installing multiple TRTs (example TRT 5 for cuda 10, TRT 5 for cuda 9) on the same machine?

3

If you have specific path requirement, please consider the tar file installation method.

Installation Guide :: NVIDIA Deep Learning TensorRT Documentation

4

When installing CUDA 10 (CentOS rpm) and then
“TensorRT 5.0.0.10 RC for CentOS/RedHat 7 and CUDA 10.0 RPM local repo packages” (rpm)
then CUDA 10 and CUDNN/TensorRT do not mesh:

cd /usr/src/cudnn_samples_v7/conv_sample
make

CUDA VERSION: 10000
SMS: 30 35 50 53 60 61 70 72
g++ -I/usr/local/cuda/include -o conv_sample.o -c conv_sample.cpp
conv_sample.cpp: In function ‘int doConv(cudnnHandle_t, T_ELEM*, T_ELEM*, T_ELEM*, T_ELEM*, T_ELEM*, T_ELEM*, cudnnTensorDescriptor_t, cudnnFilterDescriptor_t, cudnnTensorDescriptor_t, cudnnConvolutionDescriptor_t, float, float, cudnnTensorFormat_t, cudnnDataType_t, const int*, const int*, const int*, const int*, const int*, const int*, const int*, const int*, int)’:
conv_sample.cpp:671:36: error: ‘CUDNN_DATA_INT8x32’ was not declared in this scope
} else if (dataType == CUDNN_DATA_INT8x32) {//resizeFactor = 32
^

etc.

The problem is that all NVIDIA sample codes search for include and lib64 paths here:

Location of the CUDA Toolkit

CUDA_PATH ?= /usr/local/cuda

In fact, these are the 1st two lines of /usr/src/cudnn_samples_v7/conv_sample/Makefile

Now, you may blame the author of this sample code for CUDNN for writing such a Makefile. But note that all of the CUDA sample codes are written the same way. Besides, earlier CUDNN installations (including sample codes) also assumed /usr/local/cuda as a common prefix.

For me, I get a mix of /usr/local/cuda with older paths (e.g., include/cudnn.h from v6), which is not updated to cudnn.h from v7 in this place, but rather /usr/lib64. So when you Makefiles search /usr/local/cuda 1st, they find the old version – without CUDNN_DATA_INT8x32.

I cannot fix this w/ the rpm since it is non-relocatable!!!

Suggestions:

  • quick fix: change rpm (and other deb etc. packages) to install to /usr/include/cuda
    . if you can, make them relocatable
  • longer-term fix: have the installer check for the cuda location and ensure that rpms use the same path

Sure, I can install from tar.gz. But 5.0 is a release candidate.
This is my feedback of what needs to be fixed for the final release, IMHO.

5

And as an aside: Yes, I do have CUDNN 5.0 and 6.0 installed as well. One is needed by an earlier Tensorflow-based software stack, the other by another package. So it’s important that these new versions mesh with the older ones (and be internally consistent, i.e., conv_sample → uses /usr/local/cuda, so this should also be the install path and the first prefix in LD_LIBRARY_PATH etc.)

6

Hello,

TRT5 RC examples look in CUDA_INSTALL_DIR env variable first, if undefined, then defaults to /usr/local/cuda.

[root@volta3202 sampleMNIST]# make
../Makefile.config:5: CUDA_INSTALL_DIR variable is not specified, using /usr/local/cuda by default, use CUDA_INSTALL_DIR=<cuda_directory> to change.
../Makefile.config:8: CUDNN_INSTALL_DIR variable is not specified, using $CUDA_INSTALL_DIR by default, use CUDNN_INSTALL_DIR=<cudnn_directory> to change.
:

Will defining CUDA_INSTALL_DIR and CUDNN_INSTALL_DIR address your concerns?

7

I don’t see your Makefile.config in the rpm, maybe you’re referencing the tar ball?

  1. CUDA_INSTALL_DIR=/usr/local/cuda and CUDNN_INSTALL_DIR=/usr will not change anything for /usr/src/cudnn_samples_v7/conv_sample, make will result in the same problems as before

  2. CUDA_INSTALL_DIR/CUDNN_INSTALL_DIR will be undefined when root installs from rpm

  3. The /usr/src/tensorrt/sample examples work fine, with or without the said variables defined, since it finds

8

Never mind, I thought Nvidia wanted feedback on their RC. Apparently not so. Consider it not solved.