TensorFlow on Drive PX2

Hi

We are using TensorFlow to develop some of our deep learning applications on Drive PX2.

During the training process we get following error message:

2018-11-08 11:25:52.005486: E tensorflow/stream_executor/cuda/cuda_dnn.cc:343] Loaded runtime CuDNN library: 7.0.4 but source was compiled with: 7.1.5. CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.

Where can I find the right documentation to update CuDNN or installation of Tensorflow on Drive PX2?

Kind Regards,
Arun

Dear ArunRajappan,

Did you update DrivePX2 SDK to the latest version?


Drive OS 5.0.10.3 has cuDNN 7.1 library. Thanks.

Ok, Thanks. I will update it.

Arun

Hi Steve,

error-target-configure

Bootburn operation unsuccessful

The following tips can help troubleshoot bootburn failures. These are based on
known issues related to setup or missing configuration. If it doesn’t help
resolve the issue, contact module owners with a copy of the full log.

error= error-target-configure

  • Make sure target is in recovery-mode

bootburn flashing failed! error code = 28

Where can I find the information on flashing the DRIVE PX2?
Best,
Arun

Dear ArunRajappan,

Please refer to below link.
https://docs.nvidia.com/drive/active/5.0.10.3L/nvvib_docs/index.html#page/NVIDIA%20DRIVE%20Linux%20SDK%20Development%20Guide%2FFlashing%2Fflash_dpx.html%23

BTW did you flash the DPX2 using SDK manager? Thanks.

Okay thanks.

How to use TensorRT in DRIVE PX2 ?
vbutorin@innoteamnvidia:~/prj/tensorflow$ ./configureorflow/
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command “bazel shutdown”.
You have bazel 0.15.0- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python]:

Found possible Python library paths:
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages]

Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]:
jemalloc as malloc support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]:
Google Cloud Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Hadoop File System support? [Y/n]:
Hadoop File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Amazon AWS Platform support? [Y/n]:
Amazon AWS Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Apache Kafka Platform support? [Y/n]: n
No Apache Kafka Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [y/N]:
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GDR support? [y/N]:
No GDR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with VERBS support? [y/N]:
No VERBS support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 9.0]: 9.2

Please specify the location where CUDA 9.2 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: /usr/local/cuda-9.2

Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.1

Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-9.2]:/usr/lib/aarch64-linux-gnu/

Do you wish to build TensorFlow with TensorRT support? [y/N]: y
TensorRT support will be enabled for TensorFlow.

Please specify the location where TensorRT is installed. [Default is /usr/lib/aarch64-linux-gnu]:

Please specify the NCCL version you want to use. If NCCL 2.2 is not installed, then you can use version 1.3 that can be fetched automatically but it may have worse performance with multiple GPUs. [Default is 2.2]:

Please specify the location where NCCL 2 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda-9.2]:

Invalid path to NCCL 2 toolkit, /usr/local/cuda-9.2/lib/libnccl.so.2 or /usr/local/cuda-9.2/include/nccl.h not found. Please use the O/S agnostic package of NCCL 2
Please specify the NCCL version you want to use. If NCCL 2.2 is not installed, then you can use version 1.3 that can be fetched automatically but it may have worse performance with multiple GPUs. [Default is 2.2]: 1.3

Could you direct me to documentation ?

Dear ArunRajappan,

Please see /usr/share/doc/tensorrt/ in DPX2 for tensorrt documentation. Thanks.