Setting up CUDA/TensorRT manually on Jetson AGX Xavier

sam.jenkins · February 28, 2019, 8:29pm

When I try to install Cuda/TensorRT/OpenCV through the Jetpack 4.1 installer it doesn’t actually install any of those packages.

This is almost exactly the problem I have but its for the TX2: https://devtalk.nvidia.com/default/topic/1001520/jetson-tx2/setting-up-cuda-manually-on-tx2/

After asking me for the ssh password of nvidia@ its says copying cuda-repo-l4t-10-0-local-10.0.117_0-1_arm64.deb from the jetpack_download directory, then it asks for the nvidia@ ssh password again, and after that it asks again and just hangs, I’ve let it run for 2 hours with no change, and when i quit and check the Xavier nothing is installed.

And yes ssh is correctly set and I’ve confirmed that it works

Any Ideas?
Thanks
Sam

dusty_nv · March 1, 2019, 3:35am

Hi Sam, before installing the packages to Jetson, JetPack downloads the packages on the host to the jetpack_downloads/ directory. You can manually copy the CUDA, cuDNN, and TensorRT packages from this folder to your Jetson. You should then be able to install the packages on your Jetson similar to how the desktop versions of the packages are installed:

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu-installation

https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#installlinux-deb

https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html#installing-debian

Alternatively, if you try installing the packages again with JetPack, it may work the second time around. Make sure you can ping your Jetson from your host PC first (and ideally test logging in with SSH from the host). In the JetPack Component Manager, you can select just the packages you wish to install again (CUDA, cuDNN, and TensorRT for the target) and it will not have to re-flash the board.

suryadi · May 27, 2019, 6:20pm

Dear all,

Is it necessary that I install CUDA 10.0, cuDNN, TensorRT 5.0 on my host ubuntu 18.04 amd 64 with GTX1080? I thought that Nvidia SDK Manager helps me to install all tools at my host and Xavier.

I will try to install nv-tensorrt-repo-ubuntu1804-cuda10.0-trt5.1.5.0-ga-20190427_1-1_amd64.deb at my ubuntu host tomorrow.

I still confuse which program should run on what platform.

Thank you very much in advance.

Warmest Regards,
suryadi

Andrey1984 · June 27, 2019, 3:09pm

with sdkmanager where to get cuda packages for manual installation?
reference https://developer.nvidia.com/cuda-toolkit-65#linux-arm
located: /sdkm_downloads

sudo dpkg -i *
(Reading database ... 149017 files and directories currently installed.)
Preparing to unpack cuda-repo-l4t-10-0-local-10.0.166_1.0-1_arm64.deb ...
Unpacking cuda-repo-l4t-10-0-local-10.0.166 (1.0-1) over (1.0-1) ...
Preparing to unpack libcudnn7_7.3.1.28-1+cuda10.0_arm64.deb ...
Unpacking libcudnn7 (7.3.1.28-1+cuda10.0) over (7.3.1.28-1+cuda10.0) ...
Preparing to unpack libcudnn7-dev_7.3.1.28-1+cuda10.0_arm64.deb ...
update-alternatives: removing manually selected alternative - switching libcudnn to auto mode
Unpacking libcudnn7-dev (7.3.1.28-1+cuda10.0) over (7.3.1.28-1+cuda10.0) ...
Preparing to unpack libcudnn7-doc_7.3.1.28-1+cuda10.0_arm64.deb ...
Unpacking libcudnn7-doc (7.3.1.28-1+cuda10.0) over (7.3.1.28-1+cuda10.0) ...
Setting up cuda-repo-l4t-10-0-local-10.0.166 (1.0-1) ...
Setting up libcudnn7 (7.3.1.28-1+cuda10.0) ...
Setting up libcudnn7-dev (7.3.1.28-1+cuda10.0) ...
update-alternatives: using /usr/include/aarch64-linux-gnu/cudnn_v7.h to provide /usr/include/cudnn.h (libcudnn) in auto mode
Setting up libcudnn7-doc (7.3.1.28-1+cuda10.0) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...

sudo apt-key add /var/cuda-repo-10-0-local-10.0.166/7fa2af80.pub

sudo apt update
Get:1 file:/var/cuda-repo-10-0-local-10.0.166  InRelease
Ign:1 file:/var/cuda-repo-10-0-local-10.0.166  InRelease
Get:2 file:/var/cuda-repo-10-0-local-10.0.166  Release [574 B]
Get:2 file:/var/cuda-repo-10-0-local-10.0.166  Release [574 B]
Get:3 file:/var/cuda-repo-10-0-local-10.0.166  Release.gpg [819 B]
Get:3 file:/var/cuda-repo-10-0-local-10.0.166  Release.gpg [819 B]
Get:4 file:/var/cuda-repo-10-0-local-10.0.166  Packages [10.9 kB]
Hit:5 http://ports.ubuntu.com/ubuntu-ports bionic InRelease
Get:6 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease [88.7 kB]
Hit:7 http://ports.ubuntu.com/ubuntu-ports bionic-backports InRelease
Get:8 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease [88.7 kB]
Get:9 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 Packages [528 kB]
Get:10 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main Translation-en [247 kB]
Get:11 http://ports.ubuntu.com/ubuntu-ports bionic-updates/universe arm64 Packages [863 kB]
Get:12 http://ports.ubuntu.com/ubuntu-ports bionic-updates/universe Translation-en [285 kB]
Fetched 2,102 kB in 4s (516 kB/s)                                 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
4 packages can be upgraded. Run 'apt list --upgradable' to see them.

sudo apt-get install cuda
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package cuda

seems resolved with

sudo apt-get install cuda-toolkit-10-0

clemrom · November 18, 2019, 3:24am

It seems that my installation of the latest jetPack 4.2 did not get CUDA and CuDNN installed.

I understand that it seems that I should should use sdkmanager to get CUDA/CuDNN installed however it wants to reflash the entire Xavier board to do it. I don’t want to reflash.

I just want to be able to installed CUDA/CuDNN after the OS has been installed?

How can I do that?

kayccc · November 28, 2019, 5:10am

Hi clemrom,

No, you still need to upgrade the device to use the new version of CUDA/CuDNn due to driver dependency in low layer.

elisabetta.senore1 · February 11, 2020, 5:36pm

dusty_nv:

Hi Sam, before installing the packages to Jetson, JetPack downloads the packages on the host to the jetpack_downloads/ directory. You can manually copy the CUDA, cuDNN, and TensorRT packages from this folder to your Jetson. You should then be able to install the packages on your Jetson similar to how the desktop versions of the packages are installed:

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu-installation

https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#installlinux-deb

https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html#installing-debian

Alternatively, if you try installing the packages again with JetPack, it may work the second time around. Make sure you can ping your Jetson from your host PC first (and ideally test logging in with SSH from the host). In the JetPack Component Manager, you can select just the packages you wish to install again (CUDA, cuDNN, and TensorRT for the target) and it will not have to re-flash the board.

Hi dusty,
I’m trying to install jetpack on the Jetson AGX Xavier without flashing it.
I downloaded the cuda, cuDNN and tensorRT packages with the sdkmanager.
The board has already cuda and cuDNN but I have some issues with them, so I want to figure out if reinstalling them will solve the problem.

Do I have to uninstall them first? Or I can follow the commands in the links you posted anyway?

dusty_nv · February 11, 2020, 6:23pm

You probably want to uninstall the packages first if you are having issues with them.

Also, SDK Manager can install CUDA/cuDNN/TensorRT without reflashing. You can de-select the flashing step and just choose to install the CUDA/cuDNN/TensorRT components.

themozel · June 15, 2020, 12:22pm

Hi
I want to install the Cuda manually but there are many different versions of them in the sdkm_downloads dir. Which one should I use for my Xavier ?

todd.fisher · November 1, 2020, 9:17pm

dafaq - you buy a nvidia dev box and you’d expect it have already installed all the dev packages for cuda :(

I found this which is very helpful for getting things going for opencv and cuda… How to install OpenCV 4.5 with CUDA 11.2 in Ubuntu 20.04 · GitHub

todd.fisher · November 1, 2020, 11:38pm

Also you need to be aware of this issue:

github.com/opencv/opencv

Build Opencv 4.4.0 with Cuda support: make error: undefined reference to

opened 08:28PM - 12 Aug 20 UTC

closed 02:30PM - 13 Aug 20 UTC

maky-hnou

question (invalid tracker) category: build/install category: gpu/cuda (contrib)

##### System information (version) - OpenCV == 4.4.0 - Operating System / Plat…form == Ubuntu 20.04 - Compiler == gcc-8 - GPU: GTX1650 - Cuda: 10.1, V10.1.243 - CuDNN: 7.6.5 - Compute capability: 7.5 - Cuda emplacement: `/usr/lib/cuda/` ##### Detailed description I wanted to build OpenCV from source with Cuda support. For that: - I downloaded both opencv-4.4.0.zip and opencv_contrib-4.4.0.zip. - Unzipped both files and renamed them opencv and opencv_contrib, respectively. - created a folder named `build` in opencv and entered it then run the command: ``` cmake -D CMAKE_BUILD_TYPE=RELEASE \ -D CMAKE_INSTALL_PREFIX=/usr/local \ -D INSTALL_PYTHON_EXAMPLES=ON \ -D INSTALL_C_EXAMPLES=OFF \ -D OPENCV_ENABLE_NONFREE=ON \ -D WITH_CUDA=ON \ -D WITH_CUDNN=ON \ -D OPENCV_DNN_CUDA=ON \ -D ENABLE_FAST_MATH=1 \ -D CUDA_FAST_MATH=1 \ -D CUDA_ARCH_BIN=7.5 \ -D WITH_CUBLAS=1 \ -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules/ \ -D HAVE_opencv_python3=ON \ -D PYTHON_EXECUTABLE=~/.virtualenvs/cv/bin/python \ -D CUDA_ARCH_PTX="" \ -D WITH_LIBV4L=ON \ -D WITH_V4L=ON \ -D WITH_GSTREAMER=ON \ -D WITH_QT=ON \ -D WITH_OPENGL=ON \ -D BUILD_opencv_python2=OFF \ -D CUDNN_INCLUDE_DIR=/usr/lib/cuda/include \ -D CUDNN_LIBRARY=/usr/lib/cuda/lib64/libcudnn_static.a \ -D CUDNN_VERSION=7.6.5 \ -D CMAKE_C_COMPILER=/usr/bin/gcc-8 \ ../ ``` Here is the output of the cmake command: [cmake_output.txt](https://github.com/opencv/opencv/files/5065409/cmake_output.txt) I guess everything went right. - After that, I run `make -j12`, which got me some errors: ``` [ 81%] Building CXX object modules/dnn_superres/CMakeFiles/opencv_perf_dnn_superres.dir/perf/perf_dnn_superres.cpp.o /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosInitializeCriticalSection' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosLeaveCriticalSection' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosProcessId' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosLoadLibrary' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosOnce' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosGetProcAddress' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosInit' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosFreeLibrary' /usr/bin/ld: ../../lib/libopencv_dnn.so.4.4.0: undefined reference to `culibosEnterCriticalSection' collect2: error: ld returned 1 exit status make[2]: *** [modules/dnn/CMakeFiles/opencv_perf_dnn.dir/build.make:190: bin/opencv_perf_dnn] Error 1 make[1]: *** [CMakeFiles/Makefile2:5831: modules/dnn/CMakeFiles/opencv_perf_dnn.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... ``` Here is the full output of the make command: [make_output.txt](https://github.com/opencv/opencv/files/5065438/make_output.txt) - After that, the make command stopped at: ``` [ 86%] Built target opencv_test_gapi make: *** [Makefile:163: all] Error 2 ``` **Did anyone face the same problem?** **How to solve this problem?** ##### Issue submission checklist - [x] I report the issue, and I ask how to solve it  - [x] I checked the problem with documentation, FAQ, open issues, answers.opencv.org, Stack Overflow, etc and have not found solution  - [x] I updated to latest OpenCV version and the issue is still there  - [x] There is reproducer code and related data files: videos, images, onnx, etc

shopebay777 · January 5, 2021, 10:12am

hi @kayccc I have Jetson nano running JetPack 4.2 LT 32.1.
Using sdkmanager I have downloaded .deb packages of TensorRT,CUDNN,Cuda10.2 for Jetpack 4.4.1

I want to install these deb packages directly on Jetson nano running Jetpack4.2, so that I can get latest version of TensorRT and Cuda on the board.

It is not feasible for me to reflash the board as it is a location that is physically not reachable.
Do you anticipate any issues in my plan?
I have 500 such boards on JP4.2 which need latest TensorRt 7.1.3

kayccc · January 5, 2021, 11:04pm

As I mentioned previously, each JatPack has it’s own corresponding low layer driver to work with CUDA, CuDNN and TensorRT…SDKs, so that will be many unknown issues if you do it in that way, and we can’t support to resolve it.