Hello, I am using the Jetson AGX Orin with the JetPack 5 DP (Developer Preview) But I am facing a problem with CUDA when running trtexec to create an engin.
trtexec ERROR :
[E] Error[1]: [executionResources.cpp::setTacticSources::159] Error Code 1: Cudnn (Could not initialize cudnn, please check cudnn installation.)
[E] Error[2]: [builder.cpp::buildSerializedNetwork::620] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[E] Engine could not be created from network
[E] Building engine failed
[E] Failed to create engine from model or file.
[E] Engine set up failed
So I checked if CUDA is installed by running the command below (after add to the PATH):
$ nvcc -V
and the result is this :
However when running the command below:
$ jetson_release -v
I receive that CUDA is not installed (look below):
I tested as well CUDA and ran some samples here and it worked perfectly BUT the samples with nvrtc didn’t work for me.
So I ran $ sudo /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2 to see the cuDNN but nothing was found.
And I tried to look at /usr/include/cudnn.h and this what I found (see below):
What should I do? Because for me CUDA is installed and run perfectly without “nvrtc”. However, it is not detected by the JetPack wich may cause this problem in trtexec.
We recommend you to please check the Support matrix doc and make sure you’re using the correct combination of TebsorRT, CUDA, and cuDNN versions. If you use older or advanced versions may face incompatibility issues.
Moving this post to Jetson AGX Orin forum to get better help on the setup-related instructions.
According to this link here, I saw that we must use for the JetPack 5 DP:
cuDNN 8.3.2
CUDA 11.4.4
tensorRT 8.4.0 EA.
For me, I have:
cuDNN is 8.4.0.27
CUDA 11.4.4
tensorRT 8.4.0.11
So I have the right versions of CUDA and TensorRT, for cuDNN I have a higher version.
However, the problem is that CUDA is not seen from my Jetson even if I have the right version.
NOTE: I installed the JetPack from the package manager see here and I don’t want to re-install the JetPack as suggested here because I don’t have an ubuntu machine.
For example when I try to run trtexec --onnx=model.onnx as suggested here (after putting the right path to the onnx model). Before It was working normaly but now it will show me theses errors below same as I put in the beginning of this topic.
I tried to use sudo apt-get -y install nvidia-cudnn8=5.0.1-b118 but I have an error and instead of installing CUDA 11.4 and cuDNN 8.3.2, it install CUDA 11.6 and cuDNN 8.4.0.27
Did you attempt to install the dependencies via the same way ?
sudo apt-get install libcudnn8=8.3.2.49-1+cuda11.4
Let me know how this works?
Or you can flash the fresh image with all components through sdk manager as @carolyuu suggested. I installed everything through sdk manager, but have faced the same issue that you are facing while installing them on non-jetson devices and I did manage to resolve them. Also always follow/refer support matrix for CUDA, cudnn and tensorrt to get the version right. Thanks
Yes it works after installing the dependencies and downgrading using this command sudo apt-get install libcudnn8=8.3.2.49-1+cuda11.4 that you suggested.
trtexec is working for me but still my Jetson Orin can’t see CUDA but for me it is not a problem because trtexec is working now
@Harry-S Great that it is working. Yes it doesn’t show over there for me as well, still it has to be fixed from Nvidia’s side hopefully it is fixed in the future jetpack versions. But CUDA will get detected for the applications you use.