CUDA not installed in JetPack 5 DP

Hello, I am using the Jetson AGX Orin with the JetPack 5 DP (Developer Preview) But I am facing a problem with CUDA when running trtexec to create an engin.

trtexec ERROR :
[E] Error[1]: [executionResources.cpp::setTacticSources::159] Error Code 1: Cudnn (Could not initialize cudnn, please check cudnn installation.)
[E] Error[2]: [builder.cpp::buildSerializedNetwork::620] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[E] Engine could not be created from network
[E] Building engine failed
[E] Failed to create engine from model or file.
[E] Engine set up failed

So I checked if CUDA is installed by running the command below (after add to the PATH):

  • $ nvcc -V

and the result is this :

image

However when running the command below:

  • $ jetson_release -v

I receive that CUDA is not installed (look below):

I tested as well CUDA and ran some samples here and it worked perfectly BUT the samples with nvrtc didn’t work for me.
So I ran $ sudo /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2 to see the cuDNN but nothing was found.
And I tried to look at /usr/include/cudnn.h and this what I found (see below):

What should I do? Because for me CUDA is installed and run perfectly without “nvrtc”. However, it is not detected by the JetPack wich may cause this problem in trtexec.

Thanks for your help in advance.

Best regards,
Harry S.

Hi,

We recommend you to please check the Support matrix doc and make sure you’re using the correct combination of TebsorRT, CUDA, and cuDNN versions. If you use older or advanced versions may face incompatibility issues.

Moving this post to Jetson AGX Orin forum to get better help on the setup-related instructions.

Thank you.

Hi,
Thank you for you reply and helping me 😊.

According to this link here, I saw that we must use for the JetPack 5 DP:

  • cuDNN 8.3.2
  • CUDA 11.4.4
  • tensorRT 8.4.0 EA.

For me, I have:

  • cuDNN is 8.4.0.27
  • CUDA 11.4.4
  • tensorRT 8.4.0.11

So I have the right versions of CUDA and TensorRT, for cuDNN I have a higher version.
However, the problem is that CUDA is not seen from my Jetson even if I have the right version.

NOTE: I installed the JetPack from the package manager see here and I don’t want to re-install the JetPack as suggested here because I don’t have an ubuntu machine.

Thank you

Please don’t use the JetPack EA release now, and should use the JetPack 5.0.1 DP version for development.

Hello,

Yes indeed, I am using the JetPack 5 DP, I am sorry I did a mistake and said EA at the begining of this topic. I will correct it.

But I am using the JetPack 5 DP and I updated using the package manager see here

Hi Harry-S,

Please run below command and check again:

$ sudo apt update
$ sudo apt install nvidia-jetpack

Hello,

Thank you for your reply.

This is the first time when I run $ sudo apt update I receive errors.
Here (below) is the errors I received

So I reboot my Jetson Orin and after rebooting I received this bellow

And when running $ sudo apt install nvidia-jetpack I receive and error and enable to correct problems

Thanks

For example when I try to run trtexec --onnx=model.onnx as suggested here (after putting the right path to the onnx model). Before It was working normaly but now it will show me theses errors below same as I put in the beginning of this topic.

Do you think it is the

  • cuDNN because it has a higher version as the Support matrix

or

  • CUDA because it is not detected by the Jetson AGX Orin

Hello again @carolyuu @kayccc ,

I tried to follow this link here to verify cuDNN on my Jetson AGX Orin, I had an issue when compiling

image

But I resolved it using this command sudo apt-get install libfreeimage3 libfreeimage-dev as suggested here (had some few warning though).

And when running the test using $ ./mnistCUDNN I had ERROR and failure see below

I thought it may help you diagnose the problem perhaps.

NOTE: I will try to see if I could install a cuDNN version that is compatible with the Support matrix doc and will let you know what will happen.

Thank you again in advance.

Hello Harry-S,

You can fix the dependency issue with:
sudo apt-get -y install

In your case sudo apt-get -y install nvidia-cudnn8=5.0.1-b118

Thanks

Hi Harry-S,

Suggest you can use SDK Manager to flash image and install sdk components.

Hello @rohithkumarp

Thank you for your reply.

I tried to use sudo apt-get -y install nvidia-cudnn8=5.0.1-b118 but I have an error and instead of installing CUDA 11.4 and cuDNN 8.3.2, it install CUDA 11.6 and cuDNN 8.4.0.27

Hello @Harry-S,

Did you attempt to install the dependencies via the same way ?
sudo apt-get install libcudnn8=8.3.2.49-1+cuda11.4

Let me know how this works?
Or you can flash the fresh image with all components through sdk manager as @carolyuu suggested. I installed everything through sdk manager, but have faced the same issue that you are facing while installing them on non-jetson devices and I did manage to resolve them. Also always follow/refer support matrix for CUDA, cudnn and tensorrt to get the version right. Thanks

1 Like

Hello @rohithkumarp ,

Yes it works after installing the dependencies and downgrading using this command sudo apt-get install libcudnn8=8.3.2.49-1+cuda11.4 that you suggested.

  • trtexec is working for me but still my Jetson Orin can’t see CUDA but for me it is not a problem because trtexec is working now

image

Thank you very much 😊

1 Like

@Harry-S Great that it is working. Yes it doesn’t show over there for me as well, still it has to be fixed from Nvidia’s side hopefully it is fixed in the future jetpack versions. But CUDA will get detected for the applications you use.

Thank you!

1 Like

yes now I can run trtexec without problem. Hopefully they will fix it.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.