How to use the GPU acceleration on the Jetson Orin NX?

Robotics & Edge Computing Jetson Systems Jetson Orin NX
1

My Jetson Orin NX system has Jetpack 6.2, CUDA 12.6.68, cuDNN 9.3.0, and Python 3.10. I have installed onnxruntime-gpu version 1.19, as well as torch 2.3.0, torchaudio 2.3.0, and torchvision 0.18.0. However, I am unable to use GPU acceleration. When running the test code, I encountered the following error:

[W:onnxruntime:Default, onnxruntime_pybind_state.cc:965 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Require cuDNN 8.* and CUDA 12.*. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

Is this issue caused by having a higher version of cuDNN?

3

Hi,

Could you share which ONNXRuntime package you use?

Thanks.

5

Hello, below is the onnxruntime-gpu package I am using, onnxruntime_gpu-1.19.0-cp310-cp310-linux_aarch64.whl, and its download address is Jetson Zoo - eLinux.org.

6

Hi,

The latest package from Jetson Zoo is built with JetPack 6.0.
If you want to use it, please try to set up your device with the same environment.

Instead, you can build ONNXRuntime with jetson-container to get the CUDAExecutionProvider.

Thanks.

7

I resolved my issue using onnxruntime-gpu version 1.22. Thanks for your help all the same!