Onnxruntime installation on jetson agx Orin developer kit

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

I have jetson agx orin developer kit l4t 36.4.7.

I tried installing onnxruntime-gpu after downloading the wheel from jp6/cu126 index

But when I try getting the providers I have only cpu and azure. Tensorrt and cpu is not showing.

please help

2

*** Please note that this reply is generated by LLM automatically ***

To address your issue with installing ONNX Runtime GPU on your Jetson AGX Orin Developer Kit with L4T 36.4.7, I’ll provide a step-by-step guide and some troubleshooting tips.

Prerequisites:

  1. Ensure you have the correct version of JetPack installed on your Jetson AGX Orin Developer Kit. You can check the version by running dpkg -l | grep jetpack.
  2. Verify that you have the NVIDIA GPU driver installed and running correctly. You can check the driver version by running nvidia-smi.

Installing ONNX Runtime GPU:

  1. Download the ONNX Runtime GPU wheel from the NVIDIA GPU Cloud (NGC) index or the official ONNX Runtime GitHub repository. Make sure to select the correct wheel for your Jetson AGX Orin Developer Kit (L4T 36.4.7).
  2. Install the ONNX Runtime GPU wheel using pip: pip3 install onnxruntime_gpu-<version>-cp37-cp37-linux_aarch64.whl (replace <version> with the actual version number).

Troubleshooting:

  1. Missing GPU provider: If you’re only seeing the CPU and Azure providers, it’s likely that the GPU provider is not being detected. Check that the NVIDIA GPU driver is installed and running correctly. You can try reinstalling the driver or updating to the latest version.
  2. TensorRT not showing: If TensorRT is not showing as a provider, ensure that you have the correct version of TensorRT installed on your system. You can check the version by running dpkg -l | grep tensorrt. If you’re using an older version, try updating to the latest version.
  3. CPU provider not showing: If the CPU provider is not showing, it’s possible that the ONNX Runtime GPU installation is not correctly configured. Try reinstalling the ONNX Runtime GPU wheel or checking the installation logs for errors.

Comparison with other Jetson devices:

The Jetson AGX Orin Developer Kit is a more powerful device compared to the Jetson Orin NX and Jetson Orin Nano. However, the installation process for ONNX Runtime GPU should be similar across all devices. If you’re experiencing issues on other Jetson devices, try following the same troubleshooting steps as above.

Additional resources:

  • NVIDIA Jetson Linux Developer Guide: This guide provides detailed information on installing and configuring JetPack on your Jetson device.
  • ONNX Runtime GitHub repository: This repository provides documentation and examples for installing and using ONNX Runtime on various platforms, including Jetson devices.
  • NVIDIA GPU Cloud (NGC) index: This index provides a list of available wheels for ONNX Runtime GPU, including those for Jetson devices.

By following these steps and troubleshooting tips, you should be able to successfully install ONNX Runtime GPU on your Jetson AGX Orin Developer Kit and resolve any issues with missing providers.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

4

Hi,

Just double-checked, TensorrtExecutionProvider and CUDAExecutionProvider are included in the package.
Please try it again:

$ md5sum onnxruntime_gpu-1.23.0-cp310-cp310-linux_aarch64.whl 
2722622ccdadd909a130b6317bce1ae3  onnxruntime_gpu-1.23.0-cp310-cp310-linux_aarch64.whl

$ pip3 install onnxruntime_gpu-1.23.0-cp310-cp310-linux_aarch64.whl --force-reinstall
$ pip3 install 'numpy<2'
$ python3
Python 3.10.12 (main, Nov  4 2025, 08:48:33) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime as ort
>>> ort.get_all_providers()
['NvTensorRTRTXExecutionProvider', 'TensorrtExecutionProvider', 'CUDAExecutionProvider', 'MIGraphXExecutionProvider', 'ROCMExecutionProvider', 'OpenVINOExecutionProvider', 'DnnlExecutionProvider', 'VitisAIExecutionProvider', 'QNNExecutionProvider', 'NnapiExecutionProvider', 'VSINPUExecutionProvider', 'JsExecutionProvider', 'CoreMLExecutionProvider', 'ArmNNExecutionProvider', 'ACLExecutionProvider', 'DmlExecutionProvider', 'RknpuExecutionProvider', 'WebNNExecutionProvider', 'WebGpuExecutionProvider', 'XnnpackExecutionProvider', 'CANNExecutionProvider', 'AzureExecutionProvider', 'CPUExecutionProvider']

Thanks.