CUDA 12 : Insufficient driver version on AGX Orin

Problem :

  • I am unable to get CUDA 12 running successfully on Jetson AGX Orin.
  • Device Query :
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL

Steps to Reproduce

1. Fresh OS Flash from SDK Manager

2. Verify System Requirements

> lspci | grep -i nvidia
0001:00:00.0 PCI bridge: NVIDIA Corporation Device 229e (rev a1)

> uname -m && cat /etc/*release
# R35 (release), REVISION: 2.1, GCID: 32413640, BOARD: t186ref, EABI: aarch64, DATE: Tue Jan 24 23:38:33 UTC 2023
VERSION="20.04.5 LTS (Focal Fossa)"
> gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

> sudo apt-get install linux-headers-$(uname -r)
E: Unable to locate package linux-headers-5.10.104-tegra
E: Couldn't find any package by glob 'linux-headers-5.10.104-tegra'
E: Couldn't find any package by regex 'linux-headers-5.10.104-tegra'

> sudo apt install nvidia-l4t-kernel-headers
nvidia-l4t-kernel-headers is already the newest version (5.10.104-tegra-35.2.1-20230124153320).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

3. Prepare Ubuntu

> sudo apt-key del 7fa2af80
> wget
> sudo dpkg -i cuda-keyring_1.0-1_all.deb

4. Update and Install

> sudo apt-get update
> sudo apt-get install cuda
> sudo reboot

5. Post Install Actions

# .bashrc File
# ------ CUDA -------------
export PATH=/usr/local/cuda-12/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-12/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

# --- Links each point to cuda-12.0

6. Build Samples

> cd cuda-samples
> cd Samples/1.../deviceQuery/
> ./deviceQuery
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL

If I point my path and LD_LIBRARY_PATH variables to cuda 11.4 instead of 12.0, everything works fine. If either one, or both, is pointing to cuda 12 I get this error.

Cuda toolkit is installed for both 11.4 and 12.0. Jetpack is installed from apt as of today; version 5.1-b147.


Could you follow the whole steps mentioned on the below page?
(A cuda-tegra-xxx.deb needs to be installed)


I followed the instructions exactly. Everything installed without errors. Still getting the same result.

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL

CUDA Versions

  int driver=0;
  int runtime=0;


  printf("Cuda Runtime Version = %d\n", runtime);
  printf("Cuda Driver  Version = %d\n", driver);


Cuda Runtime Version = 12000
Cuda Driver  Version = 11040


> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:28_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0

CUDA Driver

> apt search cuda-driver
cuda-driver-dev-11-4/stable,now 11.4.298-1 arm64 [installed,automatic]
  CUDA Driver native dev stub library

cuda-driver-dev-11-8/unknown 11.8.89-1 arm64
  CUDA Driver native dev stub library

cuda-driver-dev-12-0/unknown,now 12.0.146-1 arm64 [installed,automatic]
  CUDA Driver native dev stub library

I could be wrong, but I don’t think CUDA 12 is available on Jetsons. If the CUDA 12 you got is from most non-Jetson repositories (even if it is arm64), the integrated GPU (iGPU) of a Jetson is directly wired to the memory controller; the other non-Jetson repositories are intended for discrete PCI GPUs (dGPU).

@linuxdev, thank you for your feedback. That is something I’ve been wondering about and unsure of. It may just be my naivete but some of the documentation was unclear to me.

System Requirements

For example, the cuda toolkit 12.0 downlod page has a path specifically for aarch64-jetson → native → Ubuntu 20.04. But when I follow the instructions the drivers seem to be installed, but not enabled, or not being used.

On the cuda for tegra page, it lists that cuda 12.0 is compatible, but only through the upgrade package. I’m not sure if the upgrade package is the same as the cuda 12.0 download link, or a different package manager I need to enable and utilize.

The CUDA Installation Guide system requirements lists Arm64 Jetson (igpu) as supported, but it wasn’t clear to me if the “NO” boxes in the table mean that it’s not compatible, or that the system doesn’t come with a version that is compatible, or something else.


| Distribution          | Kernel1           | Default GCC | GLIBC | GCC2,3 | ICC3 | NVHPC3 | XLC3 | CLANG | Arm C/C++ |
| L4T4 20.04.z (z <= 5) | 5.10.120-70–tegra | 9.4.0       | 2.31  | NO     | NO   | NO     | NO   | NO    | NO        |

Does this mean that I need to install all the “NO” packages? Or there is no supported version of those packages, so there’s no working version of CUDA 12 on Jetson ? (seems odd to release one that has no supported working path) Are there just certain features I won’t be able to get working?

CUDA Driver 11.4 vs 12.0

Also the cuda-driver-dev-12-0 package is installed, but if getCudaDriverVersion( ) returns 11040, does that imply that the system is using the cuda-driver-dev-11-4 package? Do I need to tell it which cuda driver to use?

Other than setting the PATH and LD_LIBRARY_PATH in my .bashrc file, is there another way I need to instruct the system to use the correct cuda-driver?

This is why I am not certain if my answer is correct. You can check your L4T release with “head -n 1 /etc/nv_tegra_release” (this is what actually gets flashed). L4T releases from R32.x or earlier would all be CUDA 10.x. It isn’t until you get to R34.x when CUDA 11.x becomes available. Again, I could be wrong, but I think that the JetPack/SDK Manager installer software only installs CUDA 11.x. There are arm64 targets which have separate installation packages, but mostly those are not compatible with Jetsons.

Perhaps NVIDIA could answer this: At which L4T release is CUDA 12.x available? Or is this still a “future” release? I might be behind the times and just did not see CUDA 12 added.


We start to provide the Jetson package on the website from CUDA 12.
This allows users to use the latest CUDA software without waiting for the next JetPack release.

Some user has used CUDA 12 on JetPack 5.0.2 and it can work well.
However, it seems that there is an issue with JetPack 5.1.

We are going to reproduce this and check internally.
Will share more information with you later.


@AastaLLL , thank you for clarifying and looking into that. Over the weekend I was able to get it ~working if I set:

export PATH=/usr/local/cuda-12/bin:${PATH}
export LD_LIBRARY_PATH=/usr/local/cuda-12.0/compat

Can you explain what the difference is between using the compatibility library and the full cuda-12 library? With Jetpack 5.0.2 is CUDA 12 working in native mode? Or compatibility mode?


1 Like


We can reproduce similar behavior.
Same on JetPack 5.0.2 and JetPack 5.1.

We are checking internally about the difference between following two packages.
Will share more info with you after we get a feedback:



1 Like


Sorry for the late reply.

The compat version is required when using new CUDA toolkit components on systems with older base installations.
Please check the below document for more details:


@ AastaLLL,
Thanks for the update. I look forward to a time when CUDA driver and runtime releases are better synced up, as this and other similar stack compatibility issues has taken up over half our development ime so far…

I look forward to further updates, improvements in the software release workflow and a final resolution of his issue

1 Like


We release the package from the CUDA website to allow users to get the latest CUDA faster.

Since JetPack is a relatively complicated system with lots of compatibilities.
In general, it takes a longer time to update the library version.


1 Like

Have the same issue here. CUDA 12.1 with tegra release R35.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.