Issue with CUDA Availability on Jetson Orin AGX 64GB in Container dustynv/nano_llm:r36.2.0

[avivash71]

Hello,

I am using a Jetson Orin AGX 64GB with the container dustynv/nano_llm:r36.2.0 and encountering an issue related to CUDA availability.

When I run the following command inside the container:
nvcc --version
I get the following output:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:08:11_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0

This suggests that CUDA is correctly installed inside the container.

However, when I try to check CUDA availability using PyTorch with the following code:

print(f"Cuda test:{torch.cuda.is_available()}")
It returns False.

Additionally, when I attempt to run a different model on the GPU, I encounter this error:
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from Download The Latest Official NVIDIA Drivers
Could you please advise on how to resolve this issue and get CUDA working properly inside the container on my Jetson Orin AGX?

Thank you in advance!

Best regards,
Aviv

[AastaLLL]

Hi,

r36.2.0 is the BSP version of JetPack 6 DP.
As we already have a GA release, it’s recommended that you try the GA release instead.

For example:
nano_llm:r36.4.0 with JetPack 6.1 GA.
nano_llm:r36.3.0 with JetPack 6.0 GA.

Thanks.