Jetson Orin Nano CUDA 12.6 in JetPack 6.1 in Tensorflow and Pytorch

[joerg.hoenig]

I installed via SDK Manager JetPack 6.1 on a NVIDIA Orin Nano Development kit with 8GB RAM and 512GB M.2 SSD

including CUDA runtime and CUDA SDK components
Via miniconda I reated an environment with Python 3.10. Afterwards I installded Tensorflow and Pytorch with NVIDIAS prebuilt installation wheels.
All installation processes went without error messages or warnings.
Nevertheless, the Jetson cannot be used properly neither with Tensorflow, nor with Pytorch.

[joerg.hoenig]

First, Tensorflow behaviour. Tensorflow and Keras can be imported. But already during the import in the console there are warning (?) that “… cuda_executor.cc:984 could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have built without NUMA support.”
This occurs several times.

The GPU seems to me recognized correctly “… device:GPU:0 with 2447MB memory : device:0, name: Orin, pci bus id: 0000:00:00.0, compute capability: 8.7”

Later there seems a more severe problem with "… can’t find libdevice directory ${CUDA_DIR}/nvvm/libdevice. … it seems, that the CUDA-related directory structure, as well as the required files have not been established during the initial installation process by the SDK manager.

under /usr/local one may find 3 “CUDA” directories: cuda, cuda-12, cuda-12.6. All of them consist of the subdirectories gds, include, lib64, targets but no “bin” subdirectory as suggested by the $PATH variable, composed by the SDK Manager

Thus, a nvcc cannot be found in the whole file system.

[AastaLLL]

Hi,

We can find nvcc under the cuda folder:

$ ll /usr/local/cuda/bin/nvcc
-rwxr-xr-x 1 root root 16370032  八  15 01:59 /usr/local/cuda/bin/nvcc*

Is this issue related to the virtual environment?
Could you check it outside of the conda environment?

Thanks.

[joerg.hoenig]

I deactivated cuda - but there is still no " …/cuda/bin" directory. The directory structure looks like in the picture “CUDA_directories” above

[joerg.hoenig]

beg your pardon - I deactivated “conda” :-)

[joerg.hoenig]

I guess, that the CUDA installation should have been done during the initial software installation via the SDK Manager.
During the installation process it was a little bit confusing for me, that the Jetson started to boot the Linux even the Linux image transfer to the Jetson was not finished yet (at about 70%) and the shortcut between pins 9 and 10 still has been there. Maybe this prevents the complete software installation ? Should I disconnect from the DevKit all other connections beside the USB-C cable to the control PC ?

[AastaLLL]

Hi,

Could you try the below command to check if any package missing in the SDKmanager installation?

$ sudo apt install nvidia-jetpack

Thanks.

[joerg.hoenig]

Hi, you have been right, there was missing a lot as you see on the first screenshot

after running the installation, the proper directories and files could be found.

I was running then under spyder some little programs using tensorflow and keras. The speed was about 3 - 4 times that of running on my old AMD Ryzen7 CPU.

Unfortunately Pytorch is still not possible to run

should I create a different different environment and repeat the installation with the pytorch wheel there ?

[joerg.hoenig]

I created a different environment and installed the pytorch wheel again - unfortunately same result

[joerg.hoenig]

Is there any idea what to do to get also Pytorch running ???

[AastaLLL]

Hi,

Sorry for the late update.
Please find the document to install cuSparseLt.

Installing PyTorch for Jetson Platform - NVIDIA Docs

Thanks.