Trying to install pytorch with cuda in jetson orin NX

[juliocesargcosta123]

So, im trying to install the pytorch with cuda support in my jetson orin NX (dev kit), i already have done a fresh install using the SDK Manager and i have jetpack 6.0, and ubuntu 22.04, i tried to follow the instructions from the nvidia manual, and configured a virtual enviroment: https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html, the issue is that im getting an error in the pre-requisites installation of the libopenblas-dev like the following: sudo apt-get -y update;
sudo apt-get -y install python3-pip libopenblas-dev;
… OTHER LINKS …
Ign:9 https://packagecloud.io/headmelted/codebuilds/debian stretch InRelease
Err:10 https://packagecloud.io/headmelted/codebuilds/debian stretch Release
404 Not Found [IP: 2600:1f1c:2e5:6901:e375:73d1:461b:b889 443]
Reading package lists… Done
E: The repository ‘https://packagecloud.io/headmelted/codebuilds/debian stretch Release’ does not have a Release file.
N: Updating from such a repository can’t be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
libopenblas-dev is already the newest version (0.3.20+ds-1).
python3-pip is already the newest version (22.0.2+dfsg-1ubuntu0.4).
0 upgraded, 0 newly installed, 0 to remove and 18 not upgraded.,

Besides that i kept following the instructions and successfully installed the torch in the right version as it shows in the pip list, but when i try to import torch in a python program i get the following error: python3
Python 3.10.12 (main, Mar 22 2024, 16:50:05) [GCC 11.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import torch
Traceback (most recent call last):
File “/home/cedri/Documents/torch_test/venv/lib/python3.10/site-packages/torch/init.py”, line 225, in _load_global_deps
ctypes.CDLL(global_deps_lib_path, mode=ctypes.RTLD_GLOBAL)
File “/usr/lib/python3.10/ctypes/init.py”, line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: libnvToolsExt.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “”, line 1, in
File “/home/cedri/Documents/torch_test/venv/lib/python3.10/site-packages/torch/init.py”, line 288, in
_load_global_deps()
File “/home/cedri/Documents/torch_test/venv/lib/python3.10/site-packages/torch/init.py”, line 246, in _load_global_deps
_preload_cuda_deps(lib_folder, lib_name)
File “/home/cedri/Documents/torch_test/venv/lib/python3.10/site-packages/torch/init.py”, line 168, in _preload_cuda_deps
raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError: libcublas.so.*[0-9] not found in the system path [‘’, ‘/usr/lib/python310.zip’, ‘/usr/lib/python3.10’, ‘/usr/lib/python3.10/lib-dynload’, ‘/home/cedri/Documents/torch_test/venv/lib/python3.10/site-packages’],

I suspect that could be a problem related to the pre-requistes installation error, but as the link gives a 404 error, i dont have a clue on how to solve this.

[AastaLLL]

Hi,

Have you edited the source list?
The below errors should occur when running the sudo apt-get -y update.

Ign:9 https://packagecloud.io/headmelted/codebuilds/debian stretch InRelease
Err:10 https://packagecloud.io/headmelted/codebuilds/debian stretch Release

But it’s not a default path in our JetPack 6.0.
Do you add them manually?

Could you edit the /etc/apt/sources.list and remove the link to see if it helps?

Thanks.

[juliocesargcosta123]

Hi AastaLLL,

Thank you in advance, I discovered that the dep links were from another file within the /etc/apt folder, and most likely because of the VS Code installation I did previously, so when i removed these links the error no longer occurred.

Going further into the torch installation manual again, another error occurs when I try to import the torch library:

from torch._C import * # noqa: F403
ImportError: libcupti.so.12: cannot open shared object file: No such file or directory

I suspect that this error ocurred because of the cuda version that i had installed in the jetson which is currently the cuda-12.2. Also looking to the $LD_LIBRARY_PATH which points to the cuda installation folder i found a lot of libcu*.so.12 files but not the libcupti.so.12 file or similar.

Another thing that i would like to ask is about the step showing below from the tutorial, i could not find the llvm-8 directory in usr/lib/.

$ export LD_LIBRARY_PATH=/usr/lib/llvm-8/lib:$LD_LIBRARY_PATH

Best Regards,
Julio César

[AastaLLL]

Hi,

Sorry for the late update.
Do you get PyTorch work?

Thanks.

[juliocesargcosta123]

Hi AastaLLL,

Yes, i could make pytorch work just by changing the pytorch version in the virtual enviroment. To help others that can have the same problem as i, here is the specs that i used:

Jetson Orin NX
Jetpack: 6.0 L4T 36.3
CUDA runtime version: 12.2 (Was installed via SDK Manager)
Pytorch version: torch-2.3.0a0+ebedce2.nv24.02-cp310-cp310-linux_aarch64.whl (in a python venv)

To verify if the pytorch is working with CUDA integration, you can use a python file with following code:

import torch
print("CUDA is available: ", torch.cuda.is_available())

Also verifying with the jtop (jtop installation guide https://jetsonhacks.com/2023/02/07/jtop-the-ultimate-tool-for-monitoring-nvidia-jetson-devices/) command in the GPU tab if the gpu resources are being used.

Best Regards,
Júlio César