Cannot install Pytorch 2.x with CUDA support

jj-nvidv · August 22, 2024, 8:50am

I need to install Pytorch 2.x with CUDA support on my Jetson Orin Nano. I’ve followed the instructions specified here:

https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html

But after importing Pytorch I got the following error:

>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/<username>/miniconda3/envs/<envname>/lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403
ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory

Here are my specs:

Device: Jetson Orin Nano
OS: Ubuntu 20.04.6 LTS 64bits
JetPack version: 5.3 (nvidia-l4t-core 35.3.1-xxx)
CUDA version: 11.4
Python version: 3.8

This is the .whl that I want to install:
(from PyTorch for Jetson)

Note that I can install Pytorch successfully with the following pip command:

pip3 install --pre torch==2.1 --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v53

However, for some reason CUDA support is not there:

>>> import torch
>>> torch.cuda.is_available()
False

(I want to install it in a Conda venv but I have the same result after installing it without using a venv.)

Please, any help will be appreciated.

AastaLLL · August 23, 2024, 3:59am

Hi,

Could you share the output of below command with us?

$ sudo apt show nvidia-jetpack

$ python -c "import torch; print(torch.__version__); print(torch.backends.cudnn.version())"

Thanks.

jj-nvidv · August 26, 2024, 7:47am

Thanks for the response!

This is the output for apt show nvidia-jetpack:

Package: nvidia-jetpack
Version: 5.1.1-b56
Priority: standard
Section: metapackages
Maintainer: NVIDIA Corporation
Installed-Size: 199 kB
Depends: nvidia-jetpack-runtime (= 5.1.1-b56), nvidia-jetpack-dev (= 5.1.1-b56)
Homepage: http://developer.nvidia.com/jetson
Download-Size: 29,3 kB
APT-Sources: https://repo.download.nvidia.com/jetson/common r35.3/main arm64 Packages
Description: NVIDIA Jetpack Meta Package

When I install PyTorch following the guide from docs.nvidia.com I got the error right after importing torch library so I cannot print output of torch and cudnn versions:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/.../miniconda3/envs/.../lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403
ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory

If I install PyTorch with the alternative command I mentioned I got the following outputs:

2.1.0
None

I was installing PyTorch compilation for JP v5.3 but it seems I actually have v5.1. However, I got the same outputs if I install PyTorch from:
pip3 install --pre torch==2.1 --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v51

Just in case, this is the output from nvcc --version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Sun_Oct_23_22:16:07_PDT_2022
Cuda compilation tools, release 11.4, V11.4.315
Build cuda_11.4.r11.4/compiler.31964100_0

I hope these details could help to solve the problem.

Thank you.

AastaLLL · August 28, 2024, 5:47am

Hi,

It looks like you have a CPU version of PyTorch.
Could you try if our package works outside of the virtual environment?

Thanks.

jj-nvidv · August 28, 2024, 10:13am

I tried installing it outside of the virtual environment following the installation guide but I’m still having the same error after importing PyTorch:

>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/<username>/.local/lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403
ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory

I’m also using Python 3.8 outside of the virtual environment.

It seems that some file or directory related with cudnn is missing so I added this to my ~/.bashrc but it made no effect:

export PATH=/usr/local/cuda-11.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH

AastaLLL · August 29, 2024, 6:41am

Hi,

Could you try the below command to see if it works?

$ wget https://developer.download.nvidia.cn/compute/redist/jp/v512/pytorch/torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl -O torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl 
$ pip3 install ./torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl

Thanks.

jj-nvidv · August 29, 2024, 2:59pm

I tried but I’m still having the same error.

I tried to reinstall cuDNN but it had no effect neither:

wget https://developer.download.nvidia.com/compute/cudnn/9.3.0/local_installers/cudnn-local-tegra-repo-ubuntu2004-9.3.0_1.0-1_arm64.deb
sudo dpkg -i cudnn-local-tegra-repo-ubuntu2004-9.3.0_1.0-1_arm64.deb
sudo cp /var/cudnn-local-tegra-repo-ubuntu2004-9.3.0/cudnn-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cudnn

(from NVIDIA webpage)

AastaLLL · September 2, 2024, 6:35am

Hi,

It looks like you upgraded the cuDNN version from the website.

But our prebuilt package is built with the default cuDNN version.
For JetPack 5.1.2, it should be 8.6.0.

Please uninstall cuDNN 9.3.0, install the cuDNN from JetPack, and try it again.
Thanks.

jj-nvidv · September 2, 2024, 10:06am

Great! That solved the problem, thank you so much!

I removed my current cuDNN installation:

$ dpkg -l | grep cudnn
...
cudnn-local-tegra-repo-ubuntu2004-9.3.0
...
$ sudo apt-get purge cudnn-local-tegra-repo-ubuntu2004-9.3.0

After that I installed cuDNN 8.6.0:

$ sudo apt-get install libcudnn8 libcudnn8-dev libcudnn8-samples
$ dpkg -l | grep cudnn
ii  libcudnn8                                  8.6.0.166-1+cuda11.4                  arm64        cuDNN runtime libraries
ii  libcudnn8-dev                              8.6.0.166-1+cuda11.4                  arm64        cuDNN development libraries and headers
ii  libcudnn8-samples                          8.6.0.166-1+cuda11.4                  arm64        cuDNN samples

Now I can import PyTorch with CUDA support:

$ python
>>> import torch
>>> torch.cuda.is_available()
True

system · October 9, 2024, 4:38am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Installation Cudnn for Jetson Orin Nano Jetson Orin Nano cudnn	3	161	October 3, 2024
Orin Pytorch/CUDA issue Jetson Orin NX cuda , pytorch , jetson-nano	11	2493	September 11, 2023
No torchaudio package that works with torch for Cuda Jetson Orin Nano cuda	6	254	August 12, 2024
OSError: libcufft.so.10: cannot open shared object file: No such file or directory Jetson Orin NX cuda , pytorch	4	380	October 14, 2024
Jetson orin nano Cuda Cudnn torch torchauido torchvision Jetson Orin Nano cuda	11	702	May 27, 2024
Installing Pytorch OSError: libcurand.so.10: cannot open shared object file: No such file or directory Jetson AGX Xavier pytorch	26	33757	October 21, 2021
Pytorch installed on l4t-jetpack:r35.4.1 container on Jetson Orin Nano (JetPack 6.0 Developer Kit) fails to recognize CUDA Jetson Orin Nano cuda , docker , pytorch , python , containers	2	65	October 22, 2024
Jetson Nano Torch 1.6.0 PyTorch Vision v0.7.0-rc2 Runtime Error Jetson Nano pytorch	4	1433	October 18, 2021
Pytorch with CUDA support Jetson Orin Nano Jetson Orin Nano cuda , pytorch , python , cudnn	4	703	May 14, 2024
Pytorch for jetson nano orin does not work Jetson Orin Nano pytorch	5	1339	August 22, 2023

Cannot install Pytorch 2.x with CUDA support

Related topics