Pytorch Installation Issue: print(torch.cuda.is_available()) - False

Hi, I have installed PyTorch, but print(torch.cuda.is_available()) returns False. I checked the installation command and versions, and they seem correct. I’m not sure what the issue is. Please help analyze this.

Device: Jetson AGX Orin
Cuda version: 12.1
Python: 3.10.12

Installed Pytorch:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

the installed version is:
nvidia@nvidia-desktop:~$ pip list | grep torch
torch 2.0.1
torchaudio 2.0.2
torchvision 0.15.2

nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import torch
print(torch.version)
2.0.1
print(torch.cuda.is_available())
False
print(torch.version.cuda)
None

Here is the detailed record:
nvidia@nvidia-desktop:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:19:26_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0
nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$ python3 -V
Python 3.10.12
nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$ pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://download.pytorch.org/whl/cu121
Collecting torch
Using cached https://download.pytorch.org/whl/torch-2.0.1-cp310-cp310-manylinux2014_aarch64.whl (74.0 MB)
Collecting torchvision
Using cached https://download.pytorch.org/whl/torchvision-0.17.0-cp310-cp310-linux_aarch64.whl (14.0 MB)
Collecting torchaudio
Using cached https://download.pytorch.org/whl/torchaudio-2.2.0-cp310-cp310-linux_aarch64.whl (1.6 MB)
Requirement already satisfied: sympy in /usr/lib/python3/dist-packages (from torch) (1.9)
Requirement already satisfied: typing-extensions in ./.local/lib/python3.10/site-packages (from torch) (4.9.0)
Requirement already satisfied: filelock in ./.local/lib/python3.10/site-packages (from torch) (3.13.1)
Requirement already satisfied: jinja2 in ./.local/lib/python3.10/site-packages (from torch) (3.1.3)
Requirement already satisfied: networkx in ./.local/lib/python3.10/site-packages (from torch) (3.2.1)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/lib/python3/dist-packages (from torchvision) (9.0.1)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from torchvision) (2.25.1)
Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (from torchvision) (1.21.5)
Collecting torchvision
Using cached https://download.pytorch.org/whl/torchvision-0.16.2-cp310-cp310-linux_aarch64.whl (14.0 MB)
Using cached https://download.pytorch.org/whl/torchvision-0.16.1-cp310-cp310-linux_aarch64.whl (14.0 MB)
Using cached https://download.pytorch.org/whl/torchvision-0.16.0-cp310-cp310-linux_aarch64.whl (14.1 MB)
Using cached https://download.pytorch.org/whl/torchvision-0.15.2-cp310-cp310-manylinux2014_aarch64.whl (1.2 MB)
Collecting torchaudio
Using cached https://download.pytorch.org/whl/torchaudio-2.1.2-cp310-cp310-linux_aarch64.whl (1.6 MB)
Using cached https://download.pytorch.org/whl/torchaudio-2.1.1-cp310-cp310-linux_aarch64.whl (1.6 MB)
Using cached https://download.pytorch.org/whl/torchaudio-2.1.0-cp310-cp310-linux_aarch64.whl (1.6 MB)
Using cached https://download.pytorch.org/whl/torchaudio-2.0.2-cp310-cp310-manylinux2014_aarch64.whl (4.0 MB)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib/python3/dist-packages (from jinja2->torch) (2.0.1)
Installing collected packages: torch, torchvision, torchaudio
WARNING: The scripts convert-caffe2-to-onnx, convert-onnx-to-caffe2 and torchrun are installed in ‘/home/nvidia/.local/bin’ which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed torch-2.0.1 torchaudio-2.0.2 torchvision-0.15.2
nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$ pip list | grep torch
torch 2.0.1
torchaudio 2.0.2
torchvision 0.15.2
nvidia@nvidia-desktop:~$
nvidia@nvidia-desktop:~$ import torch
-bash: import: command not found
nvidia@nvidia-desktop:~$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import torch
print(torch.version)
2.0.1
print(torch.cuda.is_available())
False
print(torch.version.cuda)
None

A quick check to do when having gpu detection issue is to verify if you are using a gpu enabled torch.

For that first check where torch is installed in python with:

import torch
print(torch.file)

Then list the file in the torch/lib folder.
If its a cuda installation of torch you should see library like libtorch_cuda.so .
If you don’t see them its not a gpu problem but a torch installation problem.

In summary for torch to detect gpu you need two things:

  1. A cuda enable torch installation.
  2. A gpu and driver installation.

Thank you for your reply.
nvidia@nvidia-desktop:~$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import torch

print(torch.file)
/home/nvidia/.local/lib/python3.10/site-packages/torch/init.py

[1]+ Stopped python3

nvidia@nvidia-desktop:~$ cd /home/nvidia/.local/lib/python3.10/site-packages/torch/lib/
nvidia@nvidia-desktop:~/.local/lib/python3.10/site-packages/torch/lib$ ls
libc10.so libtorch_cpu.so libtorch_python.so
libshm.so libtorch_global_deps.so libtorch.so

There is no libtorch_cuda.so file in the torch/lib directory, so using the installation command pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 has issues.

The reason is that the .whl files in https://download.pytorch.org/whl/cu121 are for Linux (x86_64).

Solution: Follow the instructions from NVIDIA Developer Forums - PyTorch for Jetson to install the correct .whl files (linux_aarch64).


The issue is resolved.

import torch
printf(torch.version)
Traceback (most recent call last):
File “”, line 1, in
NameError: name ‘printf’ is not defined. Did you mean: ‘print’?
print(torch.version)
2.3.0
print(torch.cuda.is_available())
True
print(torch.version.cuda)
12.2

Yes, this is generally how you end up with a “cpu” only version by having for a reason or an other the gpu version not compatible and falling back to the cpu version. I personally find that there should be a warning or something that the reason is not on the gpu but on pytorch installation itself. Myself and coworker have had to debug issue with that.