Hi, i have just bought a Jetson orin Developer Kit and i am trying to use Torch with CUDA but i am always getting :
torch.cuda.is_available()
False
I have tried many things (spent hours) to go through the support but i cannot get it to to work. Issue with compilation, various version etc… Is there any proper documentation from the standard SD image to make Torch using CUDA? My System configuration is coming from the original SD card image provided by Nvidia site.
I cannot install the drivers properly and end up with the following issue: I cannot fix this anymore
sudo apt purge libnvidia-decode-535
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
You might want to run ‘apt --fix-broken install’ to correct these.
The following packages have unmet dependencies:
libnvidia-encode-535 : Depends: libnvidia-decode-535 (>= 535.154.05) but it is not going to be installed
nvidia-compute-utils-535 : Depends: libnvidia-compute-535 but it is not going to be installed
nvidia-driver-535 : Depends: libnvidia-gl-535 (= 535.154.05-0ubuntu0.22.04.1) but it is not going to be installed
Depends: libnvidia-compute-535 (= 535.154.05-0ubuntu0.22.04.1) but it is not going to be installed
Depends: libnvidia-extra-535 (= 535.154.05-0ubuntu0.22.04.1) but it is not going to be installed
Depends: libnvidia-decode-535 (= 535.154.05-0ubuntu0.22.04.1) but it is not going to be installed
Depends: nvidia-utils-535 (= 535.154.05-0ubuntu0.22.04.1) but it is not going to be installed
E: Unmet dependencies. Try ‘apt --fix-broken install’ with no packages (or specify a solution).
I read many topics on this but none is working. It is quite wierd that it should take so much time to setup an cuda environment for nvidia jetson.
@laurent049pc the SD card image already comes with the GPU drivers, CUDA Toolkit, ect pre-installed on it. The Jetson’s integrated GPU drivers are different than the discrete GPU drivers in the ubuntu apt repo. Since you installed those, I would just re-flash your SD card and start fresh to alleviate any conflicts from them in the future.
Install one of the pre-built PyTorch wheels that have CUDA support for your version of JetPack from this thread:
Or you can also use l4t-pytorch container which comes with PyTorch, torchvision, and torchaudio pre-installed in them.
I also have same issue. Tried to install from scratch. Btw, @dusty_nv my Jetson Orin NX has SSD setup, no sd card. As I saw in the SDK manager and jetpack documentations, Jetson Orin NX Series don’t have sd card port or image and don’t have pre-builded OS Images. Is it me doesn’t see or also you can validate it?
I took the SD card and was capable to run initiate the system with it. CUDA was installed with version 12.2.
Then i found the following link.
which was great help to make the Torch works and finally succeeded to make torch and cuda work together. But torchvision was not compatible with torch version.
Then when i tried to install ultralytics with pip, the torch version was removed breaking the cuda/torch relationship and a new version from pip was installed. The problem with version from pip is that i does not compile filed included.
I used indeed the hyperlink you suggested.
i installed PyTorch 2.1.0 and torchvision v0.16.1. as i cold not get the 2.2.0 to work.
I then succeed to install torch working with Cuda. But even with 2.1.0 i got error with TorchVision saying it was not compatible.
I see that this link is your link ;-) but is dated from 2019. do you have any suggestion for new version of pytorch. For example when i use Ultralytics, it asked for torch 2.2.1 and desintalled existing torch version
@msaidbilgehan in lieu of SD card image, SDK Manager will install the compute JetPack components to your board (like CUDA/cuDNN/TensorRT/ect) after it flashes it. Or you can install them from the nvidia l4t apt server. Can you sudo apt-get update && apt-cache search cuda-toolkit ?
@laurent049pc that thread was started in 2019 but is updated with the recent wheels for PyTorch 2.0 and newer. You will need to workaround ultralytics always trying to uninstall your version of PyTorch by commenting out it’s requested PyTorch version in the ultralytics requirements.txt, setup.py, or project.toml. If you still have issues I would recommend using l4t-pytorch container because it helps keep your environment sane when working through things like this.
I succeeded to install Torchvision and get the version.
I downloaded Ultralytics and remove the requirements for higher version of PyTorch.
But now i am getting the following issue:
Traceback (most recent call last):
File "/home/jetson/code/IoTEx.Dorothy/./yoloBulb.py", line 1, in <module>
from ultralytics import YOLO
File "/home/jetson/ultralytics/ultralytics/__init__.py", line 5, in <module>
from ultralytics.data.explorer.explorer import Explorer
File "/home/jetson/ultralytics/ultralytics/data/__init__.py", line 3, in <module>
from .base import BaseDataset
File "/home/jetson/ultralytics/ultralytics/data/base.py", line 17, in <module>
from ultralytics.utils import DEFAULT_CFG, LOCAL_RANK, LOGGER, NUM_THREADS, TQDM
File "/home/jetson/ultralytics/ultralytics/utils/__init__.py", line 1015, in <module>
SETTINGS = SettingsManager() # initialize settings
File "/home/jetson/ultralytics/ultralytics/utils/__init__.py", line 923, in __init__
from ultralytics.utils.torch_utils import torch_distributed_zero_first
File "/home/jetson/ultralytics/ultralytics/utils/torch_utils.py", line 17, in <module>
import torchvision
File "/home/jetson/.local/lib/python3.10/site-packages/torchvision/__init__.py", line 6, in <module>
from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils
File "/home/jetson/.local/lib/python3.10/site-packages/torchvision/_meta_registrations.py", line 164, in <module>
def meta_nms(dets, scores, iou_threshold):
File "/home/jetson/.local/lib/python3.10/site-packages/torch/_custom_ops.py", line 253, in inner
custom_op = _find_custom_op(qualname, also_check_torch_library=True)
File "/home/jetson/.local/lib/python3.10/site-packages/torch/_custom_op/impl.py", line 1076, in _find_custom_op
overload = get_op(qualname)
File "/home/jetson/.local/lib/python3.10/site-packages/torch/_custom_op/impl.py", line 1062, in get_op
error_not_found()
File "/home/jetson/.local/lib/python3.10/site-packages/torch/_custom_op/impl.py", line 1052, in error_not_found
raise ValueError(
ValueError: Could not find the operator torchvision::nms. Please make sure you have already registered the operator and (if registered from C++) loaded it via torch.ops.load_library.
I found the issue. I found out that pip had installed torchvision 0.17.1. I unistalled this by running pip3 uninstall torchvision and everything worked fine.
Do you think you could update your post for the torch version 2.2.1?
I will try later the container approach. Thanks for the help.