Pytorch compatibility issues (torch 2.0.0+nv23.5 && torchvision 0.15.1)

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
1

Hi there,

I’ve got an Orin NX and am really struggling to understand how to use the GPU for running yolov7. I’ve installed the Jetson Pytorch library from this nvidia link:

https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html

I can see that Pytorch is installed in pip and reports torch version is 2.0.0+nv23.5. But when I run yolo on the GPU I get an incompatibility error saying that my PyTorch & torchvision versions aren’t compatible. According to PyTorch I believe I have installed the right version of torchvision 0.15.1.

Here is some info about my Orin:
P-Numer: p3767-0000
Module: NVIDIA Jetson Orin NX (16GB ram)
Jetpack 5.1 [L4T 35.2.1]
CUDA 11.4.315
cuDNN 8.6.0.166
TensorRT: 5.1
VPI 2.2.4
Vulkan 1.3.204
OpenCV 4.5.4 - with CUDA: NO
torch 2.0.0+nv23.5
torchvision 0.15.1

Is there a specific torchvision library for Jetson products? Any suggestions would be helpful

Thanks,

Simon

2

Hi @simonnv, are you able to build torchvision as shown under the Installation section of this post:

Alternatively, you can try using the l4t-pytorch or l4t-ml containers which come with PyTorch/torchvision pre-installed.

4

Hi @dusty_nv, I appreciate the prompt reply!

From the forum you mentioned, I downloaded & installed the following:
“pip install torch-2.0.0+nv23.05-cp38-cp38-linux_aarch64.whl”

According to the same forum:
PyTorch v2.0.0 - torchvision v0.15.1

So I ran the command:
“pip install torchvision==0.15.1”

Ignore the weight/model issue below, I don’t think it’s relevant or the cause of this problem.

The console output is as follows:

rebotnix@rebotnix:~/Documents/yolov7$ ls -l ~/Pictures/peopleonbeach.webp
-rw-rw-r-- 1 rebotnix rebotnix 71178 Jun 9 12:50 /home/rebotnix/Pictures/peopleonbeach.webp

rebotnix@rebotnix:~/Documents/yolov7$ ls -l ~/Documents/weights/yolov7/
total 1101276
-rw-rw-r-- 1 rebotnix rebotnix 268261125 Jun 1 15:15 yolov7-d6.pt
-rw-rw-r-- 1 rebotnix rebotnix 304425133 Jun 1 15:16 yolov7-e6e.pt
-rw-rw-r-- 1 rebotnix rebotnix 195040397 Jun 1 15:14 yolov7-e6.pt
-rw-rw-r-- 1 rebotnix rebotnix 75587165 Jun 1 15:10 yolov7.pt
-rw-rw-r-- 1 rebotnix rebotnix 141255765 Jun 1 15:13 yolov7-w6.pt
-rw-rw-r-- 1 rebotnix rebotnix 143099649 Jun 1 15:13 yolov7x.pt

rebotnix@rebotnix:~/Documents/yolov7$ python detect.py --weights ~/Documents/weights/yolov7/yolov7.pt --conf 0.25 --img-size 640 --device 0 --source ~/Pictures/peopleonbeach.webp

/home/rebotnix/.local/lib/python3.8/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ‘/home/rebotnix/.local/lib/python3.8/site-packages/torchvision/image.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSs’If you don’t plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source?
warn(
Namespace(agnostic_nms=False, augment=False, classes=None, conf_thres=0.25, device=‘0’, exist_ok=False, img_size=640, iou_thres=0.45, name=‘exp’, no_trace=False, nosave=False, project=‘runs/detect’, save_conf=False, save_txt=False, source=’/home/rebotnix/Pictures/peopleonbeach.webp’, update=False, view_img=False, weights=[‘/home/rebotnix/Documents/weights/yolov7/yolov7.pt’])
YOLOR 🚀 v0.1-126-g84932d7 torch 2.0.0+nv23.05 CUDA:0 (Orin, 14485.75390625MB)

Downloading https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7.pt to /home/rebotnix/documents/weights/yolov7/yolov7.pt…
Download error: [Errno 2] No such file or directory: ‘/home/rebotnix/documents/weights/yolov7/tmp3rvxjn7c’
ERROR: Download failure: /home/rebotnix/documents/weights/yolov7/yolov7.pt missing, try downloading from Releases · WongKinYiu/yolov7 · GitHub

Fusing layers…
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
RepConv.fuse_repvgg_block
Model Summary: 306 layers, 36905341 parameters, 6652669 gradients
Convert model to Traced-model…
traced_script_module saved!
model is traced!

/home/rebotnix/.local/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3483.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Traceback (most recent call last):
File “detect.py”, line 196, in
detect()
File “detect.py”, line 92, in detect
pred = non_max_suppression(pred, opt.conf_thres, opt.iou_thres, classes=opt.classes, agnostic=opt.agnostic_nms)
File “/home/rebotnix/Documents/yolov7/utils/general.py”, line 684, in non_max_suppression
i = torchvision.ops.nms(boxes, scores, iou_thres) # NMS
File “/home/rebotnix/.local/lib/python3.8/site-packages/torchvision/ops/boxes.py”, line 40, in nms
_assert_has_ops()
File “/home/rebotnix/.local/lib/python3.8/site-packages/torchvision/extension.py”, line 48, in _assert_has_ops
raise RuntimeError(
RuntimeError: Couldn’t load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check GitHub - pytorch/vision: Datasets, Transforms and Models specific to Computer Vision for the compatibility matrix. Please check your PyTorch version with torch.version and your torchvision version with torchvision.version and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install.

What do you think?

All the best,

Simon

1 Like
5

Installing torchvision from pip won’t have CUDA enabled and may run into other errors, as the torchvision wheels on PyPi for aarch64 architecture are built for generic ARM platforms and not JetPack. Instead, uninstall it and try again building it from source (linked to above)

Or if you keep having problems, you can use one of the containers and simply mount in your YOLO project code.

2 Likes
6

Thanks @dusty_nv,

That has done the job and the model is running. Getting around 50ms inference so will be interesting to compare to what other people are getting.

7

I also porting my yolov5 project on Jetson Orin NX 16GB development kit platform. Based on the instruction of pytorch.org pytorch install for previous versions, i use the following command to install toch and torchvision for CUDA 11.3

conda install pytorch==1.12.1 torchvision==0.13.1 cudatoolkit=11.3  -c pytorch -c conda-forge

The installation is success but the torch.cuda.is_available return false, so the inference result is not as expected. After using jtop to monitor, you can find the GPU utilization is also zero. Hence, the CPU version is installed in this way.
Here is the result of collect_env.py which is provided by pytorch utility to collect env:

PyTorch version: 1.12.1
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (aarch64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31  

Python version: 3.8.16 (default, Mar  2 2023, 03:16:31)  [GCC 11.2.0] 
(64-bit runtime)
Python platform: Linux-5.10.104-tegra-aarch64-with-glibc2.26
Is CUDA available: False
CUDA runtime version: 11.4.315
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/aarch64-linux-gnu/libcudnn.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8.6.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A 
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       2
Vendor ID:                       ARM
Model:                           1
Model name:                      ARMv8 Processor rev 1 (v8l)
Stepping:                        r0p1
CPU max MHz:                     1984.0000
CPU min MHz:                     115.2000
BogoMIPS:                        62.50
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        2 MiB
L3 cache:                        4 MiB
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store 
Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 
atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp uscat 
ilrcpc flagm

Versions of relevant libraries:
[pip3] numpy==1.21.0
[pip3] torch==1.12.1
[pip3] torchvision==0.13.1a0
[conda] cudatoolkit               11.3.1              h89cd5c7_10    conda- 
forge
[conda] numpy                     1.21.3           py38hb9da153_0    conda- 
forge
[conda] pytorch                   1.12.1          cpu_py38ha09e9da_1  
[conda] torchvision               0.13.1          cpu_py38heb4ea19_0

Based on the previous discussion, besides of build from source and container, is there any success way from conda install?

8

Hi,

Could you send a link to how you installed Conda? I didn’t realise it was supported for aarch64 systems and I think it’s the best way to manage environments.

You will need to check the version of JetPack you have for PyTorch version needed:
https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform-release-notes/pytorch-jetson-rel.html#pytorch-jetson-rel

However, I found building PyTorch from source to be a lot easier than anticipated and it was working straight away after following these instructions.
https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html

One thing I would say is that I am getting 50ms ish inference when running in max power mode. I was hoping for faster speeds. Some people in other forums suggest using TensorRT. I will be trying that at some point to see if I can get faster speeds.

All the best,

Simon

9

@user101843 the upstream PyTorch wheels that these install from PyPi (i.e. via pip install torch or conda install torch) are not built for JetPack or with CUDA support. Instead, please install the PyTorch wheels from the link that @simonnv provided or from this post:

Unlike those on PyPi, these PyTorch wheels were built for JetPack and with CUDA/cuDNN enabled.

10

Update for test result of yolov5 on jetson Orin NX 16GB with Jetpack 5.1.1.
Following the instruction of nvidia developer forums, I select Pytorch 1.13.0 and tochvision 0.13.0 pre-built wheel. Except small modification of torch upsampling.py and torchvision qnms_kernel.cpp, yolov5 is run as expected performance on the Orin NX 16GB with torch.cuda.is_available true.
Thanks @dusty_nv @simonnv
ps. My yolov5 project code is on free space vehicle license plate detection(lpd) and recognition(lpr). Before deploying TensorRT, the lpd in 1920x1080 frame is about 0.16 seconds and lpr is about 0.056 seconds in my case.

1 Like
11

Thanks for letting us know @user101843, glad you were able to get it working!

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.