YOLOv7(8,9) on Jetson Orin Nano 4GB. RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)`

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
1

Hi, all!

I am trying to use YOLOv7 for object detection. But when I run this code I get CUDA error:
python3 detect.py --weights yolov7.pt --conf 0.25 --img-size 640 --source resized.jpg --device 0

Namespace(agnostic_nms=False, augment=False, classes=None, conf_thres=0.25, device=‘0’, exist_ok=False, img_size=640, iou_thres=0.45, name=‘exp’, no_trace=False, nosave=False, project=‘runs/detect’, save_conf=False, save_txt=False, source=‘resized.jpg’, update=False, view_img=False, weights=[‘yolov7.pt’]) YOLOR 🚀 v0.1-128-ga207844 torch 2.1.0a0+41361538.nv23.06 CUDA:0 (Orin, 3310.73828125MB) Fusing layers… Traceback (most recent call last): File “detect.py”, line 196, in detect() File “detect.py”, line 34, in detect model = attempt_load(weights, map_location=device) # load FP32 model File “/home/eventjetson/Raf/yolov7/models/experimental.py”, line 253, in attempt_load model.append(ckpt[‘ema’ if ckpt.get(‘ema’) else ‘model’].float().fuse().eval()) # FP32 model File “/home/eventjetson/Raf/yolov7/models/yolo.py”, line 703, in fuse m.conv = fuse_conv_and_bn(m.conv, m.bn) # update conv File “/home/eventjetson/Raf/yolov7/utils/torch_utils.py”, line 194, in fuse_conv_and_bn fusedconv.weight.copy_(torch.mm(w_bn, w_conv).view(fusedconv.weight.shape)) RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle) .

I have:

import torchvision >>> import torch >>> print(torch.version) 2.1.0a0+41361538.nv23.06 >>> print(torchvision.version) 0.16.0 >>> print(torch.cuda.is_available()) True >>>

Package: nvidia-jetpack Version: 5.1.1-b56 Architecture: arm64 Maintainer: NVIDIA Corporation Installed-Size: 194 Depends: nvidia-jetpack-runtime (= 5.1.1-b56), nvidia-jetpack-dev (= 5.1.1-b56) Homepage: Jetson - Embedded AI Computing Platform | NVIDIA Developer Priority: standard Section: metapackages Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_5.1.1-b56_arm64.deb Size: 29304 SHA256: 7b6c8c6cb16028dcd141144b6b0bbaa762616d0a47aafa3c3b720cb02b2c8430 SHA1: 387e4e47133c4235666176032af0f2ec86461dbb MD5sum: 0a8692031bf35cc46f7a498e2937bda9 Description: NVIDIA Jetpack Meta Package Description-md5: ad1462289bdbc54909ae109d1d32c0a8

I tried different torchvivion version: 0.16.0, 0.16.1, 0.16.2.
And tried these versions of torch: 2.1.1 and 2.0.0

I installed these libraries according this manual:

How can I tackle with this CUDA Error?

2

Hello,

Welcome to the NVIDIA Developer forums! Your topic will be best served in the Jetson category.

I will move this post over for visibility.

Cheers,
Tom

4

Hi,

Based on the below issue, the error can appear when the device is running out of memory.

Thanks.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.