YOLO v3 crashed on Nvidia Jetson Xavier NX

Hi:

I have Nvidia Jetpack 4.5.1 installed on my Jetson Xavier NX dev kit. It has CUDA 10.2, CUDNN 8.0. I am trying to run YOLO v3 (GitHub - pjreddie/darknet: Convolutional Neural Networks) If I build the CPU version, I can run the demo. However, if I build the GPU version (GPU=1, CUDNN=0), it crashed when loading the weights as shown below. How can I fix it?

nvidia@XavierNx:~/apps/darknet$ ./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg
layer filters size input output
0 1 2 3 4 res 1 304 x 304 x 64 → 304 x 304 x 64
5 6 7 8 res 5 152 x 152 x 128 → 152 x 152 x 128
9 10 11 res 8 152 x 152 x 128 → 152 x 152 x 128
12 13 14 15 res 12 76 x 76 x 256 → 76 x 76 x 256
16 17 18 res 15 76 x 76 x 256 → 76 x 76 x 256
19 20 21 res 18 76 x 76 x 256 → 76 x 76 x 256
22 23 24 res 21 76 x 76 x 256 → 76 x 76 x 256
25 26 27 res 24 76 x 76 x 256 → 76 x 76 x 256
28 29 30 res 27 76 x 76 x 256 → 76 x 76 x 256
31 32 33 res 30 76 x 76 x 256 → 76 x 76 x 256
34 35 36 res 33 76 x 76 x 256 → 76 x 76 x 256
37 38 39 40 res 37 38 x 38 x 512 → 38 x 38 x 512
41 42 43 res 40 38 x 38 x 512 → 38 x 38 x 512
44 45 46 res 43 38 x 38 x 512 → 38 x 38 x 512
47 48 49 res 46 38 x 38 x 512 → 38 x 38 x 512
50 51 52 res 49 38 x 38 x 512 → 38 x 38 x 512
53 54 55 res 52 38 x 38 x 512 → 38 x 38 x 512
56 57 58 res 55 38 x 38 x 512 → 38 x 38 x 512
59 60 61 res 58 38 x 38 x 512 → 38 x 38 x 512
62 63 64 65 res 62 19 x 19 x1024 → 19 x 19 x1024
66 67 68 res 65 19 x 19 x1024 → 19 x 19 x1024
69 70 71 res 68 19 x 19 x1024 → 19 x 19 x1024
72 73 74 res 71 19 x 19 x1024 → 19 x 19 x1024
75 76 77 78 79 80 81 82 yolo
83 route 79
84 85 upsample 2x 19 x 19 x 256 → 38 x 38 x 256
86 route 85 61
87 88 Killed

Try rebuilding with CUDNN=1, I believe when enabled with GPU they use Cudnn as well.

Thank you

I also crashed when I compiled with CUDNN=1

It would be great if you can post a screenshot of your terminal. Thank you

Hello,
Did you modify this line:
ARCH= -gencode arch=compute_30,code=sm_30
-gencode arch=compute_35,code=sm_35
-gencode arch=compute_50,code=[sm_50,compute_50]
-gencode arch=compute_52,code=[sm_52,compute_52]

if not replace that with

ARCH=-gencode arch=compute_72,code=sm_72

This is just adding the proper CUDA arch number for Jetson Xavier NX. Try this if not please share the screenshot. Thanks

After change the line, still the same

pls share your Makefile

Makefile (3.0 KB)

Hi,

Usually ‘killed’ is caused by out-of-memory.
Could you monitor your device with tegrastats and share the output log with us?

Thanks.

I agree with AastaLLL,

  1. Makefile

GPU=1
CUDNN=1

  1. modify cfg/yolov3.cfg

batch=1
subvisions=1

  1. tiny

nvidia@XavierNx:~/apps/darknet$ ./darknet detect cfg/yolov3-tiny.cfg yolov3-tiny.weights data/dog.jpg

You are right. The cfg file was set to tranning mode. After I change it to

batch=1
subvisions=1

It worked fine.

Thank you very much.

However, if I build with

CUDNN=1

It crashed when loading the weight.

It worked if I use YoLo V3 tiny, even for CUDNN. But I really need YOLO, not the tiny.

^.^

G’luck

Has anyone run Yolo V3 inference with GPU=1 CUDNN=1 on Xavier NX successfully?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.