Problem in using cuDNN during inference with YOLOv4-CSP

Hi!

I am trying to run a YOLOv4-CSP object detector on my Jetson AGX Xavier. The information is as follows:

CUDA Version: Release 11.4 (per nvcc --version)
OpenCV Version: 4.5.4

GitHub for Object Detector : Link

During setting up the object detector per the guide on GitHub (buidling Darknet) I set the following parameters in the Makefile:

GPU=1
CUDNN=1
CUDNN_HALF=1
OPENCV=1

Also based on the guide I set up the model to use Jetson AGX Xavier GPU

# Jetson XAVIER
ARCH= -gencode arch=compute_72,code=[sm_72,compute_72]

When running inference on the model however, it appears that I am not using the cuDNN_half optimization and the value for it is set to 0. As seen in the following code copied from terminal.

CUDA-version: 11040 (11040), cuDNN: 8.3.2, GPU count: 1  
 OpenCV version: 4.5.4
Demo
 0 : compute_capability = 720, cudnn_half = 0, GPU: Xavier 
net.optimized_memory = 0 
mini_batch = 1, batch = 1, time_steps = 1, train = 0 
   layer   filters  size/strd(dil)      input                output
   0 Create CUDA-stream - 0 

Also, using sudo jtop to get info on the system shows the following:

I have seen in videos online that cuDNN value was set to 1 during inference on Jetson. During inference model (YOLOv4-CSP.weights) the model achieves 12-15 FPS. But I am curious how can I use cuDNN and if cuDNN is corrcectly set up on my Jetson. Using whereis command, here is the location of CUDA and cuDNN libraries:

cuda: /usr/local/cuda
cudnn: /usr/include/cudnn.h

Hi,

Is your inference run on half mode?
The half mode should be specified in the source code.

Thanks.

I understood as this, when I change the values in the Makefile and set CUDNN_HALF=1 then detector should use it. But can you point me to which part of the source code I should modify to get half mode in inference?

Hi,

Do you use the script below?

If yes, maybe you can try to call model.half() to see if it can be converted into half automatically.

Thanks.

My question was based on using the Darknet implementation from https://github.com/AlexeyAB/darknet.
I will switch and try the script you posted.
Thanks!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.