YOLOv5 with Deepstream 5.1 having errors with batch-size > 1

kn1ght · May 13, 2021, 2:47am

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU RTX 2060
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only) N/A
• TensorRT Version 7.2.2
• NVIDIA GPU Driver Version (valid for GPU only) 460.32/CUDA11.2
• Issue Type( questions, new requirements, bugs)

Hi! So I have converted yolov5 from here https://github.com/ultralytics/yolov5 to onnx and wrote a custom parser to parse the output.

When I set the model batch size to 1, the output I get looks fine -

When I increase the batch size, I don’t get detections for certain streams -

I am not sure what can cause this issue as the parse-bbox-func-name is called per stream per frame.

mchi · May 13, 2021, 12:54pm

seems the images with even batch number get BBOX drawed, could you try other batch size,e.g. 6, 4, to see if it works like this.
If it’s, I suspect it’s caused by the post-processing.

Thanks!

kn1ght · May 13, 2021, 4:26pm

Thanks for reaching out so quickly @mchi

Yes, there is some weird pattern it follows with each batch number.

Batch Size 3 -

Batch Size 4 -

Also I am using deepstream-app for this so sometimes when I pause and resume, the streams it is processing shifts.

Under more investigation, I have no results in

float *out = (float *)outputLayersInfo[OutputLayerIndex].buffer

for the streams that don’t have an output. So the issue must be between the tensor output of yolov5 and my custom-parser.

I tried writing my own engine creation function and set it using engine-create-func-name. I also tried setting explicit batch like so -

const auto explicitBatch = 1U << static_cast<uint32_t>(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
auto network = builder->createNetworkV2(explicitBatch);

but I had the same result.

Is there a way I could debug this and get more information?

mchi · May 14, 2021, 5:10am

what do you mean “don’t have an output”? do you mean the buffer pointer is NULL or all data value in the buffer is zero?

kn1ght · May 14, 2021, 5:13am

Junk data which after processing and nms has no objects.

mchi · May 14, 2021, 5:17am

could you share the DS config files?

kn1ght · May 14, 2021, 5:29am

Attaching the model config and deepstream-app config. My config is slightly different modified for our product but it’s more or less similar.

config_infer_primary_yolov5s.txt (641 Bytes)
demo.json (12.5 KB)

mchi · May 15, 2021, 12:00pm

There is such issue iwht this Yolov4 from yolov4_deepstream.
https://drive.google.com/drive/folders/1LGxz_-xMqP-N8gk2rubcrELwBRfBVRNZ?usp=sharing

The instruction to run it are:

untar deepstream_yolov4.tgz under /opt/nvidia/deepstream/deepstream/sources/
$ deepstream-app -c deepstream_app_config_yoloV4.txt
check the output of yolov4.mp4

It has very similar nvinfer primary config.
So, I think you can make small changes to the config_infer_primary_yoloV4.txt and nvdsinfer_custom_impl_Yolo to run your yolov5 to check if it’s caused by the config.

kn1ght · May 15, 2021, 11:20pm

Thanks for the solution @mchi. I will try this. However, I have solved my issue by getting rid of onnx and directly converting .pt to tensorrt engine by using IPLuginV2Layer.