Object detection pre-trained model inference issue in deepstream

[aniketrohara]

as per your suggestions i have added the batch-size dimension to model and now i have observed that the output in the bbox layer are distances from anchor centers to bbox edges where anchors are generated in a mesh grid (in onnxruntime code it was done using function numpy.mgrid()) so tell me how to do this in deepstream

[yuweiw]

@aniketrohara

now i have observed that the output in the bbox layer are distances from anchor centers to bbox edges where anchors are generated in a mesh grid

Can you describe in detail what you want to do? These algorithms need to be implemented by yourself in the post-processing.

[aniketrohara]

In post processing when i am returning the parameters in custom parser i am getting the output as expected but the issue is that the performance is less than the onnxruntime and the number of bounding boxes per frame i am getting in deepstream output are less than the onnxruntime so , i want you to help me in getting atleast similar performance.

i have uploaded both the videos in the drive link shared previously, demo_out.avi is output of onnxruntime and out_new.mp4 is output of deepstream with all the necessary config parameters

[yuweiw]

Please refer to our FAQ to tune the accuracy parameters.

[aniketrohara]

i referred the FAQ’s and i wanted to know that as my model expects 640x640 input dimensions and i am using the “maintain-aspect-ratio” flag for nvinfer so i guess this will not give appropriate input dimensions for inference, am i right?..and also i wanted to know how the “symmetric-padding” flag will affect the processing, like should i use it for my model as square shaped input image is expected?

[yuweiw]

@aniketrohara

this will not give appropriate input dimensions for inference, am i right?

No. This will give the right dimesions, but the scale pattern is a bit different. You need to set the scaling parameters here according to the that when your model is trained.

@aniketrohara

and also i wanted to know how the “symmetric-padding” flag will affect the processing, like should i use it for my model as square shaped input image is expected?

You can refer to our guide. symmetric-padding indicates whether to pad image symmetrically while scaling input.

As we said in the FAQ, when deploying a ONNX model to DeepStream with nvinfer plugin, confirm below nvinfer parameters are set correctly to align with the corresponding settings in Training.

[aniketrohara]

hey thanks for your help, i have an update i am using the following yolo code for reference and i modified it a little bit for my scrfd model , and i have been able to get pretty accurate bounding boxes but for some reason keypoints are not visible on my output video so can you tell me what could be the possible reason for this?

marcoslucianops/DeepStream-Yolo-Face/blob/master/nvdsinfer_custom_impl_Yolo_face/nvdsparseface_Yolo.cpp

i am using deepstream-app -c command to run the model

[yuweiw]

Deepstream-app cannot draw the keypoint by default. You need to modify that yourself, you can refer to our deepstream_faciallandmark_app.cpp to learn how to draw the keypoints.

[aniketrohara]

Where in the deepstream app i need to modify? please guide the function or section. Also which portion of the code I need to go through in faciallandmark app?

[yuweiw]

@aniketrohara

Where in the deepstream app i need to modify?

sources\apps\sample_apps\deepstream-app\deepstream_app.c
static GstPadProbeReturn
gie_processing_done_buf_prob (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{

@aniketrohara

Also which portion of the code I need to go through in faciallandmark app?

static GstPadProbeReturn
tile_sink_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)

[aniketrohara]

i got the keypoints, thanks for your help !!!