Output instance mask

Description

I had a probem when working with my simple deepstream app (streammux → nvpreprocess → pose_engine_nvinfer → my_custom_pose_decode_mask_to_keypoint_plugin → …)
When I use roi param with full HD resolution, the output is good but if I change roi param to another resolution, the output is a bit weird. The bounding boxes are not drawn well.
I have a question. Is the instance mask scaled to input resolution or to ROI resolution? Could you guys give me some info?

Thanks

Environment

Deepstream 6.1.1 docker

Relevant Files

streamux: 1920x1080 - roi-params-src-0=0;0;1920;1080


streamux: 1920x1080 -roi-params-src-0=0;0;1080;720
image

Hi,

We are moving this post to the Deepstream forum to get better help.

Thank you.

  1. what is the model’s input and output? there is a PeopleSegNet model peoplesegnet, it will outputs bounding-box coordinates and segmentation mask for each detected person in the input image, why is there no segment mask in your pictures?
  2. About “Is the instance mask scaled to input resolution or to ROI resolution”, from the code attach_metadata_detector, which is opensource, the bbox will be converted to a new coordinate in the original frame, but mask dose not do this conversion.
1 Like

@fanzh
Thank for your response,
From my side,

  1. my model output only has cif and caf, there are no bounding-box coordinates. However, I built a custom plugin to convert cif and caf to keypoints and then construct the bounding box coordinates from those key points.
  2. Thank for your confirmation. I debugged the nvinfer and I also realized that mask on inferenced ROI does not do the conversion to the original frame. In my case, what is the solution here? I should write a conversion code by myself, right? It there any example code to convert model output mask of ROI to original frame coordinates in deepstream source code? Thanks

yes, please refer to bbox coordinates conversation logics in attach_metadata_detector.

@fanzh
actually, I cannot resize mask instance output to original coordinates. The bbox coordinates conversion does not work with segmentation model.

I used preprocess + peoplenet model to test this case, I found the masks are correct, I did not modify any code.

1 Like

thank you so much, maybe bug is from my plugin code.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.