I am currently trying to draw masks in addition to the bounding boxes using the Mask-RCNN custom app given here: https://github.com/NVIDIA-AI-IOT/deepstream_4.x_apps/tree/master/nvdsinfer_customparser_mrcnn_uff and the default Deepstream App. However the default application does not support visualizing the masks or extracting the mask data with the default metadata output. I am able to dump the masks separately using the nvdsinfer_custombboxparser_mrcnn_uff.cpp to text files but not with the default Deepstream app.
Is there any way we can modify the existing deepstream app to draw the masks on the output videos and get the masks as part of the deepstream metadata?
I can potentially use polygons to drawing. However, the major issue currently faced by us is how do we pass on the mask data to the deepstream application?
One potential way of solving this could be by modifying the following function in the file nvdsinfer_custombboxparser_mrcnn_uff.cpp (I’ve removed some code intentionally here for better readability) :
In line 14 above we are making an object of type NvDsInferObjectDetectionInfo which is then pushed into objectList, I’m assuming this vector is then passed on to the main deepstream app. Would it be possible to modify. the struct NvDsInferObjectDetectionInfo in nvdsinfer.h and add in another variable e.g. an array or a vector?
typedef struct
{
/** ID of the class to which the object belongs. */
unsigned int classId;
/** Horizontal offset of the bounding box shape for the object. */
unsigned int left;
/** Vertical offset of the bounding box shape for the object. */
unsigned int top;
/** Width of the bounding box shape for the object. */
unsigned int width;
/** Height of the bounding box shape for the object. */
unsigned int height;
/** Object detection confidence. Should be a float value in the range [0,1] */
float detectionConfidence;
} NvDsInferObjectDetectionInfo;
I did adding an extra integer variable in this struct but I ran into issues. The code compiles fine but when I run the deepstream app I end up getting an Segmentation Fault. Any leads as to why changing the struct leads to this error?
Or maybe there is another approach we could use to get the masks to the main deepstream app in addition to the bounding box coordinates, classIds etc.