How to Extend BBox Area Detected by the PGIE?

[omniwise]

• Hardware Platform (Jetson / GPU): Both Jetson and dGPU
• DeepStream Version: 6.0.1 and 6.1.1
• JetPack Version (valid for Jetson only): 4.6
• TensorRT Version: 8.0.1
• NVIDIA GPU Driver Version (valid for GPU only): 515.76
• Issue Type( questions, new requirements, bugs): Question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi All,

Currently, I’m trying to extend or add margin to the cropped area created by the detection model which is PGIE. My pipeline looks like below:

[DECODING] → nvstreammux → nvinfer [PGIE] → nvtracker → nvinfer [SGIE] → nvvideoconvert → capsfilter → fakesink

As far as I know, DeepStream feeds the area detected area by the PGIE to one or multiple SGIEs for classification. However, I think that my classification models require a little more extended area than what the PGIE currently detects since they are producing the same classification output for every detection. My sample scenario involves detecting faces and feeding these areas to the age and gender classification models.

To solve this, I have implemented a callback (probe) function and did the following:

• Attached callback function to src pad of the PGIE (nvinfer) element
• Attached callback function to sink pad of the Tracker (nvtracker) element
• Attached callback function to sink pad of the SGIE (nvinfer) element

My callback function looks like the following:

def nvinfer_src_pad_buffer_probe(pad, info, u_data):
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        print("Unable to get GstBuffer ")
        return
    
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))

    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        l_obj = frame_meta.obj_meta_list

        while l_obj is not None:
            try:
                # Casting l_obj.data to pyds.NvDsObjectMeta
                obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break
            
            rect_params = obj_meta.rect_params

            # rect_params.left = int(rect_params.left - 20)
            # rect_params.top = int(rect_params.top - 20)
            # rect_params.width = int(rect_params.width + 40)
            # rect_params.height = int(rect_params.height + 40)

        return Gst.PadProbeReturn.OK

And the callback function call made as follows:

...
pgie_src_pad = pgie.get_static_pad("src")
if not pgie_src_pad:
    sys.stderr.write(" Unable to get src pad \n")
else:
    pgie_src_pad.add_probe(Gst.PadProbeType.BUFFER, nvinfer_src_pad_buffer_probe, 0)

Once I implement the whole idea like this, the code gets stuck at try-catch blocks. Is there any other possible way to manipulate detection areas of the PGIE before feeding them into SGIEs?

[yuweiw]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one.
Thanks

You idea is correct, just change the bbox size by the probe function. You should add some log and debug why it gets stuck.