Request for Support: Integrating P2PNet Model in DeepStream Framework

[bimalsagarwork]

I am seeking assistance with integrating the P2PNet model into the DeepStream framework. P2PNet is a straightforward point-based framework designed for both crowd counting and individual localization. However, I’ve noticed that the current Python applications in DeepStream do not support crowd counting models, and the P2PNet model operates on a different pipeline compared to the standard object detection flow.

Specifically, Gst-nvinfer currently supports the following network types:

• Multi-class object detection
• Multi-label classification
• Semantic segmentation
• Instance segmentation

After reviewing the custom model integration section in the DeepStream documentation, I found that the configuration parameters do not include those required for the P2PNet model.

I would greatly appreciate any support or guidance on how to integrate our P2PNet model using DeepStream plugins.

Thank you in advance for your help.

inception

[Fiona.Chen]

@bimalsagarwork

Specifically, Gst-nvinfer currently supports the following network types:

• Multi-class object detection
• Multi-label classification
• Semantic segmentation
• Instance segmentation

There is another type - Others(network-type=100)

@bimalsagarwork

After reviewing the custom model integration section in the DeepStream documentation, I found that the configuration parameters do not include those required for the P2PNet model.

The “network-type=100” is always been used with “output-tensor-meta=1”. Gst-nvinfer — DeepStream documentation 6.4 documentation.
There is sample /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-infer-tensor-meta-test for how to use the APIs.

You need to customize the postprocessing and meta data for your special models.

[bimalsagarwork]

Thank you for your response.

I require further assistance regarding the configuration of model output metadata within the pgie_src_pad_buffer_probe function.

The model output format is as follows:
outputs=
{
‘pred_logits’: tensor([[,,…]], grad_fn=),
‘pred_points’: tensor([[,,…]], grad_fn=)
}

Given that we need to extract points from the output, we can use the following approach:

threshold = 0.5
outputs_scores = torch.nn.functional.softmax(outputs[‘pred_logits’], -1)[:, :, 1][0]
outputs_points = outputs[‘pred_points’][0]
points = outputs_points[outputs_scores > threshold]

Could you please provide guidance on how to extract this data within the probe function?

inception

[Fiona.Chen]

@bimalsagarwork

Could you please provide guidance on how to extract this data within the probe function?

Just implement the same algorithm in c/c++ language. Please google the c/c++ implementation of these pytorch functions.

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks