Filtering objects with nvdsanalytics to reduce messages sent to the broker

[dddd713]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only) NA
• TensorRT Version 7.2.2
• NVIDIA GPU Driver Version (valid for GPU only) 460
• Issue Type( questions, new requirements, bugs) question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi,

with DS4.0 we wanted to filter out detections based on some roi https://forums.developer.nvidia.com/t/how-to-filter-messages-before-sending-to-kafka-server/119027.

With 5.1 I’m wondering if something similar is possible using nvdsanalytics? I have looked at deepstream-occupancy-analytics and deepstream-analytics-test but they would generate as many messages as there are objects detected and only append number of objects entering/exiting or items within roi.

How can I access information that a particular object is within nvdsanalytics-defined roi and send the message only then?

And when defining multiple roi-RFn for one stream, would it be possible to know in which roi the object was detected?

[Amycao]

@dddd713

With 5.1 I’m wondering if something similar is possible using nvdsanalytics? I have looked at deepstream-occupancy-analytics and deepstream-analytics-test but they would generate as many messages as there are objects detected and only append number of objects entering/exiting or items within roi.

the message generation called from msg2p_generate_multiple or msg2p_generate based on flag multiplePayloads in nvmsgconv plugin code sources/gst-plugins/gst-nvmsgconv/gstnvmsgconv.c and these two functions will call into generate_deepstream_message_minimal or generate_schema_message based on payload type is NVDS_PAYLOAD_DEEPSTREAM_MINIMAL or NVDS_PAYLOAD_DEEPSTREAM. you can comment code like generate_sensor_object, anything you do not want, if you use payload type NVDS_PAYLOAD_DEEPSTREAM, or does NVDS_PAYLOAD_DEEPSTREAM_MINIMAL meet your requirements, it’s using minimal DeepStream schema.

How can I access information that a particular object is within nvdsanalytics-defined roi and send the message only then?
And when defining multiple roi-RFn for one stream, would it be possible to know in which roi the object was detected?

You can refer app deepstream-nvdsanalytics-test, function about how to extract nvanalytics metadata

See inline bold type.

[dddd713]

Thanks for reaching out, but I don’t think that either of the answers in bold really is what I’m looking for.

I already have the message schema the way I want it to be, that was not an issue.

The thing is that right now, for every object that was detected, in the frame, I have a message. Similarly, analytics-test would tell me there are x objects in roi and y objects total. Instead I’m trying to figure out how to know that object a is within roi but object b is outside of roi so that meta for object b can be removed from the list.
I don’t want to sent messages for objects outside of ROI.

[Amycao]

“nvmsgconv” plugin uses NVDS_EVENT_MSG_META type of metadata from the buffer and generates the “DeepStream Schema” payload in Json format. Static properties of schema are read from configuration file in the form of key-value pair. Check dstest5_msgconv_sample_config.txt for reference. Generated payload is attached as NVDS_META_PAYLOAD type metadata to the buffer.

“nvmsgbroker” plugin extracts NVDS_META_PAYLOAD type of metadata from the buffer and sends that payload to the server using protocol adaptor APIs.

So you can limit generating NvDsEventMsgMeta by when condition objects in ROI met.