How to send nvdsanalytics plugin data to azure

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I am trying to send the output of the new analytics plugin (NvDsAnalyticsFrameMeta) to azure using the conv and broker plugins.

I am not sure how to attach the NvDsAnalyticsFrameMeta to the extMsG, does it need to be as a custom object? I read this: You can add a group of custom objects to the NvDsEventMsgMeta structure in the extMsg field and specify their size in the extMsgSize field. The meta copy (copy_func) and free (release_func) functions must handle the custom fields accordingly.

So I would need to create a copy and release function? Do i get the analytics Frame Meta by using NvDsUserMeta *user_event_meta = nvds_acquire_user_meta_from_pool (batch_meta);
then user_event_meta->base_meta.meta_type = NVDS_USER_FRAME_META_NVDSANALYTICS;

Any help would be greatly appreciated.

Thanks

3

Hi,

Have you checked our sample apps?
The following sample should be able to give you some idea:

apps/sample_apps/deepstream-test4
apps/sample_apps/deepstream-test5

Thanks

4

Yeah so I have had a look at both test apps. I guess I am struggling to understand how i link the NvDsAnalyticsFrameMeta to NvDsEventMsgMeta so it can be sent to azure. The test apps show how to send person and vehicle type objects and the add custom data to these types which are supported by the NvDsEventMsgMeta. The deepstream_gst_metadata app shows how to create a new custom object but not how to attach it to the extMsg field of NvDsEventMsgMeta.

6

Still having trouble with the above, from the above examples and everything I have read I am not sure if have to create a new custom object as the FrameMeta already has the analytic information.

8

Hi,

Unfortunately, there is no exist sample demonstrates this usage.
So you will need to create a Deepstream plugin on your own.

Please check deepstream-test5 and deepstream-nvdsanalytics-test to get some information.

Thanks.

9

Hi,

In general, you will need to generate custom payload using msg2p lib interface.
Please refer to our message converter plugin here:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.3.14.html#

Thanks.

11

I was also confused about this at first (we need to include analytics data as well, though we’re ultimately using AMQP). I included the solution that worked for me in case it helps anyone. It uses the approach found in the deepstream-test4 sample app (it’s also discussed briefly in the manual on this page).

The issue was that the data we want to create the custom payload with was not available to us during payload creation, and we were struggling to figure out how to make it available. The nvds_msg2p_generate() function call in the class that implements the msg2p interface needs to grab its data from the provided NvDsEvent object, or rather its NvDsEventMsgMeta field. We were supposed to have already attached any custom data we wanted to include to the extMsg pointer inside NvDsEventMsgMeta. But we weren’t sure when/where/how we should be adding stuff to extMsg.

The steps to do that seem to be as follows:

  1. Create a probe on the sink pad of the pipeline element where it’s at a point where it’s got all the data you’d want to include. In this probe you can grab all the data currently in the buffer and choose what to send.
static GstPadProbeReturn
custom_metadata_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data) {
  GstBuffer *buf = (GstBuffer *) info->data;
  NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);
  ...
}
  1. In the probe function, create your NvDsEventMsgMeta struct and add your custom metadata via the extMsg and extMsgSize fields.
NvDsEventMsgMeta *msg_meta = (NvDsEventMsgMeta *) g_malloc0 (sizeof (NvDsEventMsgMeta));
msg_meta->extMsg = ... ;
msg_meta->extMsgSize = ... ;
...
  1. In the probe function, grab the NvDsUserMeta struct from the batch pool and add your custom metadata to it. During this you also specify the copy and release functions for your custom metadata.
NvDsUserMeta *user_event_meta = nvds_acquire_user_meta_from_pool (batch_meta);
if (user_event_meta) {
  user_event_meta->user_meta_data = (void *) msg_meta;
  user_event_meta->base_meta.meta_type = NVDS_EVENT_MSG_META;
  user_event_meta->base_meta.copy_func = (NvDsMetaCopyFunc) meta_copy_func;
  user_event_meta->base_meta.release_func = (NvDsMetaReleaseFunc) meta_free_func;
  nvds_add_user_meta_to_obj(obj_meta, user_event_meta);
} else {
  g_print ("Error in attaching event meta to buffer\n");
}

Instead of nvds_add_user_meta_to_obj(), you can also nvds_add_user_meta_to_frame() or nvds_add_user_meta_to_batch().

  1. Create the copy and release functions for your custom metadata.
static gpointer meta_copy_func (gpointer data, gpointer user_data) {
  ...
}
static void meta_free_func (gpointer data, gpointer user_data) {
  ...
}

Again, the deepstream-test4 sample app was used as a reference for all this, so you can check that for sample code.

12

Correction, use nvds_add_user_meta_to_frame() instead of nvds_add_user_meta_to_obj().

It seems like the provided AMQP implementation won’t search through object user metadata, only frame user metadata, so if you attach it to object user metadata the event message won’t get picked up for publishing.