How to add custom data to NvDsObjectMeta

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi guys,I am trying add my custom data to NvDsObjectMeta.It is an angle of each detected object in each frame.With this opinion,I have seen some posts in the forum，and I see the code below:

void
add_meta_to_buffer (GstBuffer * buf, gchar * meta)
{
  NvDsBatchMeta *batch_meta;
  NvDsUserMeta *user_meta;
  batch_meta = gst_buffer_get_nvds_batch_meta (buf);
  user_meta = nvds_acquire_user_meta_from_pool (batch_meta);
  if (user_meta) {
    NvDsEventMsgMeta *msg_meta =
        (NvDsEventMsgMeta *) g_malloc0 (sizeof (NvDsEventMsgMeta));
    msg_meta->otherAttrs = meta;
    msg_meta->objectId = g_strdup ("custom");
    user_meta->user_meta_data = (void *) msg_meta;
    user_meta->base_meta.batch_meta = batch_meta;
    user_meta->base_meta.meta_type = NVDS_EVENT_MSG_META;
    user_meta->base_meta.copy_func = (NvDsMetaCopyFunc) custom_meta_copy_func;
    user_meta->base_meta.release_func =
        (NvDsMetaReleaseFunc) custom_meta_free_func;
    nvds_add_user_meta_to_batch (batch_meta, user_meta);
  }
}

The description is :
NvDsEventMsgMeta is a type of DeepStream user meta and there is a NvDsUserMetaList in the batch, frame and object levels of DeepStream metadata, so you can add message meta per-batch, per-frame, or per-object. We use it on the batch level adding a JSON as message meta

As you can see,it is on the batch level to add data.So what if it is on the object level?My initial opinion is :

for (NvDsMetaList * l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) {
      NvDsFrameMeta *frame_meta = l_frame->data;
      for (NvDsMetaList * l_obj = frame_meta->obj_meta_list; l_obj != NULL;
        l_obj = l_obj->next) {
        NvDsObjectMeta *obj = (NvDsObjectMeta *) l_obj->data;
        user_meta = nvds_acquire_user_meta_from_pool (obj);
        if (user_meta) {
        NvDsEventMsgMeta *msg_meta =(NvDsEventMsgMeta *) g_malloc0 (sizeof (NvDsEventMsgMeta));
        msg_meta->otherAttrs = meta;
        msg_meta->objectId = g_strdup ("custom");
        user_meta->user_meta_data = (void *) msg_meta;
        user_meta->base_meta.batch_meta = batch_meta;
        user_meta->base_meta.meta_type = NVDS_EVENT_MSG_META;
        user_meta->base_meta.copy_func = (NvDsMetaCopyFunc) custom_meta_copy_func;
        user_meta->base_meta.release_func =(NvDsMetaReleaseFunc) custom_meta_free_func;
        nvds_add_user_meta_to_batch (obj, user_meta);
  }

So as you can see the only difference is to replace batch_meta to obj,namely the obj_meta.The angle will be the meta parameter in the above codes.I wonder if my opinion is practical

3

Yes, you are right.There are some code snippets in the sample application.
Such as the following code in /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvtracker/nvtracker_proc.cpp

if (m_Config.outputConvexHull && pTrackedObj->convexHull.numPointsAllocated != 0)
  {
      NvDsUserMeta *user_meta = nvds_acquire_user_meta_from_pool (pBatchMeta);
      NvDsObjConvexHull* pConvexHull = new NvDsObjConvexHull;
      pConvexHull->list = new int [pTrackedObj->convexHull.numPointsAllocated * 2];
      for (uint32_t i=0; i < pTrackedObj->convexHull.numPoints; i++)
      {
        pConvexHull->list[2*i] = (int) (pTrackedObj->convexHull.list[2*i] * scaleWidth);
        pConvexHull->list[2*i+1] = (int) (pTrackedObj->convexHull.list[2*i+1] * scaleHeight);
      }
      pConvexHull->numPoints = pTrackedObj->convexHull.numPoints;
      pConvexHull->numPointsAllocated = pTrackedObj->convexHull.numPointsAllocated;
      user_meta->user_meta_data = (void *)pConvexHull;
      user_meta->base_meta.meta_type = (NvDsMetaType) NVDS_OBJ_IMAGE_CONVEX_HULL;
      user_meta->base_meta.copy_func = (NvDsMetaCopyFunc) copy_nvtracker2_obj_convex_hull_meta;
      user_meta->base_meta.release_func = (NvDsMetaReleaseFunc) free_nvtracker2_obj_convex_hull_meta;
      nvds_add_user_meta_to_obj (pObjectMeta, user_meta);
  }
4

Thanks,can I delete some lines of the codes I provide?For example:

user_meta->base_meta.copy_func = (NvDsMetaCopyFunc) custom_meta_copy_func;
user_meta->base_meta.release_func =(NvDsMetaReleaseFunc) custom_meta_free_func;

Because I don’t define such custom function,if the two lines are deleted,any influence?
Beside, about this line:
user_meta->base_meta.batch_meta = batch_meta;
Is it necessary to replace the batch_meta to obj?

5

No, this code must be exist. These callbacks will be called when meta data is copied or released between elements.

Please refer to the example above

6

Thanks,could you please give me some code snippet about these callbacks?
Besides,according to your codes,this line:

user_meta->base_meta.batch_meta = batch_meta;

seems to be unnecessary?I don’t see similar code in your codes

7

This depends on how you use it.

If you want to find the corresponding batchmeta when accessing objmeta, this is necessary.

Otherwise, it is not necessary.

8

Thanks,could you please give me some code snippet about these callbacks?namely the :

user_meta->base_meta.copy_func = (NvDsMetaCopyFunc) custom_meta_copy_func;
user_meta->base_meta.release_func =(NvDsMetaReleaseFunc) custom_meta_free_func;

custom_meta_copy_func and custom_meta_free_func.I don’t know how they are defined.

9

You can refer to /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-user-metadata-test/deepstream_user_metadata_app.c. Here is code snippet.

static gpointer copy_user_meta(gpointer data, gpointer user_data)
{
  NvDsUserMeta *user_meta = (NvDsUserMeta *)data;
  gchar *src_user_metadata = (gchar*)user_meta->user_meta_data;
  gchar *dst_user_metadata = (gchar*)g_malloc0(USER_ARRAY_SIZE);
  memcpy(dst_user_metadata, src_user_metadata, USER_ARRAY_SIZE);
  return (gpointer)dst_user_metadata;
}

/* release function set by user. "data" holds a pointer to NvDsUserMeta*/
static void release_user_meta(gpointer data, gpointer user_data)
{
  NvDsUserMeta *user_meta = (NvDsUserMeta *) data;
  if(user_meta->user_meta_data) {
    g_free(user_meta->user_meta_data);
    user_meta->user_meta_data = NULL;
  }
}

This is another sample.