Adding cv::Mat object to NvDsUserMeta

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello, I am looking for a solution to add opencv cv::Mat object to NvDsUserMeta and access it from buffer probes. I have seen an example to encode jpg image to user meta in the example deepstream-image-meta-test . I have also found an example to blur detected objects using dsexample plugin on deepstream-opencv-test. Now all I want is to add the cv::Mat object from dsexample plugin to be encoded and added to the pipeline to be accessed from other probes.

For this, I have added a workaround in the gstdsexample.cpp file:

void *set_metadata_ptr(cv::Mat fram)
{
  int i = 0;
 
    cv::Mat *user_metadata = (cv::Mat*)g_malloc0(1000);

    fram.copyTo(*user_metadata);

  return (void *)user_metadata;
}

static void
attach_metadata_object (GstDsExample * dsexample, NvDsObjectMeta * obj_meta,
    DsExampleOutput * output, cv::Mat newFrame)
{

NvDsUserMeta *user_meta = NULL;
NvDsMetaType user_meta_type = NVDS_USER_FRAME_META_EXAMPLE;
  if (output->numObjects == 0)
    return;
  NvDsBatchMeta *batch_meta = obj_meta->base_meta.batch_meta;

  NvDsClassifierMeta *classifier_meta =
    nvds_acquire_classifier_meta_from_pool (batch_meta);

  classifier_meta->unique_component_id = dsexample->unique_id;

  NvDsLabelInfo *label_info =
    nvds_acquire_label_info_meta_from_pool (batch_meta);
  g_strlcpy (label_info->result_label, output->object[0].label, MAX_LABEL_SIZE);
  nvds_add_label_info_meta_to_classifier(classifier_meta, label_info);
  nvds_add_classifier_meta_to_object (obj_meta, classifier_meta);


user_meta = nvds_acquire_user_meta_from_pool(batch_meta);

        /* Set NvDsUserMeta below */
       user_meta->user_meta_data = (void *)set_metadata_ptr(newFrame);
        user_meta->base_meta.meta_type = user_meta_type;
        user_meta->base_meta.copy_func = (NvDsMetaCopyFunc)copy_user_meta;
        user_meta->base_meta.release_func = (NvDsMetaReleaseFunc)release_user_meta;

        /* We want to add NvDsUserMeta to frame level */
        nvds_add_user_meta_to_obj(obj_meta, user_meta);


  nvds_acquire_meta_lock (batch_meta);
  NvOSD_TextParams & text_params = obj_meta->text_params;
  NvOSD_RectParams & rect_params = obj_meta->rect_params;

  
  nvds_release_meta_lock (batch_meta);
}

This works somewhat, but it slows down the pipeline, as I am not sure if its correctly attaches the metadata formatted or encoded properly. As there is no such usrMetaData->base_meta.meta_type for cv::Mat by default, I need help regarding the correct metadata attachment as like in the example deepstream-image-meta-test

3

The sample of user metadata is deepstream-user-metadata-test. The codes you have posted seem OK.