[Nano] Attaching custom metadata to "_NvOSD_RectParams" :: Unexpected result !!

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I am trying to attach custom metadata “unsigned int kitty” to the _NvOSD_RectParams. For this, I have modified the “nvll_osd_struct.h” like this:

typedef struct _NvOSD_RectParams {
  ............
  unsigned int height; /**< Holds height of the box in pixels. */
  
  unsigned int kitty;  // Custom metadata

...............
} NvOSD_RectParams;

And added this metadata to the “gstdsexample.cpp”

static void
attach_metadata_object (GstDsExample * dsexample, NvDsObjectMeta * obj_meta,
    DsExampleOutput * output)
{
  if (output->numObjects == 0)
    return;
  NvDsBatchMeta *batch_meta = obj_meta->base_meta.batch_meta;

  NvDsClassifierMeta *classifier_meta =
    nvds_acquire_classifier_meta_from_pool (batch_meta);

  classifier_meta->unique_component_id = dsexample->unique_id;

  NvDsLabelInfo *label_info =
    nvds_acquire_label_info_meta_from_pool (batch_meta);
  g_strlcpy (label_info->result_label, output->object[0].label, MAX_LABEL_SIZE);
  nvds_add_label_info_meta_to_classifier(classifier_meta, label_info);
  nvds_add_classifier_meta_to_object (obj_meta, classifier_meta);

  nvds_acquire_meta_lock (batch_meta);
  NvOSD_TextParams & text_params = obj_meta->text_params;
  NvOSD_RectParams & rect_params = obj_meta->rect_params;

  rect_params.kitty = (int)50;  // modified custom metadata value

 nvds_release_meta_lock (batch_meta);
}

Finally I wanted to read the metadata from deepstream_test3_app. For this, I have added the dsexample to the pipeline:

gboolean fullFrame = FALSE;

dsExample = gst_element_factory_make ("dsexample", "deepstream-example"); 

g_object_set (G_OBJECT (dsExample), "full-frame", fullFrame, NULL);

gst_bin_add_many (GST_BIN (pipeline), pgie, tiler, nvvidconv, dsExample, nvosd, transform, sink,
NULL);
/* we link the elements together
* nvstreammux -> nvinfer -> nvtiler -> nvvidconv -> nvosd -> video-renderer */
if (!gst_element_link_many (streammux, pgie, tiler, nvvidconv, dsExample, nvosd, transform, sink,
  NULL)) {
g_printerr ("Elements could not be linked. Exiting.\n");
return -1;
}

But when I read the custom metadata value from a probe:

static GstPadProbeReturn
tiler_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
    GstBuffer *buf = (GstBuffer *) info->data;
    guint num_rects = 0; 
    NvDsObjectMeta *obj_meta = NULL;
    guint vehicle_count = 0;
    guint person_count = 0;
    NvDsMetaList * l_frame = NULL;
    NvDsMetaList * l_obj = NULL;
    //NvDsDisplayMeta *display_meta = NULL;

    NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);

    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) {
        NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
        //int offset = 0;
        for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
                l_obj = l_obj->next) {
            obj_meta = (NvDsObjectMeta *) (l_obj->data);
            if (obj_meta->class_id == PGIE_CLASS_ID_VEHICLE) {
                vehicle_count++;
                num_rects++;
            }
            if (obj_meta->class_id == PGIE_CLASS_ID_PERSON) {
                person_count++;
                num_rects++;
            }

 	int kitti =  obj_meta->rect_params.kitty;
	printf(" Custom Object meta:: kitty = %d\n", kitti);
        
       }
          g_print ("Frame Number = %d Number of objects = %d "
            "Vehicle Count = %d Person Count = %d\n",
            frame_meta->frame_num, num_rects, vehicle_count, person_count);

}
    return GST_PAD_PROBE_OK;
}

It was expected to get the value 50 on output, But its getting a value 3 here.

Custom Object meta:: kitty = 3 // Expected 50
Frame Number = 36 Number of objects = 4 Vehicle Count = 4 Person Count = 0

I just wanted to know of the method is correct or anything I am missing?

2

Hi,
Please check the sample:

deepstream_sdk_v4.0_jetson\sources\apps\sample_apps\deepstream-gst-metadata-test
3

Actually what I am trying to achieve is to attach the cropped bounding box images of detected objects in cv::Mat format as rect_params metadata which is available in “gst-dsexample” plugin. Is there any way to pass the Mat to the buffer probe?

4

Hi,
Maybe create a custom metadata with GstBuffer,width,height, and copy the cv::Mat to the GstBuffer.
Modifying existing structures in header files is not good since these are referred in building the package. You need to make your own custom metadata.

5

Hi , I have same problem
@xhuv_NV were you able to attach the cropped bbox ?