Nvdsosd printing OSD info only on the first frame of the batch, why?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU) Jetson Nano
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only) 4.4-b144
• TensorRT Version 7.1.3
• Issue Type( questions, new requirements, bugs) question

Good afternoon,

I have setup a pipeline starting from deepstream_test3_app.
The pipeline is structured as follows:

  if (!gst_element_link_many (streammux, pgie, nvvidconv, nvosd, tee, NULL)) {
    g_printerr ("Elements could not be linked. Exiting.\n");
    return -1;
  }

  if (!gst_element_link_many (queue1, saveframe, msgconv, msgbroker, NULL)) {
    g_printerr ("Elements could not be linked. Exiting.\n");
    return -1;
  }

 // if (!gst_element_link_many (queue2, transform, sink, NULL)) {
  if (!gst_element_link_many (queue2, sink, NULL)) {
      g_printerr ("Elements could not be linked. Exiting.\n");
      return -1;
  }

I have 3 video RTSP video stream. I run the PGIE to detect objects.
If any object is detected from any of the source, the frame is saved and a message is sent through AMQP.
“sink” is just a “fakesink” for I do not need to visualize any stream.

Originally deepstream_test3_app was including also a tiler. I do not need it for I do not intend to visualize any stream.
However, when I removed it the “nvosd” started to behave oddly.
With the tiler, nvosd is printing OSD info on the frames correctly.
Without the tiler, nvosd is printing OSD info only on the first frame of the batch.
(in my case I have setup a batch of size 3 for I have 3 sources)

Do you have any idea about why nvosd is not printing OSD info on every frame of the batch?

Thank you very much

3

It’s expected, tiler is used to transform the meta data however you had removed it.

4

Dear @bcao ,

Good morning and thank you for your reply.

I believe that the transformation of meta data is still working properly.
The OSD info that I see on the saved frames are not always displayed in the first source, but also in frames for source 2 and source 3. However it is mainly from source 1, that is why I believe that it is displayed only on the first frame of the batch.

As you can see below, the probe that creates mes_meta before the tiler is still working:

static GstPadProbeReturn
tiler_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
    GstBuffer *buf = (GstBuffer *) info->data;
    guint num_rects = 0; 
    NvDsObjectMeta *obj_meta = NULL;
    NvDsFrameMeta *frame_meta = NULL;
    guint vehicle_count = 0;
    guint person_count = 0;
    NvDsMetaList * l_frame = NULL;
    NvDsMetaList * l_obj = NULL;
    gboolean is_first_object = TRUE;
    //NvDsEventMsgMeta *msg_meta = NULL;
    //NvDsDisplayMeta *display_meta = NULL;

    NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);

    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) 
    {
        frame_meta = (NvDsFrameMeta *) (l_frame->data);
        //int offset = 0;
        is_first_object = TRUE;
        for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
                l_obj = l_obj->next) 
        {
            obj_meta = (NvDsObjectMeta *) (l_obj->data);
            if (obj_meta->class_id == PGIE_CLASS_ID_VEHICLE) {
                vehicle_count++;
                num_rects++;
            }
            if (obj_meta->class_id == PGIE_CLASS_ID_PERSON) {
                person_count++;
                num_rects++;
            }
          
            #if 1
//              if (is_first_object && !(frame_number % 15)) {
	    // if (!(frame_number % 15)) {


                NvDsEventMsgMeta *msg_meta = (NvDsEventMsgMeta *) g_malloc0 (sizeof (NvDsEventMsgMeta));
                msg_meta->bbox.top = obj_meta->rect_params.top;
                msg_meta->bbox.left = obj_meta->rect_params.left;
                msg_meta->bbox.width = obj_meta->rect_params.width;
                msg_meta->bbox.height = obj_meta->rect_params.height;
                msg_meta->frameId = frame_number;
                msg_meta->trackingId = obj_meta->object_id;
                msg_meta->confidence = obj_meta->confidence;
		msg_meta->sensorId = frame_meta->source_id;


                generate_event_msg_meta (msg_meta, obj_meta->class_id, obj_meta);

                NvDsUserMeta *user_event_meta = nvds_acquire_user_meta_from_pool (batch_meta);
                if (user_event_meta) {
                  user_event_meta->user_meta_data = (void *) msg_meta;
                  user_event_meta->base_meta.meta_type = NVDS_EVENT_MSG_META;
                  user_event_meta->base_meta.copy_func = (NvDsMetaCopyFunc) meta_copy_func;
                  user_event_meta->base_meta.release_func = (NvDsMetaReleaseFunc) meta_free_func;
                  nvds_add_user_meta_to_frame(frame_meta, user_event_meta);
                } else {
                  g_print ("Error in attaching event meta to buffer\n");
                }
                //g_free (msg_meta);

                is_first_object = FALSE;
              //}
            #endif 
            
        }
/*
          g_print ("Frame Number = %d Number of objects = %d "
            "Vehicle Count = %d Person Count = %d\n",
            frame_meta->frame_num, num_rects, vehicle_count, person_count);
*/

#if 0
        display_meta = nvds_acquire_display_meta_from_pool(batch_meta);
        NvOSD_TextParams *txt_params  = &display_meta->text_params;
        txt_params->display_text = g_malloc0 (MAX_DISPLAY_LEN);
        offset = snprintf(txt_params->display_text, MAX_DISPLAY_LEN, "Person = %d ", person_count);
        offset = snprintf(txt_params->display_text + offset , MAX_DISPLAY_LEN, "Vehicle = %d ", vehicle_count);

        /* Now set the offsets where the string should appear */
        txt_params->x_offset = 10;
        txt_params->y_offset = 12;

        /* Font , font-color and font-size */
        txt_params->font_params.font_name = "Serif";
        txt_params->font_params.font_size = 10;
        txt_params->font_params.font_color.red = 1.0;
        txt_params->font_params.font_color.green = 1.0;
        txt_params->font_params.font_color.blue = 1.0;
        txt_params->font_params.font_color.alpha = 1.0;

        /* Text background color */
        txt_params->set_bg_clr = 1;
        txt_params->text_bg_clr.red = 0.0;
        txt_params->text_bg_clr.green = 0.0;
        txt_params->text_bg_clr.blue = 0.0;
        txt_params->text_bg_clr.alpha = 1.0;

        nvds_add_display_meta_to_frame(frame_meta, display_meta);
#endif    
      
      }
  
    frame_number++;
    return GST_PAD_PROBE_OK;
}

As a confirmation of that, I can tell you that i can correctly recognize the source of each frame thanks to the attached metadata later when I compose the messages to be sent in AMQP.
It is only the OSD information that appears only for few frames.

Would you be so kind to suggest me a solution?

Thanks again!

5

Hey, you are referring the display_meta, right?
I’m confused since you said you don’t need to visualize them, I guess what you need is the meta data attatched by the nvinfer such as the object class or its cordinates, right?

6

Dear @bcao ,

Thanks again for your reply.
Please let me clarify.
I do not need to visualiza all the frame. I am using a fakesink.
However, when a frame is detected to contain an object , it is saved on the disk.
When those frames are saved only a few contains the bounding box and the label.
And the frames that do contains the bounding box and the label are the ones that came from source #1 most of the time. However, in the other few cases also in frames from the other sources I see the bounding box and the label.
This behaviour makes me thinking that nvosd is drawing the bounding box and the label only on the first frame of the batch.

I mentioned the metadata in my earlier post because you might say that metadata are not generated correctly.
I verified that they are generated correctly for I exploit those data afterwards in sending the message via AMQP.

Thus, it is the nvosd that I am not able to understand why it behaves like I described.

I hope that you could help me.

Thanks a lot!

7

Did you find a solution for this? We’re noticing the same thing trying to draw bounding boxes on batched inputs without the tiler. nvosd seems to pick one stream and draws all the bounding boxes for all the feeds on that one stream. How do we need to transform the meta data so that nvosd will draw on the correct feeds?

8

@chadtgreen can you create a new topic and share your setup with us, it’s better to use the latest DS6.0

9

@Fiona.Chen answered this one for me here: How to transform NvDsBatchMeta for Gst-nvdsosd when no tiler present - #3 by Fiona.Chen

10

Great, so we will close this topic