Sync inputs=True in DeepStream, buffer inconsistency occurs

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU)GPU
**• DeepStream Version6.4
• JetPack Version (valid for Jetson only)
**• TensorRT Version8.6.1
**• NVIDIA GPU Driver Version (valid for GPU only)535.183.01
**•
When I perform streammux on multiple sources and change sync_inputs to true, after nvtrack, I obtain batch_meta and change the detection box to green. Some sources may be empty, but when tiler ->sink is displayed, a red detection box appears. Why is there metadata in this frame? Why wasn’t it obtained by me?

3

what is the complete pipeline? how many sources did you test? what do you mean about “Some sources may be empty”? what do you mean about “Why is there metadata in this frame? Why wasn’t it obtained by me?”?

4

The complete pipeline diagram is as follows:
pipeline.zip (468.7 KB)

Here is my code. Our intention is to map the detection results of 18 sources to the first source, but when I turn on the sync inputs option, the detection box keeps flashing. Through printing, I found that sometimes batch_sta may be incomplete. According to nvstreammux’s instructions, when the synchronization frame arrival time timeout is enabled, incomplete batches will indeed be pushed. However, when I finally tiled it, I found that the batch_sta I obtained will be mapped to the first source, with a green box. In addition, there is also data that has not been obtained by me, with a red box.

static GstPadProbeReturn tracker_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info, gpointer u_data){
    GstBuffer *buf = (GstBuffer *) info->data;  //getting buffer

    NvDsObjectMeta *obj_meta_src0 = NULL;
    NvDsMetaList * l_frame = NULL;
    NvDsMetaList * l_obj_src0 = NULL;
    NvDsFrameMeta *panorama_frame = NULL;

    NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf); //extracting batch meta
    
    std::unordered_map<guint, NvDsFrameMeta*> frame_map;
    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;l_frame = l_frame->next) {
        NvDsFrameMeta *frame_meta = (NvDsFrameMeta *)l_frame->data;
        frame_map[frame_meta->source_id] = frame_meta;
    }
    // g_print("src_0 %d,src_1 %d,src_2 %d,src_3 %d,src_4 %d,src_5 %d,src_6 %d,src_7 %d,src_8 %d,src_9 %d,src_10 %d,src_11 %d,src_12 %d,src_13 %d,src_14 %d,src_15 %d,src_16 %d,src_17 %d,src_18 %d\n",frame_map.count(1),frame_map.count(2),frame_map.count(3),frame_map.count(4),frame_map.count(5),frame_map.count(6),frame_map.count(7),frame_map.count(8),frame_map.count(9),frame_map.count(10),frame_map.count(11),frame_map.count(12),frame_map.count(13),frame_map.count(14),frame_map.count(15),frame_map.count(16),frame_map.count(17),frame_map.count(18));
    if(frame_map.count(0))
    {
      panorama_frame = frame_map[0];
      nvds_clear_obj_meta_list(panorama_frame, panorama_frame->obj_meta_list); 
      nvds_clear_display_meta_list(panorama_frame, panorama_frame->display_meta_list);
      for (int src_id = 1; src_id <= 18; ++src_id) {
        if (frame_map.count(src_id)) {
            transform(batch_meta, frame_map[src_id], panorama_frame);
        }
      }
    }
    return GST_PAD_PROBE_OK;
}

void transform(NvDsBatchMeta *batch_meta,NvDsFrameMeta * frame_meta_src0,NvDsFrameMeta * frame_meta_src1){

  NvDsObjectMeta *obj_meta_src0 = NULL;

  for (NvDsMetaList *l_obj_src0 = frame_meta_src0->obj_meta_list; l_obj_src0 != NULL;l_obj_src0 = l_obj_src0->next) {

      //getting object metadata
      obj_meta_src0 = (NvDsObjectMeta *) (l_obj_src0->data);

      //acquiring object metadata
      NvDsObjectMeta *new_obj_meta = nvds_acquire_obj_meta_from_pool(batch_meta);

      //copying to new object
      nvds_copy_obj_meta(obj_meta_src0,new_obj_meta);

      obj_meta_src0->rect_params.border_color.red=0;
      obj_meta_src0->rect_params.border_color.green=1;
      obj_meta_src0->rect_params.border_color.blue=0;
      obj_meta_src0->rect_params.border_width=5;

      //applying transformation
      apply_homography_obj_meta(frame_meta_src0->source_id,new_obj_meta);

      //adding modified object metadata to the frame of source1
      nvds_add_obj_meta_to_frame(frame_meta_src1,new_obj_meta,NULL);
  }  
}

This is a screenshot of the problem that occurred：
Screenshot from 2025-03-28 11-26-18.zip (5.8 MB)

5
  1. are you testing rtsp sources? what are the fps respectively? If there is no source0 in batch, “if(frame_map.count(0))” will not be entered. you can add log in tracker_src_pad_buffer_probe and “if(frame_map.count(0))” to check if each batch include source0.
  2. Before leaving tracker_src_pad_buffer_probe, you can check if the object color of 18 sources is red.
6

Thank you, I know where the problem lies

7

Is this still an DeepStream issue to support? Thanks! what is apply_homography_obj_meta used to do? What is the fps, reoslution of the first source? why not use the ROI feature of nvdspreprocess? please refer to ready-made sample deepstream-preprocess-test, in which you can set many ROIs.

8

Thank you, it’s no longer needed

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.