Displaying source specific metadata with multiple sources

Please provide complete information as applicable to your setup.

• Hardware Platform: Jetson
• DeepStream Version 6.2
• JetPack Version 5.
• Issue Type( question/bugs)

How to reproduce the issue:

I have a pipeline that roughly looks like the image below.


multi source metadata display.zip|attachment, for higher res (55.4 KB)

and I have a pad probe function (in the doagram I have shown two places where I have attached it)

This is the source code functionof the padprobe function, (I have added some unclear bits in the comments and it would be grerat if you can answer these.

def dummy_display_meta_injector_to_a_frame(pad, info, u_data, app_context):
    """
    A dummy display meta injector to test that I can overlay camera name in each display feed tile 
    

    cameras = ... this is a dictionary with information such as camera_name, location, is_detected etc.
    muxpad_to_cam_index = ... this is a mapping created at `muxer0` so we can know which muxer 
                              pad is linked to which camera in case all defined cameras are not responding to 
                              communications at init

    """
    
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        logger.error("unable to get *** src pad buffer")
        return Gst.PadProbeReturn.OK
    
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    if batch_meta is None:
        logger.error("Unable to get batch meta from buffer")
        return Gst.PadProbeReturn.OK
    
    l_frame = batch_meta.frame_meta_list

    try:

        while l_frame is not None:
            try:
                frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            except StopIteration:
                logger.error("Error casting frame_meta")
                break
            try:
                l_frame = l_frame.next
            except StopIteration:
                break
            if frame_meta is None:
                logger.error("Frame meta is None")
                continue

            """
             source_id, where does this come from? (e.g. origin elemement of the source? what does this really mean?
             I looked at the API referece (headers etc. but they are not clear to the origin of this, please explain) 
            """
            source_id = frame_meta.source_id  
                                                                          

            if int(source_id) != 0:
                continue


            pad_index = frame_meta.pad_index # is this the immediate upstream muxpad? or the first stream enconters?
            timestamp = frame_meta.buf_pts
            is_infer_done = f"Yes" if frame_meta.bInferDone else "No"
            ntp_timestamp = frame_meta.ntp_timestamp
            ntp_time = datetime.datetime.utcfromtimestamp(ntp_timestamp / 1e9).strftime('%Y-%m-%d %H:%M:%S')
            time_stamp = datetime.timedelta(seconds=int(timestamp / 1e9))

            frame_display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)
            frame_display_meta.num_labels = 1 #  int, Number of labels/strings present in display meta.

            pynvosd_text_params = frame_display_meta.text_params[0]

            camera_index = muxpad_to_cam_index.get(str(pad_index), None)
            if camera_index is not None:
                camera_name = cameras[camera_index]['name']
                camera_position = cameras[camera_index]['position']
            else:
                camera_name = "Unknown"
                camera_position = "Unknown"

            pynvosd_text_params.display_text = (
                f"Timestamp: {ntp_time} \n"
                f"Session uptime: {time_stamp} \n"
                f"Frame number: {frame_meta.frame_num} \n"
                f"Source ID: {source_id}, Pad index: {pad_index}\n"
                f"Camera name: {camera_name}\n"
                f"Camera position: {camera_position}\n"
                f"Infer done: {is_infer_done} \n"
            )

            pynvosd_text_params.x_offset = 10
            pynvosd_text_params.y_offset = 12
            pynvosd_text_params.font_params.font_name = "Serif"
            pynvosd_text_params.font_params.font_size = 10
            pynvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)
            pynvosd_text_params.set_bg_clr = 1
            pynvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)

            pyds.nvds_add_display_meta_to_frame(frame_meta, frame_display_meta)

            # free resources

    except Exception as e:
        logger.error(f"Error in dummy_meta_injector: {e}")
        

    return Gst.PadProbeReturn.OK

The problem is when i keep re runing this the cose the metadata window keeps popping out in different places,
I have set sync = 0in my pipeline to stop getting it stuck

in the frame grabbing while loop I have added a condition to only insert metadata in source_id = 0

  while l_frame is not None:
            try:
                frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            except StopIteration:
                logger.error("Error casting frame_meta")
                break
            try:
                l_frame = l_frame.next
            except StopIteration:
                break
            if frame_meta is None:
                logger.error("Frame meta is None")
                continue

            source_id = frame_meta.source_id


            if int(source_id) != 0: # this is meant to filter aout other sources?
                continue

I am wondering if the source id field is something we have to set manually at initalisaiton. (I can’t see a changable filed in sources for example nvarguscamerasrc does not have a manually configurable field)

I also notice that the display window order in the tiler is always the same (this is good) but the the place the metadata window keeps ending up is not consistant, it seems to flicker back into windows while running (but usualy stays in the window it came in initially)

finally not sure if the osd should come before or after the tiler (i have place the OSD before)

This is the answer!! Tiler --> OSD Seem to have fixed the issue!! (but if you can still annswer about where those frame.source_id and frame.pad_index come from. That will make life easy (it;s always good to be 100% sure than 95% (: )

Thanks a lot!
Ganindu.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.