Add custom NvDsDisplayMeta for each frame individually

Please provide complete information as applicable to your setup.

• Hardware Platform (GPU)
• DeepStream Version 6.4
• NVIDIA GPU Driver Version (valid for GPU only) NVIDIA GeForce GTX 1650 / Driver Version: 525.147.05 / CUDA Version: 12.0
• Issue Type( questions, new requirements, bugs) question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hello,

I have multiple streams which I later display with “tiler”.

I want to draw different ares to each stream using NvDsDisplayMeta.

I realized, that I need to add it to each frame separetly, as otherwise it will draw everything on the last frame processed if I use the basic example from GitHub.

def src_pad_buffer_probe(pad, info, u_data, config, gates):
    
    gst_buffer = info.get_buffer()
 
    if not gst_buffer:
        logger.exception("Unable to get GstBuffer ")
        return

    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)

 
    l_frame = batch_meta.frame_meta_list

    buff_images = {}
    execution = False

    while l_frame is not None:

        try:

            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)

        except StopIteration:
            break

        display_data = pyds.NvDsDisplayMeta()

        if frame_meta.batch_id == 0: # if it the first stream

            display_data.num_circles = 1
            py_nvosd_circle_params = display_data.circle_params[0]
            py_nvosd_circle_params = draw_circle(params=py_nvosd_circle_params, xc=100, yc=100, r=30)


However, I do not know how to add the information (display_data) to the frame data… I know the information must be stored in frame_meta.display_meta_list, but how do I add it there?

Thank you very much in advance.

What do you mean above? Do you want different sources to display in different areas?

So what I want is, i.e.:

  • in stream 1: draw a circle on place a
  • in stream 2: draw a rectangle place b

stream 1 should not display the rectangle from stream 2 and …
stream 2 should not dispaly the circle from stream 1…

Following the examples from github, the shapes are drawn on every stream.

I wanted to create the new class pyds.NvDsDisplayMeta() but I could not add it…

What steps should I follow to achieve my goal?

You can use the source_id from NvDsFrameMeta to identify which source the frame comes from.
Then you can draw a circle on the frames come from source1 and draw a rectangle on the frames come from source2.

Many thanks for your quick reply!

Can you give me an example please or reference a similar code?

You can refer to our demo code deepstream_test_1.py. What we’re drawing here is texts, you can draw circles or rectangles instead.

But this what I am saying dont work as it will draw everything on ALL streams.

How can I specify a spicific stream where to draw on?

I have attached that. You can identify that with the source_id from NvDsFrameMeta.

Ok, back to previous question - I do not understand how the code should look like while adding the draw elements to the correct source_id - can you just provide sample python code to do this?

Sorry for the numb quesiton, but I cannot find any reference how you can directly do this…

You can just add a conditional statement at while l_frame is not None:.

frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
if frame_meta.source_id == 1:
  #draw circle;
else if frame_meta.source_id == 2:
  #draw rectangle;

Ok, in my code I had:

 if frame_meta.batch_id == 0:

Whats the difference, as it did not work?

There are no batch_id in the frame_meta structure. Please refer to our source code: nvdsmeta.h.

While doing:

    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    
    display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)

    l_frame = batch_meta.frame_meta_list
 
    while l_frame is not None:

        logger.info(l_frame.source_id) 

gives an error:

AttributeError: 'pyds.GList' object has no attribute 'source_id'

You can try to get frame_meta first, like below

So now coming back to my original question - how do I draw something specific on this particular frame?

As dispay_meta is coming from batch_meta it will only draw the circle either on the one stream or the other, but not consistently on the one provided.

This is the code and as you can see on the images, the circle gets drawn either on the one or the other stream. So how can I draw it only one specific?

def test_src_pad_buffer_probe(pad, info, u_data, config, gates):


    gst_buffer = info.get_buffer()
 
    if not gst_buffer:
        logger.exception("Unable to get GstBuffer ")
        return
    
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    
    display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)

    l_frame = batch_meta.frame_meta_list
 
    while l_frame is not None:

        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break
        
        logger.info(frame_meta.source_id)
        # if frame_meta.source_id == 1:


        if frame_meta.source_id == 0:

            # logger.info('CAST 1')
            display_meta.num_circles = 1
            py_nvosd_circle_params = display_meta.circle_params[0]
            py_nvosd_circle_params = draw_circle(params=py_nvosd_circle_params, xc=100, yc=100, r=30)

        user_event_meta = pyds.nvds_acquire_user_meta_from_pool(batch_meta)

        if user_event_meta:
            pass


        try:
            l_frame=l_frame.next
        except StopIteration:
            break

        # Update FPS
        global perf_data
        perf_data.update_fps(f'stream{frame_meta.pad_index}')

    pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
    
    return Gst.PadProbeReturn.OK    


It’s weird. According to the code you attached, it should be drawn only on the source 1. Could you attach the whole code so that I can run that on my side?

You can add the following function as probe prior NvOSD block in any of your tests and see if it is working…

It just need to be at least two streams…

Apart from that, my pipeline is very similar to the test ones in the github.

def test_src_pad_buffer_probe(pad, info, u_data):


    gst_buffer = info.get_buffer()
 
    if not gst_buffer:
        logger.exception("Unable to get GstBuffer ")
        return
    
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    
    display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)

    l_frame = batch_meta.frame_meta_list
 
    while l_frame is not None:

        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break
        
        if frame_meta.source_id == 0:

            display_meta.num_circles = 1
            py_nvosd_circle_params = display_meta.circle_params[0]
            py_nvosd_circle_params = draw_circle(params=py_nvosd_circle_params, xc=100, yc=100, r=30)


        try:
            l_frame=l_frame.next
        except StopIteration:
            break


    pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
    
    return Gst.PadProbeReturn.OK

I have tried your code in our deepstream-test3. It works well. Could you try to upgrade DeepStream to 7.0 and try that?

Hmmm… does not work on my side…

To which probe did you attach the custom function? Did you also activate the pgie probe?

I cannot play the video unfortunately.

In the meantime, I will try to downsize the code, which you can reproduce on your side.

I use our demo code deepstream_test_3.py.

Then add your probe function to the sink pad of nvdsosd plugin.

I use our latest version DS 7.0. Maybe you can try that.