Add custom NvDsDisplayMeta for each frame individually

oleg.s · July 5, 2024, 12:09pm

Please provide complete information as applicable to your setup.

• Hardware Platform (GPU)
• DeepStream Version 6.4
• NVIDIA GPU Driver Version (valid for GPU only) NVIDIA GeForce GTX 1650 / Driver Version: 525.147.05 / CUDA Version: 12.0
• Issue Type( questions, new requirements, bugs) question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hello,

I have multiple streams which I later display with “tiler”.

I want to draw different ares to each stream using NvDsDisplayMeta.

I realized, that I need to add it to each frame separetly, as otherwise it will draw everything on the last frame processed if I use the basic example from GitHub.

def src_pad_buffer_probe(pad, info, u_data, config, gates):
    
    gst_buffer = info.get_buffer()
 
    if not gst_buffer:
        logger.exception("Unable to get GstBuffer ")
        return

    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)

 
    l_frame = batch_meta.frame_meta_list

    buff_images = {}
    execution = False

    while l_frame is not None:

        try:

            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)

        except StopIteration:
            break

        display_data = pyds.NvDsDisplayMeta()

        if frame_meta.batch_id == 0: # if it the first stream

            display_data.num_circles = 1
            py_nvosd_circle_params = display_data.circle_params[0]
            py_nvosd_circle_params = draw_circle(params=py_nvosd_circle_params, xc=100, yc=100, r=30)

However, I do not know how to add the information (display_data) to the frame data… I know the information must be stored in frame_meta.display_meta_list, but how do I add it there?

Thank you very much in advance.

yuweiw · July 8, 2024, 2:00am

What do you mean above? Do you want different sources to display in different areas?

oleg.s · July 8, 2024, 5:54am

So what I want is, i.e.:

in stream 1: draw a circle on place a
in stream 2: draw a rectangle place b

stream 1 should not display the rectangle from stream 2 and …
stream 2 should not dispaly the circle from stream 1…

Following the examples from github, the shapes are drawn on every stream.

I wanted to create the new class pyds.NvDsDisplayMeta() but I could not add it…

What steps should I follow to achieve my goal?

yuweiw · July 8, 2024, 6:19am

You can use the source_id from NvDsFrameMeta to identify which source the frame comes from.
Then you can draw a circle on the frames come from source1 and draw a rectangle on the frames come from source2.

oleg.s · July 8, 2024, 6:22am

Many thanks for your quick reply!

Can you give me an example please or reference a similar code?

yuweiw · July 8, 2024, 7:46am

You can refer to our demo code deepstream_test_1.py. What we’re drawing here is texts, you can draw circles or rectangles instead.

oleg.s · July 8, 2024, 8:12am

But this what I am saying dont work as it will draw everything on ALL streams.

How can I specify a spicific stream where to draw on?

yuweiw · July 8, 2024, 8:36am

I have attached that. You can identify that with the source_id from NvDsFrameMeta.

oleg.s · July 8, 2024, 9:06am

Ok, back to previous question - I do not understand how the code should look like while adding the draw elements to the correct source_id - can you just provide sample python code to do this?

Sorry for the numb quesiton, but I cannot find any reference how you can directly do this…

yuweiw · July 9, 2024, 3:17am

You can just add a conditional statement at while l_frame is not None:.

frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
if frame_meta.source_id == 1:
  #draw circle;
else if frame_meta.source_id == 2:
  #draw rectangle;

oleg.s · July 9, 2024, 6:59am

Ok, in my code I had:

 if frame_meta.batch_id == 0:

Whats the difference, as it did not work?

yuweiw · July 9, 2024, 7:22am

There are no batch_id in the frame_meta structure. Please refer to our source code: nvdsmeta.h.

oleg.s · July 9, 2024, 7:50am

While doing:

    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    
    display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)

    l_frame = batch_meta.frame_meta_list
 
    while l_frame is not None:

        logger.info(l_frame.source_id)

gives an error:

AttributeError: 'pyds.GList' object has no attribute 'source_id'

yuweiw · July 9, 2024, 9:57am

You can try to get frame_meta first, like below

oleg.s · July 9, 2024, 10:35am

So now coming back to my original question - how do I draw something specific on this particular frame?

As dispay_meta is coming from batch_meta it will only draw the circle either on the one stream or the other, but not consistently on the one provided.

This is the code and as you can see on the images, the circle gets drawn either on the one or the other stream. So how can I draw it only one specific?

def test_src_pad_buffer_probe(pad, info, u_data, config, gates):


    gst_buffer = info.get_buffer()
 
    if not gst_buffer:
        logger.exception("Unable to get GstBuffer ")
        return
    
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    
    display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)

    l_frame = batch_meta.frame_meta_list
 
    while l_frame is not None:

        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break
        
        logger.info(frame_meta.source_id)
        # if frame_meta.source_id == 1:


        if frame_meta.source_id == 0:

            # logger.info('CAST 1')
            display_meta.num_circles = 1
            py_nvosd_circle_params = display_meta.circle_params[0]
            py_nvosd_circle_params = draw_circle(params=py_nvosd_circle_params, xc=100, yc=100, r=30)

        user_event_meta = pyds.nvds_acquire_user_meta_from_pool(batch_meta)

        if user_event_meta:
            pass


        try:
            l_frame=l_frame.next
        except StopIteration:
            break

        # Update FPS
        global perf_data
        perf_data.update_fps(f'stream{frame_meta.pad_index}')

    pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
    
    return Gst.PadProbeReturn.OK

yuweiw · July 10, 2024, 5:33am

It’s weird. According to the code you attached, it should be drawn only on the source 1. Could you attach the whole code so that I can run that on my side?

oleg.s · July 11, 2024, 7:47am

You can add the following function as probe prior NvOSD block in any of your tests and see if it is working…

It just need to be at least two streams…

Apart from that, my pipeline is very similar to the test ones in the github.

def test_src_pad_buffer_probe(pad, info, u_data):


    gst_buffer = info.get_buffer()
 
    if not gst_buffer:
        logger.exception("Unable to get GstBuffer ")
        return
    
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    
    display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)

    l_frame = batch_meta.frame_meta_list
 
    while l_frame is not None:

        try:
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break
        
        if frame_meta.source_id == 0:

            display_meta.num_circles = 1
            py_nvosd_circle_params = display_meta.circle_params[0]
            py_nvosd_circle_params = draw_circle(params=py_nvosd_circle_params, xc=100, yc=100, r=30)


        try:
            l_frame=l_frame.next
        except StopIteration:
            break


    pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
    
    return Gst.PadProbeReturn.OK

yuweiw · July 11, 2024, 10:09am

I have tried your code in our deepstream-test3. It works well. Could you try to upgrade DeepStream to 7.0 and try that?

oleg.s · July 12, 2024, 9:07am

Hmmm… does not work on my side…

To which probe did you attach the custom function? Did you also activate the pgie probe?

I cannot play the video unfortunately.

In the meantime, I will try to downsize the code, which you can reproduce on your side.

yuweiw · July 12, 2024, 9:30am

I use our demo code deepstream_test_3.py.

Then add your probe function to the sink pad of nvdsosd plugin.

I use our latest version DS 7.0. Maybe you can try that.

Topic		Replies	Views
Configure different drawing in different video source DeepStream SDK	4	1420	October 12, 2021
Adding unique display meta to each stream in a tiled OSD (python) DeepStream SDK python , deepstream	2	290	January 19, 2023
Drawing Line on top of video using NvDsLineMeta DeepStream SDK	11	1942	October 12, 2021
Adding line_params to display_meta does not draw DeepStream SDK	5	924	October 12, 2021
Unable to retrieve source_id in frame_meta, hence unable to distinguish which stream is generating what output DeepStream SDK	12	1830	October 12, 2021
After acquiring display meta from the pool and filling it's text params, not able to see it on scree... DeepStream SDK	4	1002	October 12, 2021
Draw different graphics with nvdsosd on two parallel streams DeepStream SDK	5	665	October 12, 2021
Fail to get frame in Tensor Metadata DeepStream SDK	24	2559	October 12, 2021
Nvdsosd printing OSD info only on the first frame of the batch, why? DeepStream SDK	9	1180	October 12, 2021
DeepStream 4.0 how to get meta? DeepStream SDK	2	965	October 12, 2021

Add custom NvDsDisplayMeta for each frame individually

Related topics