Filtering display artefacts (bounding boxes, annotations) in a Gstreamer/Deepstream 'osdsinkprobe'

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
#1

Note: I’m using deepstream 5 due to a dependency in custom hardware

Hi,
I have a deepstream pipeline working on a live feed (it is working fine) I’m using an ‘osdsinkprobe’ pad to extract metadata.
At the display time I want to limit/change what goes into to the display. (e.g. Let’s say I want to filter out a subset of classes from a multi class classifier)

looking at the (metadata docs)[MetaData in the DeepStream SDK — DeepStream 6.0 Release documentation] I can see several functions that may possibly achieve this. for example (pyds_remove_object_meta_from_frame)[get_segmentation_masks — Deepstream Deepstream Version: 6.0 GA documentation]

These functions do not seem to remove the bounding boxes, I tried, pyds_remove_object_meta_from_frame() ← nothing happens, free_buffer() ← does not work, nvds_clear_obj_meta_list() ← clears all bboxes

Also I noticed that the
pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta) seem to have no effect (even my bbox color change seem to have somehow propagated to the C side doing the display overview, or again this might be a tell for my lack of understanding on how this works.

please help…

further info: This is a osd sink pad buffer probe, is it possible I’m looking at the wrong place ?

Cheers,
Ganindu.

EDIT:
I get the feeling that it deletes something and then looses the reference to the object list. I can find the list index of the objects then if I can point nvds_remove_obj_meta_from_frame it might work but I don’t know if there is a function to access that way.

here is my test code

    print("N objects = {}".format(objects))
    l_obj = frame_meta.obj_meta_list

    obj_id = 0
    deleted = 1
    increment  = True

    while l_obj is not None:
        try:
            obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)

            if obj_meta.class_id == 0 :
                pyds.nvds_remove_obj_meta_from_frame(frame_meta, obj_meta)
                print("class = {} object deleted , total deleted = {} (0 - car, 1 - bike, 2 - person)".format(obj_meta.class_id, deleted))
                deleted += 1
                # increment = False
            
            else:
                # skip_increment = False   <- This causes a segfault, (Try to increment only if not deleted)
                pass
                


            print(" class ID = {}, object index = {}, confidance = {}".format(obj_meta.class_id, obj_id, obj_meta.confidence))
            obj_id += 1


        except StopIteration:
            break

        try:
            if increment:
                l_obj = l_obj.next
        except StopIteration:
            break
#3

Hi,

Suppose you can achieve this by the filter-out-class-ids variable in the nvinfer configure file.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications

Please give it a try.
Thanks.

#4

Hi Thanks for the reply!
Making that change at the config file seems to be a solution. but what is the intended behaviour of

It seems to remove the first object meta from the frame and then somehow not let the list iterate over, If I understand correctly removing the object from the frame seems to cause the python object to loose the held reference, does that mean the NvDsObjectMeta.cast has to be used again and the process repeated to remove a subsequent object (if someone happened to chose this method)

Cheers,
Ganindu.

#5

Hi,

Based on the below document, the function is to remove the given object meta from the frame metadata.

nvds_remove_obj_meta_from_frame

Do you think this might be an issue?
If yes, could you share more information about it?

Thanks

#6

Hi Asta,
Thanks a lot for getting back to me. Unless I’m making some obvious mistake repeatedly (in the code bit I’ve shared above ) it seems strange. What I noticed is it is deleting the bounding box of the first object that matches the criteria.

(here I’ve done a simple comparison to demonstrate the behaviour but I’m doing a set/list comparison but same result)

and then It seems to loose the object list handle (maybe that’s caused by the object pointer being affected), In summary basically if the object (it has to be more than one that matches the criteria in the list) I want to remove is in the middle of the list it seems to fail to do it.

from my intuition I believe if the list is if size one or if the target for deletion is at the end of the list (only one item to be deleted) it would seem as it’s doing what’s needed with the underlying cause (for my pains) still being in effect

Cheers,
Ganindu.