Deepstream-test2.py has got me wrong classificaton, the predicted bounding box show on whole video frame

Hi, Hope you all doing well .
Please have a look at this problem.

i have a problem with the classifier not getting the class id for which it will operate.

as I have set the above parameter in secondary-gie

gie-unique-id=2   # this is classifier gie that why it id is 2 and primary gie id is 1
operate-on-gie-id=1   # it mean operate on primary gie which id is 1
operate-on-class-ids=4  # which class of primary gie you want to apply classification so say id 0 or may be other.

I have set the id =4 which in my case is a person but the classifier inference show on the Whole video frame like this.


The code that I used is deepstream-test2.py I just added the following code instead of src_pad_probe.

color_list = {"balcony_with_railing":  (0, 0, 1),
              "balcony_without_railing": (1, 0.8, 0),
              "helmet": (0.6, 0.2, 1),
              "incomplete_railing":  (1, 0, 0),
              "person": (0.2, 0.8, 0.2),
              "rail": (0.5, 0.8, 0.5)}



past_tracking_meta=[0]
PGIE_CLASS_ID_balcony_with_railing = 0
PGIE_CLASS_ID_balcony_without_railing = 1
PGIE_CLASS_ID_helmet = 2
PGIE_CLASS_ID_incomplete_railing = 3
PGIE_CLASS_ID_person = 4
PGIE_CLASS_ID_rail  = 5
pgie_classes_str= ["balcony_with_railing", "balcony_without_railing", "helmet","incomplete_railing", "person", "rail"]

def pgie_src_pad_buffer_probe(pad,info,u_data):
   
    #Intiallizing object counter with 0.
#    obj_counter = {
#        PGIE_CLASS_ID_VEHICLE:0,
#        PGIE_CLASS_ID_PERSON:0,
#        PGIE_CLASS_ID_BICYCLE:0,
#        PGIE_CLASS_ID_ROADSIGN:0
#    }
    field_names = ["Timestamp",'Frame Number', 'Number of Objects', 'Helmet_count',
               'Person_count', 'Balcony_with_railing', "Balcony_without_railing","Incomplete_railing","Rail"]
    frame_number=0
    obj_counter ={
            PGIE_CLASS_ID_balcony_with_railing:0,
            PGIE_CLASS_ID_balcony_without_railing:0,
            PGIE_CLASS_ID_helmet:0,
            PGIE_CLASS_ID_incomplete_railing:0,
            PGIE_CLASS_ID_person: 0,
            PGIE_CLASS_ID_rail: 0,

            }
    num_rects=0
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        print("Unable to get GstBuffer ")
        return

    # Retrieve batch metadata from the gst_buffer
    # Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
    # C address of gst_buffer as input, which is obtained with hash(gst_buffer)
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        frame_number=frame_meta.frame_num
        num_rects = frame_meta.num_obj_meta
        l_obj=frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                # Casting l_obj.data to pyds.NvDsObjectMeta
                obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
            except StopIteration:
                break
                #---------------------this code is for adding custom label to each class -----------
            txt_params = obj_meta.text_params
            # Set display_text. Any existing display_text string will be
            # freed by the bindings module.
            txt_params.display_text = pgie_classes_str[obj_meta.class_id]
            obj_counter[obj_meta.class_id] += 1
    #-----------------end of code ---------------------------------

   #---------------the below code is for setting different color to each class------------------        
            color = color_list[obj_meta.obj_label.split("\\")[0]]
            rect_params = obj_meta.rect_params
            rect_params.border_color.set(*color, 1)
            
            try: 
                l_obj=l_obj.next
            except StopIteration:
                break

        # Acquiring a display meta object. The memory ownership remains in
        # the C code so downstream plugins can still access it. Otherwise
        # the garbage collector will claim it when this probe function exits.
        display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)
        display_meta.num_labels = 1
        py_nvosd_text_params = display_meta.text_params[0]
        # Setting display text to be shown on screen
        # Note that the pyds module allocates a buffer for the string, and the
        # memory will not be claimed by the garbage collector.
        # Reading the display_text field here will return the C address of the
        # allocated string. Use pyds.get_string() to get the string content.
#        py_nvosd_text_params.display_text = "Frame Number={} Number of Objects={} Helmet_count={} Person_count={} Balcony_with_railing={} Balcony_without_railing={} Incomplete_railing={} Rail={}".format(frame_number, num_rects, obj_counter[PGIE_CLASS_ID_helmet], obj_counter[PGIE_CLASS_ID_person],obj_counter[PGIE_CLASS_ID_balcony_with_railing],
#                                                          obj_counter[PGIE_CLASS_ID_balcony_without_railing],obj_counter[PGIE_CLASS_ID_incomplete_railing],obj_counter[PGIE_CLASS_ID_rail])
        ct = datetime.datetime.now()
        
        ditcc = {"Timestamp":ct,'Frame Number':frame_number, 'Number of Objects':num_rects, 'Helmet_count':obj_counter[PGIE_CLASS_ID_helmet],
               'Person_count':obj_counter[PGIE_CLASS_ID_person], 'Balcony_with_railing':obj_counter[PGIE_CLASS_ID_balcony_with_railing],
               "Balcony_without_railing":obj_counter[PGIE_CLASS_ID_balcony_without_railing],
               "Incomplete_railing":obj_counter[PGIE_CLASS_ID_incomplete_railing],"Rail":obj_counter[PGIE_CLASS_ID_rail]}
        
        with open('event.csv', 'a') as f_object:

            dictwriter_object = DictWriter(f_object, fieldnames=field_names)
            dictwriter_object.writerow(ditcc)
            f_object.close()
            
        # Now set the offsets where the string should appear
        py_nvosd_text_params.x_offset = 12
        py_nvosd_text_params.y_offset = 12

        # Font , font-color and font-size
        py_nvosd_text_params.font_params.font_name = "Serif"
        py_nvosd_text_params.font_params.font_size = 10
        # set(red, green, blue, alpha); set to White
        py_nvosd_text_params.font_params.font_color.set(1.0, 0.0, 1.0, 1.0)

        # Text background color
        py_nvosd_text_params.set_bg_clr = 1
        # set(red, green, blue, alpha); set to Black
        py_nvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)
        # Using pyds.get_string() to get display_text as string
        print(pyds.get_string(py_nvosd_text_params.display_text))
        pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
        try:
            l_frame=l_frame.next
        except StopIteration:
            break

I also changed the value of operate-on-class-ids=4 but the result remained same.

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi, @fanzh Thank you for your response.

My system spec.

Nvidia-RTX 3060
Deepstream-6.1
process: AMD Reyzon

Here is the code with all the required files you can simply run this by going to deepstream-tes2 folder and run new.py I have put the input file named ( output.H264).

 sudo python3 new.py output.h264 

i also created fp32.engine, fp16.engine but fp32.engine for classifier is no worked and got me error Failed to Parse BBox and i test for fp16 it work during runtime but not reference showed.
the only int8.engine file work perfectly and show inference but wrongly like mentioned in above pictures .

Please run the application and you will see error like mentioned in above pictures

will the pgie recognize the frame as a person?

i have add the following parameter in classification config file .

gie-unique-id=2   # this is classifier gie that why it id is 2 and primary gie id is 1
operate-on-gie-id=1   # it mean operate on primary gie which id is 1
operate-on-class-ids=4  # this is the Id of person 

just run and it will give you the result on the whole frame rather than a person.

No the primary pgie config make only detection on different object inside each frame. but the classifier makes wrong prediction , it should predict for person object and show label along side person label.

if I remove sgie, there is no that red bbox, need to check why there is a red bbox.

please add process-mode=1 in nvinfer_classifier.txt, you might find the explanation in this link nvinfer

does not made any difference.

i used process-mode =1 in nvinfer_classifier.txt run application twice got same result as before.
then I tried to use gie-mode=1 in nvinfer_classifier.txt but still does not made any difference.

i don’t know how to solve this

sorry for typo, should add “process-mode=2”, which means sgie.

Thank you so much it worked.

but one thing I experienced that when the same model I use in reference app it don’t detect uncertain objects but this python code inference show more uncertain object

How I can set inference threshold to 60 or 70% if predicted object has 70% confidence then we will show bounding otherwise no.

Can you sort this out?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.