After the pipeline is started, there is a problem with filtering the detected objects as needed

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU):jetson
• DeepStream Version:6.3
• JetPack Version (valid for Jetson only):5.1.3
pipelinetest.pdf (36.9 KB)

The uploaded PDF file is my pipeline. I added a probe function after the src pad of the stacker plugin. Its function is to request external devices to obtain the confidence level required for detecting objects after the pipeline is started, and then request that objects with lower confidence levels not be drawn and displayed on the exported video stream. The following is the code for the probe function. After testing and printing, the above code cannot achieve the function I want. Why is this?

    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL; l_frame = l_frame->next) {

        NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);
        gchar *message = NULL;

        for (l_obj = frame_meta->obj_meta_list; l_obj != NULL; ) {
            obj_meta = (NvDsObjectMeta *) (l_obj->data);
            l_obj = l_obj->next;
            if (obj_meta == NULL || obj_meta->confidence < 0){
                continue;
            }
         if(obj_meta->confidence < (*(ZHYPipeline::zhyPostprocess->confidenceMap))[obj_meta->obj_label])
            {
             nvds_remove_obj_meta_from_frame(frame_meta,obj_meta);
             continue;
            }
         }
       }
3

why not use pre-cluster-threshold, topk and nms-iou-threshold in nvinfer configuration file to filter the objects?

4

Due to practical needs, different scenarios require configuring different categories, and which categories need to be loaded into the pipeline before knowing. I understand the method you mentioned, so the question says’ the pipeline has already been loaded '. Therefore, please reply directly to me with this question

5
  1. here is a sample to remove object meta. please add the probe before the sink of nvosd.
  2. if still can’t work, please check if the objec tmeta with “confidence < 0” is not removed.
7

so professional, so professional, The method you mentioned is effective, but there is a new issue. I have found that the confidence level of the obj_sta generated by the nvtracker plugin as NUll or tracking module is less than 0, right? Because I found that when my probe function is added to the src pad of the tracking plugin and the code is as follows, I set the inference interval to 10 frames. Even with tracking, the frames drawn in the streaming video still appear one by one. After investigation, I discovered this problem.So do I need to add my confidence filtering probe function after the src probe in the Nvidian plugin? Because if it is added after the tracker src pad, it will erroneously delete the drawing data of other categories? Is that so?


    for (l_frame = batch_meta->frame_meta_list; l_frame != NULL; l_frame = l_frame->next) {
        NvDsFrameMeta *frame_meta = (NvDsFrameMeta *) (l_frame->data);

        for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;) {
            obj_meta = (NvDsObjectMeta *) (l_obj->data);
            NvDsMetaList *obj_next = l_obj->next;
            if (obj_meta == NULL || obj_meta->confidence < 0){
                //nvds_remove_obj_meta_from_frame(frame_meta,obj_meta);
                l_obj = obj_next;
                continue;
            }
            if(ZHYPipeline::zhyPostprocess->confenable && confFilterFlag)
            {
                if(obj_meta->confidence < (*(ZHYPipeline::zhyPostprocess->confidenceMap))[obj_meta->obj_label])
                {
                    nvds_remove_obj_meta_from_frame(frame_meta,obj_meta);
                    l_obj = obj_next;
                    continue;
                }
            }
8

yes, tracker will add object meta for tracked objects if the objects don’t appear in current frame. you can add the probe on sink of tracker to filter the objects.

1 Like
9

Thank you for your reply. The method you mentioned is effective, but I encountered a new issue when I tried to use gst_ element_state (pipeline ->pipeline, GST_STATE_NULL); When the code destroys the pipeline, the processing in the probe function will report a segmentation error. Do I need to remove all probe functions before executing the code, or is there any other good way？

10

please narrow down this issue to find which code will cause the error. you don’t need to remove probe functions after setting NULL state.

11

I debug it use GDB tool, it displays that the error occurs at

nvds_obj_enc_process ((NvDsObjEncCtxHandle)ctx, &frameData, ip_surf, NULL, frame_meta);
nvds_obj_enc_finish ((NvDsObjEncCtxHandle)ctx);

if you need any other information, I can try to reproduce it。

by the way,This problem did not occur again after I tried to add the following code:

    PrintI("ZHYPipeline: pipeline: Running...\n");
    g_main_loop_run (loop);
 +   if (remove_tracker_probe_id != 0)
 +  {
 +      GstPad *sink_pad = gst_element_get_static_pad(gES.tracker, "sink");
 +       gst_pad_remove_probe(sink_pad, remove_tracker_probe_id);		
 +       gst_object_unref(sink_pad);
 +       remove_tracker_probe_id = 0;
 +    }

    nvds_obj_enc_destroy_context (obj_ctx_handle);

    if (bus_watch_id) {
        PrintW("ZHYPipeline::Returned, stopping playback normal\n");
        gst_element_set_state (gES.pipeline, GST_STATE_NULL);
        PrintW("ZHYPipeline::Deleting pipeline normal\n");
        gst_object_unref (GST_OBJECT (gES.pipeline));
        g_source_remove(bus_watch_id);
    }
12

Thanks for the sharing! if you are using C++, please refer to deepstream_test1_app.c. it is reasonable to free resource after setting NULL state.

13

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.