How to Reduce BBoxes Flickering on Deepstream

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) -Jetson Xavier NX
• DeepStream Version - 6
• JetPack Version (valid for Jetson only) - Jetpack 4.6 (32.6.1)
• TensorRT Version -
• Issue Type( questions, new requirements, bugs) Question

Hello Everyone, I am working on Xavier NX with Deepstream 6 for People Detection Using the Resnet-18 Model. According to my understanding, when the inference engine detects an object, the nvtracker is responsible to track the object. However with the configuration I am using, there seems to be occasional drop and flickering of the BBoxes. I would like to have the output with very smooth tracking and possibly no flickering at all. I am sharing the tracker configuration files below;

  • tracker_config_1.yml - First Tracker File with Interval=0 used in the PGIE config file

  • tracker_config_2.yml - Second Tracker File with Interval=2 used in the PGIE config file

tracker_config_1.yml (4.9 KB)
tracker_config_2.yml (6 KB)

I am also attaching the respective output videos for each of the configuration files



From the output video you can observer that with the second configuration the flickering has reduced for the close objects as compared to the first configuration but the flickering is still present for the object further away.

Your suggestions and ideas will be highly appreciated. looking forward to your responses.

Can you check this topic. Hope it helps.

Hello @kesong , Thank you for your reply. Unfortunately, the link you sent doesn’t help and I cannot observe any difference, Could you please have a look at it in detail and let me know what could be the possible solution

I shared topic said “past-frame” can help on flick issue.

Hii @kesong, thank you for your reply.
In the tracker properties enable-past-frames property has been set to true.
I also tired maxShadowTrackingAge=0 and earlyTerminationAge=0, but it did not make much of a difference.
My issue is that the bboxes themselves are pretty stable but they are not being tracked properly, the bboxes themselves get missed for certain number of frames and then they appear back. From the video you can see the same happening.
According to my understanding when the nvinfer plugin misses some object, the tracker comes in to fill for the missing frames thus making a consistent detection and tracking. When the tracker is activated these short flickers should not happen.
Can you please use the same tracker config files and try to replicate the issue at your end and then suggest me what parameters should be tweaked so have a better performance. I can provide you with the engine file for inference as well
Looking forward to your response

Can you check if there is “past-frame” meta data when the object appear back after disappear?

Hi @kesong, Could you please guide me how to do that ?

Please refer write_kitti_past_track_output() in /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-app/deepstream_app.c

Hello @kesong, Thank you for your reply. I referred to the function you suggested and implemented it. I am attaching the code and the output I got from it. I used 4 input streams so there are files for 4 steams.

 	* Function to dump past frame objs in kitti format.
	gchar bbox_file[1024] = { 0 };
  	FILE *bbox_params_dump_file = NULL;
    NvDsPastFrameObjBatch *pPastFrameObjBatch = NULL;
    NvDsUserMetaList *bmeta_list = NULL;
    NvDsUserMeta *user_meta = NULL;
    for(bmeta_list=batch_meta->batch_user_meta_list; bmeta_list!=NULL; bmeta_list=bmeta_list->next){
      user_meta = (NvDsUserMeta *)bmeta_list->data;
      if(user_meta && user_meta->base_meta.meta_type==NVDS_TRACKER_PAST_FRAME_META){
        pPastFrameObjBatch = (NvDsPastFrameObjBatch *) (user_meta->user_meta_data);
        for (uint si=0; si < pPastFrameObjBatch->numFilled; si++){
          NvDsPastFrameObjStream *objStream = (pPastFrameObjBatch->list) + si;
          guint stream_id = (guint)(objStream->streamID);
          for (uint li=0; li<objStream->numFilled; li++){
            NvDsPastFrameObjList *objList = (objStream->list) + li;
            for (uint oi=0; oi<objList->numObj; oi++) {
              NvDsPastFrameObj *obj = (objList->list) + oi;
              g_snprintf (bbox_file, sizeof (bbox_file) - 1,
                "%s/Stream=%01u_FrameNum=%04lu.txt", "/home/nvidia/tracker",
                stream_id, (gulong) obj->frameNum);

              float left = obj->tBbox.left;
              float right = left + obj->tBbox.width;
              float top = obj->;
              float bottom = top + obj->tBbox.height;
              // Past frame object confidence given by tracker
              float confidence = obj->confidence;
              bbox_params_dump_file = fopen (bbox_file, "a");
              if (!bbox_params_dump_file){
                "Label = %s, UniqeID = %lu, Left = %f, Top = %f, Right = %f, Bottom = %f, Confidence = %f\n",
                objList->objLabel, objList->uniqueId, left, top, right, bottom, confidence);
              fclose (bbox_params_dump_file);

The output files are below in the folder
past-tracker-data.tar.gz (139.5 KB)

Looking forward to your response

Do you see any past-frame log? I check some of you log. I don’t see it.

Hello @kesong, According to the code you suggested me, it generates the txt files for some frames and the values of Bbox inside that frame. The tar files contains the txt files for all the frames and all the streams for which PastFrame data was there I suppose

Eg. Output of the Stream=0_FrameNum=0013.txt

Label = person , UniqeID = 11, Left = 792,372253, Top = 296,502380, 
Right = 812,811707, Bottom = 345,840759, Confidence = 0,752573

Can you apply the BBox in the NvDsPastFrameObjBatch to the previously video frame to remove the flick?

Hello @kesong, Thank you for you reply. Can you please guide me how to do that ?

Suppose you can develop one plugin which buffer several video frame, so you can apply BBox in NvDsPastFrameObjBatch in previous video frame.

Hello @kesong, Thank you for your reply. Is there any other way to do it besides developing a plugin ?

Seems no.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.