Probability correlation between nvtracker and nvinfer (sgie)

[marmikshah]

Hello,
I have a pipeline with the inference plugins as: nvinfer (detectnet_v2 pgie) -> nvtracker (dcf) -> nvinfer (resnet18 sgie)
Now, I am trying to tune the tracker and I noticed that when I reduce the following properties:

    minMatchingScore4Overall
    minMatchingScore4SizeSimilarity
    minMatchingScore4Iou
    minMatchingScore4VisualSimilarity

The probability output by the SGIE seems to increase alot (for the same object and is stable).
For example, when my configuration was:

    minMatchingScore4Overall: 0.0 # Min total score
    minMatchingScore4SizeSimilarity: 0.0 # Min bbox size similarity score
    minMatchingScore4Iou: 0.0 # Min IOU score
    minMatchingScore4VisualSimilarity: 0.0 # Min visual similarity score

the SGIE’s label.result_prob seems to be very high (stable around 0.95) when compared to this setup:

    minMatchingScore4Overall: 0.0 # Min total score
    minMatchingScore4SizeSimilarity: 0.1 # Min bbox size similarity score
    minMatchingScore4Iou: 0.25 # Min IOU score
    minMatchingScore4VisualSimilarity: 0.25 # Min visual similarity score

where the probability fluctuates alot more from 0.6 to 0.8.

Is this an expected behavior? How should I go about tuning the tracker such that the SGIE’s probability is consistent?
The videos that I test on will always have 1 object only, but it moves quite frequently so I am trying to lower these values as the tracker seems to perform much better with that.

NOTE: All my models are trained using TAO Toolkit.

Thanks.

• Hardware Platform (Jetson / GPU): Both
• DeepStream Version: 6.0.1
• JetPack Version (valid for Jetson only) 4.6
• TensorRT Version: 8.0.1
• NVIDIA GPU Driver Version (valid for GPU only): 510
• Issue Type( questions, new requirements, bugs): Question

[marmikshah]

FYI, here is the whole tracker config tracker.yml (3.7 KB)

[Amycao]

Sorry for the late.
Will check, and get back to you.

[Amycao]

In SGIE config files, if secondary-reinfer-interval is not set, then its default value is INT_MAX, meaning SGIE only infers the first frame for each object and stores its prob. you changed tracker params, so the object frame goes into SGIE can be different.

[marmikshah]

That makes sense.
Thank you for your reply.

I ended up switching the position of SGIE and Tracker. (So now my pipeline is PGIE → SGIE → Tracker).

Based on your reply, I also have some more observations (for anyone who’s stuck on a similar issue).
If your pipeline is PGIE → Tracker → SGIE

1. There will be some frames when the bboxes will be discarded (maybe need to tune some tracker params to not discard them?)
2. The tracker is adjusting bbox params so my SGIE gets a different box than what the PGIE generates. (This was my main cause for getting invalid results)
3. The stricter minMatching* values, the lesses bboxes you will observe.
   For my test, when I lower the minMatching* values, an object is observed in about 170 frames whereas when I increase it, the same object is observed in much less (80-100) frames.

Thanks.