NvDCF Tracker bbox in DeepStream 6.0

[angel.corrales]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson NX
• DeepStream Version 6.0
• Issue Type( questions, new requirements, bugs) Question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

I am working with a DeepStream pipeline that includes nvinfer for object detection, followed by an nvtracker using the NvDCF tracker for object tracking. The tracker reassigns the same ID to objects after occlusion, which shows it’s functioning correctly. However, I’m encountering a challenge in visualizing the bounding boxes (bbox) during occlusion periods.

When an object undergoes an occlusion either by becoming non-visible or when my inference model fails to detect it, nvdsosd does not display any bbox until the object is detected again or the occlusion ends. The assigned ID is the same as before the occlusion, so the tracker is working correctly, but the bbox is not visualized during that period.

I’ve read about the capabilities of NvDCF in handling occlusions effectively in NVIDIA’s SDK 6.2 blog post. In the following image, we can see how the bbox of ID 10 is represented despite it being occluded:

My question is: How can I configure nvtracker or nvdsosd in my DeepStream pipeline to display the bounding box estimations of NVDCF during occlusions? Are there specific settings or methods that would enable this visualization? Any code examples, documentation, or guidance on achieving this would be very helpful.

Thank you in advance for your assistance.

[angel.corrales]

I have seen in the following post that in the past, it was possible to adjust the minTrackingConfidenceDuringInactive to achieve my goal, but apparently in DS6 this is deprecated (!! [WARNING][NvTrackerParamsTargetManager] Unknown param found: minTrackingConfidenceDuringInactive).

[kesong]

It is expected behave that not show bbox during occlusion. Why you need bbox during occlusion?

[angel.corrales]

It’s not so much about the moments of occlusion, but rather about those times when my nvinfer fails to detect an object, even though it is actually present in the image. This scenario occurs quite frequently in my system. During these periods of “non-detections”, the tracker provides me with more consistency in detections.

I operate a real-time system whose main goal is to avoid False Negatives (FN) at all costs. I am not overly concerned about the presence of False Positives (FP), as I have other systems in later stages of my workflow that take care of filtering and eliminating these FPs.

I have tried to force the values of the rect_params to match those of the tracker_bbox_info detailed in NVIDIA DeepStream SDK Documentation, but this approach has not worked either. I’ve noticed that tracker_bbox_info is empty during periods of non-detection.

Given this context, do you have any suggestions or recommendations for improving detection in DeepStream 6.0, considering these specific needs? Any guidance or alternative approaches that I can integrate would be greatly appreciated.

[kesong]

Can you fix the “non-detections” in nvinfer? such as low the the threshold of detect confidence. Maybe you can check if you can get those bbox after enable “past-frame” in nvtracker: Troubleshooting — DeepStream 6.3 Release documentation (nvidia.com)

[angel.corrales]

I have already adjusted the thresholds of nvinfer to 0.0 (post-cluster-threshold, pre-cluster-threshold, …) and tried enabling the past-frame feature, but unfortunately, these steps did not resolve the issue.

[kesong]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Suppose nvtracker will report “past-frame” which tracking target in shadow tracking mode. Can you show the log of “past-frame”? Please refer write_kitti_past_track_output() in /opt/nvidia/deepstream/deepstream-6.3/sources/apps/sample_apps/deepstream-app/deepstream_app.c to log the “past-frame”.