Tracker removes object meta

Please provide complete information as applicable to your setup.

• Hardware Platform dGPU
• DeepStream Version 6.1
• TensorRT Version 8.4.1
• NVIDIA GPU Driver Version 515.65.01
• Issue Type bugs

I have reimplemented the DeepStream Gst-nvinfer plugin and I am able to run a regular pipeline and see bounding boxes on the GL sink using OSD. However, when I add the tracker after the primary detector, the bounding boxes disappear. First I suspected that the metadata is corrupted. I found out that before the tracker the object meta attached by the inference plugin is alright, but after the tracker it is no longer present. What could be causing this?. I am creating the object meta exactly like in the original infer source code.

Can you share you pipeline? Can you have a try with different tracker type: IOU/NVDCF/DEEPSORT?

I have tried IOU and I get the same behavior. The pipeline is a regular deepstream sample app pipeline with a single inference engine and a tracker right after. I printed the contents of an object meta in a pad probe before the tracker and it looks like this:

+++ unique_component_id: 0
+++ class_id: 0
+++ object_id: 18446744073709551615
+++ detector_bbox_info: 144.758911 123.025368 436.886322 316.691101
+++ tracker_bbox_info: 0.000000 0.000000 0.000000 0.000000
+++ rect_params: 144.758911 123.025368 436.886322 316.691101
+++ confidence: 0.903996
+++ tracker_confidence: 0.000000
+++ obj_label: car

IOU tracker shouldn’t remove the object meta. Is it possible to reproduce it in my side?

Sadly, the plugin is proprietary and valuable to us since it greatly improves efficiency by implementing a lock-free pre-processing, inference and post-processing architecture. Furthermore, it employs modern C++ and leverages ViFlow and ViCUDA, our open-source libraries that encapsulate the unsafe memory management of low level GStreamer and CUDA resources.

Our motivation for rewriting the plugin was the chaining of arbitrary models like body pose and classification, where one would need to generate bounding boxes on the fly for the relevant keypoints and pass them to the classifier. However, integrating it with opaque components is far too time consuming. Therefore, we will use pad probes for now to alter the individual model output for the original plugin.

I think open-sourcing key components like the tracker and the analytics plugins will facilitate the integration with custom components and incentivize the members of the ecosystem to contribute with their own tools just like we did in this sample.

Can you have a try to set probationAge to 0?

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvtracker.html

Just tried. The problem persists with both IOU and NVDCF.

Can you also check if batch and frame meta from the customized nvinfer is consistent with the official nvinfer. Both official nvinfer and nvinferserver are open source, so you can check how those data fields are filled. For example, if nvinfer performs detection for a frame, frameMeta.bInferDone needs to be set as true for tracker to perform tracking.

I can’t thank you enough, setting bInferDone in the frame meta worked. I have completely missed that one line from the gst-nvinfer source code. If you ever hear my name around let me know and the drinks will be on me.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.