Please provide complete information as applicable to your setup.
**• Hardware Platform - GPU (2 x NVIDIA GeForce RTX 3090)
**• DeepStream Version - 6.1
• JetPack Version (valid for Jetson only)
**• TensorRT Version - v8401
**• NVIDIA GPU Driver Version - 515.105.01
**• Issue Type - bug
**• How to reproduce the issue ? - using example deepstream-test2
When running Deepstream6.1 pipeline on a 2 GPU (RTX3090) server, the behavior of nvtracker plugin differs significantly between GPU 0 Vs GPU 1. It looks like on GPU 1 nvtracker fails to re-acquire object track if it was not detected by object detector on more than 2-3 frames (while maximum allowed shadowTrackingAge parameter was set to 15)
The issues is pretty easy to reproduce in the following way :
- use deepstream-test2 sample from deepstream6.1 library
- add detected objects printout to osd_sink_pad_buffer_probe function (deepstream_test2_app.c) :
[line 119] printf(“VID[%u] FID[%d] CID[%d] OID[%lu] BBOX[%.3f %.3f %.3f %.3f]\n”, frame_meta->source_id, frame_meta->frame_num, obj_meta->class_id, obj_meta->object_id, obj_meta->rect_params.top, obj_meta->rect_params.left, obj_meta->rect_params.width, obj_meta->rect_params.height);
After running the example on GPU 0 and GPU 1 (you will need to change gpu-id settings in all configuration files inside the sample folder from “gpu-id : 0” to “gpu-id: 1”) you can notice that :
- number of different object instances is the same
- last object id on GPU 1 is ~1.5X higher compared to the results on GPU 0
Basically this means that on GPU0 tracker follows the same object track for longer.
It is also easy to find specific tracks where this happens - for example, in my case, I have found a track which is continuously followed on GPU 0 for ~400 frames, while on GPU 1 it is covered by 4 different object ids. The change in object id always happens when the object was not detected by object detector for more than 2-3 frames (on GPU 0 the same tracker is able to correctly re-id the object even when it was not detected for 10 frames)
Please advice on ways to fix this issue or some sort of a workaround.