VPI DCF Tracker - outMaxCorrelationResponses always return 0

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
, ,
1

Hello. We are currently developing a custom DeepStream plugin by modifying gst-dsexample. In this plugin, we are integrating the DCF tracker offered by VPI in order to have an inference-free object tracking algorithm (from my understanding, the NvTracker plugin from DeepStream is dependent of NvInfer, as stated here).

Currently, we have managed to have a functional tracker without Re-ID and no state transition management. For both, we are thinking about using the outMaxCorrelationResponses output from vpiSubmitDCFTrackerLocalizeBatch. However, it seems that we cannot get the outMaxCorrelationResponses data from that function, as it only produces the value 0.

We have already looked into this post as a solution but in our case, the result is still 0. Keep in mind that the tracker is still able to track the object.

I’ll attach the source code below. Could someone give us some insights on how to fix this problem? We are using:

  • VPI Version: 3.2.4
  • DeepStreamSDK: 7.1.0
  • CUDA Driver Version: 12.6
  • CUDA Runtime Version: 12.6
  • TensorRT Version: 10.3

Thank you. Best regards.
gstdspluginsample.zip (6.7 KB)

3

Hi,

Could you try to run VPI independently to see if this issue is related to Deepstream or not first?
Thanks.

4

Hello @AastaLLL . Thank you for your response.

If i understood you correctly, we tested the sample VPI DCF Tracker code by itself:

./vpi_sample_19_dcf_tracker cuda ../assets/pedestrians.mp4 ../assets/pedestrians_bboxes.txt

and made the necessary changes to see if outMaxCorrelationResponses was being populated, similar to this. The result is still zero, even with vpiStreamSync after vpiSubmitDCFTrackerLocalizeBatch and before locking the array. Furthermore, I used vpiArrayGetSize to see if there were changes regarding the size of the array and none were observed; the size is always zero.

Kindly waiting for updates. Thank you in advance!
jamorim

5

Hello @AastaLLL . I managed to solve the problem. I changed the array buffer type to:

CHECK_VPI_STATUS(vpiArrayLockData(dspluginsample->outMaxCorrelationResponses, VPI_LOCK_READ, VPI_ARRAY_BUFFER_CUDA_AOS, &outMaxCorrelationResponsesData));

All works as intended now. Thank you. Best regards!

6

Hi,

Thanks a lot for the update.
Good to know it works now.

7

Hello @jamorim,
seems like I had a similar idea last year. it is possible to track objects using Deepstreams gst-nvtracker without inference, however, the functionality is limited (at least in Deepstream 6.2). You can refer to (Gst-nvtracker) Tracking of manually inserted objects for a code snippet.

NvDCF was working quite well after some parameter tuning, but:

  • the bounding box size remains fixed (which is bad if the object is moving away). According to the VPI documentation, scaling changes are also not implemented in VPI DCF Tracker. Do you have some experience here?
  • the tracker does not redetect the object after it is lost
  • thus, ReID also does not work.
8

Hello @patrick2 . Thank you for your response.

We stumbled across your post actually, when we first started working on this. It was very intriguing but we never actually managed to have a working pipeline with NvTracker only. We might try again in the future but for now, we are sticking with the VPI tracker. Also, your info is very accurate, as we dealt with a lot of tracking loss problems.

As for the rescaling of the bounding box, we are working on a solution but it is still in a very early stage. We might share some results as soon as it is working.

Thank you again. Best regards

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.