VPI DCF Tracker outCorrelationResponses

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
, , ,
1

• Hardware Platform (Jetson / GPU) Jetson Orin Nano
• JetPack Version (valid for Jetson only) 6.2.1
• VPI Version 3.2
• DeepStream Version 7.1
• Issue Type (questions, new requirements, bugs) Questions

I’m experiencing an issue with the VPI DCF Tracker outCorrelationResponses and outMaxCorrelationResponses parameters, it seems these arrays are never populated:

...
CHECK_STATUS(vpiArrayCreate(MAX_TRACKED_TARGETS, VPI_ARRAY_TYPE_DCF_TRACKED_BOUNDING_BOX, 0, &in_targets));
CHECK_STATUS(vpiArrayCreate(MAX_TRACKED_TARGETS, VPI_ARRAY_TYPE_DCF_TRACKED_BOUNDING_BOX, 0, &out_targets));
CHECK_STATUS(vpiImageCreate(dcfInitParams.featurePatchSize, dcfInitParams.featurePatchSize * MAX_TRACKED_TARGETS, VPI_IMAGE_FORMAT_F32, 0, &out_correlations));
CHECK_STATUS(vpiArrayCreate(MAX_TRACKED_TARGETS, VPI_ARRAY_TYPE_F32, 0, &out_max_correlations));
...
CHECK_STATUS(vpiSubmitDCFTrackerLocalizeBatch(stream, 0, dcf, NULL, 0, NULL, patches, in_targets, out_targets, out_correlations, out_max_correlations, NULL));
CHECK_STATUS(vpiStreamSync(stream));
...
CHECK_STATUS(vpiSubmitDCFTrackerUpdateBatch(stream, 0, dcf, NULL, 0, NULL, NULL, patches, targets, NULL));
CHECK_STATUS(vpiStreamSync(stream));
CHECK_VPI_STATUS(vpiArrayLockData(targets, VPI_LOCK_READ, VPI_ARRAY_BUFFER_HOST_AOS, &targets_data));
CHECK_VPI_STATUS(vpiArrayLockData(out_max_correlations, VPI_LOCK_READ, VPI_ARRAY_BUFFER_HOST_AOS, &max_correlation_data));
// Here *targets_data.buffer.aos.sizePointer == 1 but *max_correlation_data.buffer.aos.sizePointer == 0 (even when state == VPI_TRACKING_STATE_TRACKED)
  1. Am I doing something wrong?
  2. Once this works, is outMaxCorrelationResponses actually the confidence to be used when deciding whether tracking is lost?

Thank you in advance.

5

Hi,

Are you able to share more details about your issue?
Is there any error message, or does the algorithm fail to work due to the parameter?

Thanks.

6

Hello @AastaLLL,

Thank you for your reply.

As stated in the above code comments, objects are being tracked - outObjects is being populated and bbox updated, but outCorrelationResponses/outMaxCorrelationResponses aren’t. The function signature is:

VPIStatus vpiSubmitDCFTrackerLocalizeBatch(VPIStream stream, uint64_t backend, VPIPayload payload, const int32_t *enabledSequences, int32_t numSequences, VPIImage featureMaskingWindow, VPIImage inPatches, VPIArray inObjects, VPIArray outObjects, VPIImage outCorrelationResponses, VPIArray outMaxCorrelationResponses, const VPIDCFTrackerParams *params)

Since this seems to be undocumented, can you please explain/verify which conditions can cause outMaxCorrelationResponses to not be populated even though a valid VPIArray is being passed?

Thank you!

7

Hi @AastaLLL,

Any news regarding this?

Thank you!

8

Hi,

Could you check if below comment also works for your issue?

Thanks.

9

Hello @AastaLLL, thank you for your reply.

Could you check if below comment also works for your issue?

Well, as seen in the code snippet above (from December 7th):

...
CHECK_STATUS(vpiStreamSync(stream));
CHECK_VPI_STATUS(vpiArrayLockData(targets, VPI_LOCK_READ, VPI_ARRAY_BUFFER_HOST_AOS, &targets_data));
CHECK_VPI_STATUS(vpiArrayLockData(out_max_correlations, VPI_LOCK_READ, VPI_ARRAY_BUFFER_HOST_AOS, &max_correlation_data));

vpiStreamSync is being called immediately after vpiSubmitDCFTrackerLocalizeBatch / vpiSubmitDCFTrackerUpdateBatch and before accessing the arrays, so I failed to understand your point. Can you please verify if something else is wrong in the way shown above that can explain why outCorrelationResponses and outMaxCorrelationResponses arrays are always empty?

Thank you!

10

Hi

The main difference seems to be the vpiSubmitDCFTrackerUpdateBatch call.
Could you double-check if you can get the value before calling the update batch?

Thanks.

11

Hello @AastaLLL,

Ok, per your suggestion I’ve just tried the following:

CHECK_VPI_STATUS(vpiSubmitDCFTrackerLocalizeBatch(stream, vpiBackend, dcf, NULL, 0, NULL, patches, in_targets, out_targets, out_correlations, out_max_correlations, NULL));
CHECK_VPI_STATUS(vpiStreamSync(stream));

CHECK_VPI_STATUS(vpiArrayLockData(out_targets, VPI_LOCK_READ_WRITE, VPI_ARRAY_BUFFER_HOST_AOS, &targets_data));
CHECK_VPI_STATUS(vpiArrayLockData(out_max_correlations, VPI_LOCK_READ, VPI_ARRAY_BUFFER_HOST_AOS, &max_correlation_data));

//*targets_data.buffer.aos.sizePointer == 1 (successfully tracking target)
//but *max_correlation_data.buffer.aos.sizePointer == 0 always

Can you please verify the exact requirements for outMaxCorrelationResponses to be populated? Currently, no matter what I try, the array is always empty.

Thank you.

12

Hi,

We modify the below function and get the outMaxCorrelationResponses without issue:

CHECK_STATUS(vpiArrayLockData(outMaxCorrelationResponses, VPI_LOCK_READ, VPI_ARRAY_BUFFER_CUDA_AOS, &outMaxCorrelationResponsesData));

Thanks.