DeepStream 6.3 NvTracker reidType Bug

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform: GPU
• DeepStream Version: 6.3
• TensorRT Version: 8.5.3-1
• NVIDIA GPU Driver Version: 530.30.02
• Issue Type: bug
• sample app used: deepstream-test2 with nvtracker_config attached (used new reidType: 3)
• Requirement details: NvTracker plugin

Hello!

I am trying to set up new reidType (both NvDEEPSORT and reid based reassoc=3) in my deepstream app and getting this warning message:

(changed settings are visualTrackerType: 0 and reidType: 3
config_tracker_NvDCF_accuracy.yml (9.0 KB)
to get this warnings)

~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 85]: [NvMultiObjectTracker][DataAssociator][Warning] Visual tracker disabled. minMatchingScore4VisualSimilarity is not used
~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 89]: [NvMultiObjectTracker][DataAssociator][Warning] Visual tracker disabled. matchingScoreWeight4VisualSimilarity is not used
~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 114]: [NvMultiObjectTracker][DataAssociator][Warning] Reid disabled. minMatchingScore4ReidSimilarity is not used
~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 118]: [NvMultiObjectTracker][DataAssociator][Warning] Reid disabled. matchingScoreWeight4ReidSimilarity is not used

(minMatchingScore4VisualSimilarity was not deleted from config on purpose to show that Visual tracker and reid tracker are disabled.)
So, instead of using NvDEEPSORT and reid based reassoc I am getting Reid disabled.

Actually I tried to get NvTracker to use both visual and reid trackers:

In the Data Association paragraph there is this formula

totalScore=w1∗IOU+w2∗sizeSimilarity+w3∗reidSimilarity+w4∗visualSimilarity

So, is there any possible NvTracking setting to use this formula for data association (NvDeepSORT and Visual tracker simultaneouly) ?

Thank you!

2

Currently reidSimilarity and visualSimilarity are not supported by data association at the same time. Suggest using NvDCF_accuracy for better perf-accuracy tradeoff. It uses visualSimilarity for data association and reidSimilarity for re-association.

3

So, why both terms are present in this formula?

And, reidType: 3 setting - what is it for? What tracking it is using? If ReID tracking ,these 2 warning are printed by some error?

4

That’s a valid point. The formula should look like this:

totalScore=w1∗IOU+w2∗sizeSimilarity+{w3∗reidSimilarity, w4∗visualSimilarity}

“reidType: 3 setting - what is it for?”

Please refer to NvDCF_accuracy.yml config file. reidType:3 is to enable ReID-based target re-association.

5

Actually NvDCF_accuracy.yml has set reidType:2, not 3…
More over with reidType:3 application keep spamming this:

[ERROR push 353] push failed [-2]
[ERROR push 353] push failed [-2]
[ERROR push 353] push failed [-2]
[ERROR push 353] push failed [-2]

As stated here:

The type of Re-ID network among { DUMMY=0, NvDEEPSORT=1, Reid based reassoc=2, both NvDEEPSORT and reid based reassoc=3 }

So, reidType:3 should work on NvDeepSort, but I am getting

~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 114]: [NvMultiObjectTracker][DataAssociator][Warning] Reid disabled. minMatchingScore4ReidSimilarity is not used
~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 118]: [NvMultiObjectTracker][DataAssociator][Warning] Reid disabled. matchingScoreWeight4ReidSimilarity is not used

Where is the truth?

7

Can you share us your use case? Which feature do you want to use? such as NvDCF or NvDeepSORT. need Re-ID re-association, need Re-ID feature output. So we can verify and share the right configure file to you.

8

Hello!
My use case - tracking football players. I want to use DeepSORT with ReID reassoc. So, I guessed that reidType=3 is my case (both NvDEEPSORT and reid based reassoc=3 as stated in config parameters).

But, when I using this config_tracker_reidType_3.yml (8.0 KB), I am getting this warnings:

~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 114]: [NvMultiObjectTracker][DataAssociator][Warning] Reid disabled. minMatchingScore4ReidSimilarity is not used
~~ CLOG[src/modules/DataAssociator/DataAssociator.cpp, initializeSimilarityParams() @line 118]: [NvMultiObjectTracker][DataAssociator][Warning] Reid disabled. matchingScoreWeight4ReidSimilarity is not used

So I conclude that reID is not working. But what type of tracking is working when I set reidType=3?

P.S. Actually I want to use tracking scheme that involves both visual filters and reid (like in the formula from documentation:

totalScore=w1∗IOU+w2∗sizeSimilarity+w3∗reidSimilarity+w4∗visualSimilarity

But it is not possible, for now at least.( I don’t understand why we cant compute both reidSimilarity and visualSimilarity at the same time, and just sum them up).

P.S.S. It’s possible that we will someday see the source code of libnvds_nvmultiobjecttracker.so ?

10

Also, could you please explain:

  1. How parameters of this NvDeepSORT config were calculated? I mean this exact values (0.6974, 0.8049 etc.)

  2. How these noiseWeightVar4Loc and noiseWeightVar4Vel parameters work? what this noise stands for (process or measurement)?

Thank you!

11

DeepStream can’t support NvDeepSORT with re-association currently. We will implement it in the future release. Can you use the default ./samples/configs/deepstream-app/config_tracker_NvDCF_accuracy.yml for your use case? Is there any issue if use the default config_tracker_NvDCF_accuracy.yml?

12

From configuration parameters:

reidType: The type of Re-ID network among { DUMMY=0, NvDEEPSORT=1, Reid based reassoc=2, both NvDEEPSORT and reid based reassoc=3 }

I guessed it should work, but you are telling me it don’t. So can you please fix that at the documentation page? Actually this lines were added to documentation in deepstream 6.2 release. Or this documentation from the future release and we will see it soon? This is really strange.

config_tracker_NvDCF_accuracy.yml works like a charm, but I need to get access to ReID tensors for all detected objects. And I cant use probationAge: 0 because it gives me a lot of short tracks. So my only choice is NvDeepSORT (because it calculates ReID tensors for all objects no matter what).

Can you please also answer these 3 questions:

Thank you!

13

The answers to above 3 questions are

  • So far it’s not ready to release that part of source code.
  • The parameter values tuned the parameters using an internal dataset using a semi-automated tool. We are considering to make such a tool as a part of the product, but no concrete plan yet. Customers are encouraged to looks at our Tuning Guide to perform custom tuning.
  • DeepSORT implementation makes both process and measurement noise proportional to bounding box height. The two params are the weight coefficients, but we tuned their values instead of using the values in the original implementation.
14

Hello!

Thank you for the answers to above 3 questions!

But can you also comment this one?

Why is the feature added in version 6.2 still not working?

15

Thanks for your report. We will fix it in the future release.