Not able to generate Re-id vector

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) rtx 3060
• DeepStream Version 7.1

• TensorRT Version10.3
**• NVIDIA GPU Driver Version (valid for GPU only)**560.35.03

@fanzh I have edited my config_tracker_NvDCF_perf.yml by adding this to generate the re-id embedding vectors.

ReID:
reidType: 2 # The type of reid among { DUMMY=0, NvDEEPSORT=1, Reid based reassoc=2, both NvDEEPSORT and reid based reassoc=3}

[Reid Network Info]

batchSize: 100 # Batch size of reid network
workspaceSize: 1000 # Workspace size to be used by reid engine, in MB
reidFeatureSize: 256 # Size of reid feature
reidHistorySize: 100 # Max number of reid features kept for one object
inferDims: [3, 256, 128] # Reid network input dimension CHW or HWC based on inputOrder
networkMode: 1 # Reid network inference precision mode among {fp32=0, fp16=1, int8=2 }

[Input Preprocessing]

inputOrder: 0 # Reid network input order among { NCHW=0, NHWC=1 }. Batch will be converted to the specified order before reid input.
colorFormat: 0 # Reid network input color format among {RGB=0, BGR=1 }. Batch will be converted to the specified color before reid input.
offsets: [123.6750, 116.2800, 103.5300] # Array of values to be subtracted from each input channel, with length equal to number of channels
netScaleFactor: 0.01735207 # Scaling factor for reid network input after substracting offsets
keepAspc: 1 # Whether to keep aspc ratio when resizing input objects for reid

[Output Postprocessing]

addFeatureNormalization: 1 # If reid feature is not normalized in network, adding normalization on output so each reid feature has l2 norm equal to 1
minVisibility4GalleryUpdate: 0.6 # Add ReID embedding to the gallery only if the visibility is not lower than this
outputReidTensor: 1

[Paths and Names]

tltEncodedModel: “/opt/nvidia/deepstream/deepstream-7.1/samples/models/Tracker/resnet50_market1501.etlt” # NVIDIA TAO model path
tltModelKey: “nvidia_tao” # NVIDIA TAO model key
modelEngineFile: “/opt/nvidia/deepstream/deepstream-7.1/samples/models/Tracker/resnet50_market1501.etlt_b100_gpu0_fp16.engine” # Engine file path

But this setup is not generating the engine file therefore the reid vectors are not generated. what is goig wrong here? do i need to additionally install anything?

3

please refer to sources/tracker_ReID/README in DeepStream SDK for how to prepare resnet50_market1501.etlt, which will be converted to the modelEngineFile.

4

I have set up by following that but during runtime the engine file is not being generated.

5

@fanzh do i need to install any tao packages inside my container?

6

could you share the result of “/opt/nvidia/deepstream/deepstream/samples/models/Tracker”? could you a share a complete running log?

7

Hi , the issue was sorted but now objects with same object id doesnt has obj_meta.obj_user_meta_list in different frames . therefore the embedding vector is not generated every frame for same object. i have set reidExtractionInterval: 1 , evethough same result .can you tell me what is causing this?

8

which sample are you testing or referring to? what is the complete media pipeline? where di you add the probe function? do you mean obj_meta.obj_user_meta_list is empty? Since nvtracker plugin is opensource, you can add logs in NvTrackerProc::updateFrameMeta to check.

9

I am using the tao re-identificationnet on ships to generate embedding vector for re-id . i added the probe function just after the tracker . some objects are not generating vectors in random frames. the same object is able to generate vector in other frames. can it be due to some config issue or the reid model not generating vector for certain cropped object?

10
  1. please refer to my last comments. do you mean obj_meta.obj_user_meta_list is empty? or obj_meta.obj_user_meta_list is not empty, but pReidObj->ptr_dev/pReidObj->ptr_host is empty?
  2. could you share a running log? wondering if there are some abnormal logs. can you use deepstream-test2 to reproduce this issue? if so, could you share the test video and complete tracker cfgs? Thanks!
12

1.obj_meta.obj_user_meta_list is empty is empty for some objects leading to no generation of the re-id vector while its available for some objects.

14

plesae try “reidExtractionInterval: 0”. reidExtractionInterval means “Frame interval to extract ReID features per target for re-association; -1 means only extracting the beginning frame per target”.

15

it shows for the objects with the detection confidence is negative (-0.1) obj_meta.obj_user_meta_list is empty even though i set minDetectorConfidence: -1 in config . Is That DeepStream may not extract ReID features for very low-confidence or negative-confidence detections? also what can be the reason for the negative confidence? is that an export issue?

16

yes, if the object confidence is smaller than the set minDetectorConfidence, code will not get the REID of the object. if nvinfer plugin is used, you can use pre-cluster-threshold to filter the objects having the low confidence. please refer to opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-test1\dstest1_pgie_config.txt

17

is the negative confidence reasonable? is this a wrong bboxes? are all bboxes wrong?
If there is model accuracy issue, please refer to this faq.

18

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.