Tracker between detection and classification

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) AGX Orin
• DeepStream Version 6.2

I have a built a face detection recognition pipeline, and then i have enabled the tracker between the detection and classifier. “Detect → Track → Recognize” However tracker caused false classification to many faces, and even for the first frame that goes to the secondary nvinfer, most faces are wrongly identified, which is not the case when tracker is disabled.
Is there a way to improve this? i have tested (NvDCF,IOU & NvSORT trackers)

3

Can you share some video or image to show how “wrongly identified”?
Do your enable “classifier-async-mode” in the classifier config? Gst-nvinfer — DeepStream documentation 6.4 documentation

4

I didn’t enable it because when i do so the classifier will not work at all. Only detection will be performed not sure why. and the below will be shown in command window
**H264: Profile = 66, Level = 0 **
**NVMEDIA: Need to set EMC bandwidth : 376000 **
NVMEDIA_ENC: bBlitMode is set to TRUE

The below images are when tracker in enabled and all faces in the video are classified as “David” face#2 which is wrong classification. The same ID will be assigned to all personal in the video
image

image
image485×525 36.9 KB

However the correct classification are as below where these are the output without tracker enabled
image
image
image456×508 34 KB

5

please advise

6

Can you share the configure file of the SGIE?

7

[property]
gpu-id=0
net-scale-factor=0.0039215686274
#force-implicit-batch-dim=1
onnx-file=models/facenet/facenet.onnx
model-engine-file=models/facenet/facenet.onnx_b1_gpu0_fp16.engine
#model-engine-file=models/facenet/facenet.onnx_b1_dla1_fp16.engine
batch-size=1

0=FP32 and 1=INT8 mode

network-mode=0
operate-on-class-ids=0
process-mode=2
model-color-format=0
gie-unique-id=2
operate-on-gie-id=1
#operate-on-class-ids=2
tensor-meta-pool-size=30
is-classifier=1
classifier-async-mode=0
classifier-threshold=0.51

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

output-tensor-meta=1

8

Can you add some log in probe function to check if there is changes on BBoxes which send to SGIE during enable/disable nvtracker?

9

please advise how to add

10

Can you refer function: osd_sink_pad_buffer_probe() in: /opt/nvidia/deepstream/deepstream-6.4/sources/apps/sample_apps/deepstream-test2/deepstream_test2_app.c?

11

i have the exact function in my code but i still didnt get