Secondary model can't catch the event with tracker

• Hardware Platform (Jetson / GPU) NVIDIA TESLA T4
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.0.1-1+cuda11.3
• NVIDIA GPU Driver Version (valid for GPU only) 535.86.05
• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

[primary-gie]
enable=1
gpu-id=0
model-engine-file=../../models/yolov8/yolov8n_b30_gpu0_fp16.engine
batch-size=30
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV8.txt

[secondary-gie0]
enable=1
gpu-id=0
model-engine-file=../../models/swoon_resnet/model.etlt_b30_gpu0_int8.engine
batch-size=30
#Required by the app for OSD, not a plugin property
bbox-border-color0=0.5;0;1;1
bbox-border-color1=0;0.5;0.5;1
bbox-border-color2=0;1;1;1
bbox-border-color3=0.2;1;0.5;1
interval=0
gie-unique-id=2
nvbuf-memory-type=0
config-file=config_swoon_classification.txt
operate-on-gie-id=1
operate-on-class-ids=0
#operate-on-class-ids=0;1;2

[tracker]
enable=1
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
ll-config-file=config_tracker_NvDCF_perf.yml
gpu-id=0
enable-batch-process=1
enable-past-frame=1
display-tracking-id=1

The secondary model effectively identifies swoon events without the tracker; however, when tracking is enabled, it fails to detect the swoon event at same video.

What is the mean? Can you share some code to explain more?

In my situation, Secondary model is Resnet 18 that classify scenes people fall.
As shown in the first picture below, in the same scene, an event can be detected without a tracker, but when a tracker is used, the event cannot be detected.

And looking at the third image, if the tracking ID changes during video detection, an event is detected, but even after the person stands up again, it is recognized as having fallen.

ClipboardImage_2023-11-30_094451
ClipboardImage_2023-11-30_094250
ClipboardImage_2023-11-30_095838

Can you help to check if classifier-async-mode value setted in your SGIE config? Please disalbe it if you set it.

I am using a “nvmultiobjecttracker” tracker with Resnet 18 secondary model , And I tested both 0 and 1 values ​​of classifier-async-mode, but the results were no different.

Can you have a try to set probationAge to 0 in nvtracker config?

I tested it after setting probationAge to 0 and disabling classifier-async-mode, but the result was the same as in that video.

So you need to compare the difference of nvinfer input between enable/disable tracker. Can you share the log of BBox and tracker id when enable/disable nvtracker?

This is the deepstream analysis log from the same video when enable/disable nvtracker

The order of column names is attached below

frame_num, BBOX coordinate(left, top, width, height), tracking_id, pgie class label(confidence), sgie class label(confidence)

tracking_on_log.txt (130.4 KB)
tracking_off_log.txt (131.7 KB)

Could you please review the attached log file and provide feedback? If the information is incomplete or differs from your expectations, kindly let me know, and I would provide additional data.

I found the tracking on log miss two object at the begin. Can you have a try with set probationAge to 0 with NvDeepSORT tracker? Can you also share your tracker configure file if you still don’t fix your issue?

Testing is currently unfeasible due to the poor performance of PeopleNet and TrafficCamNet models, as they fail to detect individuals in videos. I’m curious to know if Deepsort also has compatibility with Yolo.

I am anticipating your response to address the issue, and I am prepared to furnish any required files for an analysis.