Inconsistent object IDs when running on multi source pipeline

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hello,
I have a multi source pipeline.
For the sake of testing, I am using the same video file on each source.
So my pipeline looks like:

src0 - -- \                                                              / ----------- sink0
             
src1 - -- \                                                              / ----------- sink1
              ---> nvstreammux -> nvinfer -> nvtracker -> nvstreamdemux -
src2 - -- /                                                              \ ----------- sink2

src3 - -- /                                                              \ ----------- sink3

Now the issue is when I run the pipeline multiple times, the total number of detected objects are always different even when it is the same video file for all sources.
The total objects reduce per source
For example:

src0: Total 56274 detections counted
src1: Total 56102 detections counted
src2: Total 55982 detections counted
src3: Total 55901 detections counted

I removed nvtracker from the pipeline and all the results match. So I suspect something is wrong on the nvtracker setting?

The following properties have been set for nvtracker

enable-batch-process: true
display-tracking-id: true
tracker-width: 960
tracker-height: 544

I have tested out DCF and IOU tracker configs provided in samples and it happens for both.
Is there any specific setting I am missing out?

Thanks.

• Hardware Platform (Jetson / GPU): GPU
• DeepStream Version: 6.1.1
• NVIDIA GPU Driver Version (valid for GPU only): 520.61.05
• Issue Type( questions, new requirements, bugs): Bugs

1 Like
3
  1. Can you see the issue with two sources?
  2. Do you set sync=false and qos=false in all sinks?
4

Hey,
Thanks for the the reply.

  1. This happens even for 2 sources.
  2. We have kept sync=true and not modified qos so it should be false by default. (We are testing with a fakesink)
5

Can you log frame number and object count for every frame? So we can narrow down it is caused by drop video frame or miss object in some video frame.

6

Sure, we did that actually.
The total frames for each source is exactly same but total detections for certain frames are different.
They match once we remove the tracker.

Attaching files (src0.txt and src1.txt) where each line in the text file is the total objects in that frame. I.e. value in line 1 will be total objects seen in frame ID 1.

src1 (5.1 KB)
src0 (5.1 KB)

7

Is it possible to share the reproduce?

8

Umm, let me prepare a minimum reproducible code for you to verify.
Will get back to you within the next day.

Thanks!

9

Hi, the problem could be reproduced with a python code based on the deepstream_python_apps example by Nvidia. Please try out the code in GitHub - ethanlee928/DeepStream.

When there are more than 1 sources, the total number of detection of all frames (please check out the global variable TOTAL_COUNT in the script) becomes inconsistent with respect to different sources of the same video, and also with respect to different runs. However, if tracker is disabled, the total count becomes consistent.
I have tested with nvidia’s peoplenet (resnet34) and also trafficnet (resnet18). For the tracker config, it is pushed on the above repo.

For trafficnet, the pipeline was tested with a traffic video: 5.3 4K Camera Road in Thailand No 3 Interchange - YouTube

10

Hi @kesong ,
My teammate @ethanlee has pepared a small repo for you to try to reproduce the problem. Please let us know if anything else is needed.

Thanks.

11

Hi @kesong
Any update?
Thanks.

12

We are checking it. Will update here ASAP.

13

Can you share your test video? I can’t get the video from YouTube. Or can you reproduce the issue with the video in fold: /opt/nvidia/deepstream/deepstream-6.1/samples/streams/? Can you reproduce with deepstream-app?

14

We have not used deepsteam-app but the current code that we used is based on the deepstream test apps from python repo. Will there be any major difference between the C/C++ version and the Python version?

As for the video, let us try it on the sample streams and get back in sometime. We will also send you a download link for the current videos that we have tested.

Thanks.

15

@kesong , this is the download link of that video.

We tested on the sample streams. This problem does not occur on the sample stream videos.
So we suspect it could be related to the number of objects per frame? We tried increasing maxTargetsPerStream but the problem still persists in the video shared above.

Is there any limitation to number of objects per frame?

Thanks.

17

Thanks for share the test video. We will check it. Thanks.

1 Like
18

Hey!
Is there any update?

Thanks.

19

Sorry for the late response, we are still working on it, will update status once we have confirmed schedule, thanks.

20

Alright, thank you. Looking forward to your response.

21

Hey @yingliu
We also tested this using Deepstream 6.2 and can confirm the issue still exists.