DeepStream Secondary GIE issue in synchronous mode

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU)**T4
• DeepStream Version5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version7.0.0.11
**• NVIDIA GPU Driver Version (valid for GPU only)**440+

When I am running any deepstream application for a video of very low frames at high batch size(64). It is observed frequently the last two out frame timestamps are for secondary gie is 0 and instead of getting timestamp. This issue does not occur in lower batch sizes and in asynchronous mode . It happens only iin synchronous mode.

3

Can you elaborate more details? The app you run, the configuration files
How do you view the timestamp?
It is better to list the steps of how to reproduce the issue.

4

Hi @Fiona.Chen,
I am giving the area of where the -ve timestamp is coming

BATCH-NUM = 943**
Comp name = nvv4l2decoder16 in_system_timestamp = 1605072079559.451904 out_system_timestamp = 1605072079654.706055 component latency= 95.254150
Comp name = src_bin_muxer source_id = 3 pad_index = 3 frame_num = 444 in_system_timestamp = 1605072079654.736084 out_system_timestamp = 1605072079675.806885 component_latency = 21.070801
Comp name = nvv4l2decoder27 in_system_timestamp = 1605072079508.173096 out_system_timestamp = 1605072079645.302979 component latency= 137.129883
Comp name = src_bin_muxer source_id = 4 pad_index = 4 frame_num = 443 in_system_timestamp = 1605072079645.354980 out_system_timestamp = 1605072079675.806885 component_latency = 30.451904
Comp name = nvv4l2decoder10 in_system_timestamp = 1605072079563.352051 out_system_timestamp = 1605072079651.354980 component latency= 88.002930
Comp name = src_bin_muxer source_id = 6 pad_index = 6 frame_num = 444 in_system_timestamp = 1605072079651.397949 out_system_timestamp = 1605072079675.806885 component_latency = 24.408936
Comp name = nvv4l2decoder21 in_system_timestamp = 1605072079542.560059 out_system_timestamp = 1605072079644.019043 component latency= 101.458984
Comp name = src_bin_muxer source_id = 7 pad_index = 7 frame_num = 444 in_system_timestamp = 1605072079644.086914 out_system_timestamp = 1605072079675.806885 component_latency = 31.719971
Comp name = nvv4l2decoder7 in_system_timestamp = 1605072079500.864990 out_system_timestamp = 1605072079641.093018 component latency= 140.228027
Comp name = src_bin_muxer source_id = 9 pad_index = 9 frame_num = 443 in_system_timestamp = 1605072079641.125977 out_system_timestamp = 1605072079675.806885 component_latency = 34.680908
Comp name = nvv4l2decoder18 in_system_timestamp = 1605072079553.961914 out_system_timestamp = 1605072079647.616943 component latency= 93.655029
Comp name = src_bin_muxer source_id = 12 pad_index = 12 frame_num = 445 in_system_timestamp = 1605072079647.672119 out_system_timestamp = 1605072079675.806885 component_latency = 28.134766
Comp name = nvv4l2decoder29 in_system_timestamp = 1605072079540.907959 out_system_timestamp = 1605072079658.952881 component latency= 118.044922
Comp name = src_bin_muxer source_id = 14 pad_index = 14 frame_num = 444 in_system_timestamp = 1605072079658.986084 out_system_timestamp = 1605072079675.808105 component_latency = 16.822021
Comp name = nvv4l2decoder1 in_system_timestamp = 1605072079532.481934 out_system_timestamp = 1605072079666.704102 component latency= 134.222168
Comp name = src_bin_muxer source_id = 15 pad_index = 15 frame_num = 444 in_system_timestamp = 1605072079666.783936 out_system_timestamp = 1605072079675.808105 component_latency = 9.024170
Comp name = nvv4l2decoder30 in_system_timestamp = 1605072079538.857910 out_system_timestamp = 1605072079643.240967 component latency= 104.383057
Comp name = src_bin_muxer source_id = 16 pad_index = 16 frame_num = 443 in_system_timestamp = 1605072079643.288086 out_system_timestamp = 1605072079675.808105 component_latency = 32.520020
Comp name = nvv4l2decoder23 in_system_timestamp = 1605072079516.007080 out_system_timestamp = 1605072079642.083984 component latency= 126.076904
Comp name = src_bin_muxer source_id = 17 pad_index = 17 frame_num = 444 in_system_timestamp = 1605072079642.142090 out_system_timestamp = 1605072079675.808105 component_latency = 33.666016
Comp name = nvv4l2decoder15 in_system_timestamp = 1605072079511.649902 out_system_timestamp = 1605072079649.566895 component latency= 137.916992
Comp name = src_bin_muxer source_id = 20 pad_index = 20 frame_num = 443 in_system_timestamp = 1605072079649.610107 out_system_timestamp = 1605072079675.808105 component_latency = 26.197998
Comp name = nvv4l2decoder12 in_system_timestamp = 1605072079526.803955 out_system_timestamp = 1605072079652.954102 component latency= 126.150146
Comp name = src_bin_muxer source_id = 22 pad_index = 22 frame_num = 443 in_system_timestamp = 1605072079653.008057 out_system_timestamp = 1605072079675.808105 component_latency = 22.800049
Comp name = nvv4l2decoder17 in_system_timestamp = 1605072079566.533936 out_system_timestamp = 1605072079660.735107 component latency= 94.201172
Comp name = src_bin_muxer source_id = 25 pad_index = 25 frame_num = 444 in_system_timestamp = 1605072079660.788086 out_system_timestamp = 1605072079675.808105 component_latency = 15.020020
Comp name = nvv4l2decoder2 in_system_timestamp = 1605072079515.694092 out_system_timestamp = 1605072079640.035889 component latency= 124.341797
Comp name = src_bin_muxer source_id = 26 pad_index = 26 frame_num = 443 in_system_timestamp = 1605072079640.115967 out_system_timestamp = 1605072079675.808105 component_latency = 35.692139
Comp name = nvv4l2decoder24 in_system_timestamp = 1605072079521.849121 out_system_timestamp = 1605072079645.612061 component latency= 123.762939
Comp name = src_bin_muxer source_id = 27 pad_index = 27 frame_num = 444 in_system_timestamp = 1605072079645.639893 out_system_timestamp = 1605072079675.808105 component_latency = 30.168213
Comp name = nvv4l2decoder13 in_system_timestamp = 1605072079531.185059 out_system_timestamp = 1605072079664.819092 component latency= 133.634033
Comp name = src_bin_muxer source_id = 29 pad_index = 29 frame_num = 444 in_system_timestamp = 1605072079664.867920 out_system_timestamp = 1605072079675.808105 component_latency = 10.940186
Comp name = nvv4l2decoder27 in_system_timestamp = 1605072079583.988037 out_system_timestamp = 1605072079648.843018 component latency= 64.854980
Comp name = src_bin_muxer source_id = 4 pad_index = 4 frame_num = 444 in_system_timestamp = 1605072079648.884033 out_system_timestamp = 1605072079675.808105 component_latency = 26.924072
Comp name = nvv4l2decoder7 in_system_timestamp = 1605072079581.670898 out_system_timestamp = 1605072079647.083984 component latency= 65.413086
Comp name = src_bin_muxer source_id = 9 pad_index = 9 frame_num = 444 in_system_timestamp = 1605072079647.125000 out_system_timestamp = 1605072079675.809082 component_latency = 28.684082
Comp name = nvv4l2decoder23 in_system_timestamp = 1605072079581.208008 out_system_timestamp = 1605072079651.892090 component latency= 70.684082
Comp name = src_bin_muxer source_id = 17 pad_index = 17 frame_num = 445 in_system_timestamp = 1605072079651.930908 out_system_timestamp = 1605072079675.809082 component_latency = 23.878174
Comp name = nvv4l2decoder15 in_system_timestamp = 1605072079585.878906 out_system_timestamp = 1605072079653.677979 component latency= 67.799072
Comp name = src_bin_muxer source_id = 20 pad_index = 20 frame_num = 444 in_system_timestamp = 1605072079653.707031 out_system_timestamp = 1605072079675.809082 component_latency = 22.102051
Comp name = nvv4l2decoder12 in_system_timestamp = 1605072079594.454102 out_system_timestamp = 1605072079662.315918 component latency= 67.861816
Comp name = src_bin_muxer source_id = 22 pad_index = 22 frame_num = 444 in_system_timestamp = 1605072079662.447998 out_system_timestamp = 1605072079675.809082 component_latency = 13.361084
Comp name = nvv4l2decoder2 in_system_timestamp = 1605072079575.105957 out_system_timestamp = 1605072079656.961914 component latency= 81.855957
Comp name = src_bin_muxer source_id = 26 pad_index = 26 frame_num = 444 in_system_timestamp = 1605072079657.025879 out_system_timestamp = 1605072079675.809082 component_latency = 18.783203
Comp name = nvv4l2decoder24 in_system_timestamp = 1605072079583.729980 out_system_timestamp = 1605072079655.216064 component latency= 71.486084
Comp name = src_bin_muxer source_id = 27 pad_index = 27 frame_num = 445 in_system_timestamp = 1605072079655.295898 out_system_timestamp = 1605072079675.809082 component_latency = 20.513184
Comp name = nvv4l2decoder13 in_system_timestamp = 1605072079603.968018 out_system_timestamp = 1605072079666.916992 component latency= 62.948975
Comp name = src_bin_muxer source_id = 29 pad_index = 29 frame_num = 445 in_system_timestamp = 1605072079666.946045 out_system_timestamp = 1605072079675.809082 component_latency = 8.863037
Comp name = nvv4l2decoder8 in_system_timestamp = 1605072079537.941895 out_system_timestamp = 1605072079668.864990 component latency= 130.923096
Comp name = src_bin_muxer source_id = 10 pad_index = 10 frame_num = 444 in_system_timestamp = 1605072079668.974121 out_system_timestamp = 1605072079675.809082 component_latency = 6.834961
Comp name = nvv4l2decoder1 in_system_timestamp = 1605072079606.281982 out_system_timestamp = 1605072079669.206055 component latency= 62.924072
Comp name = src_bin_muxer source_id = 15 pad_index = 15 frame_num = 445 in_system_timestamp = 1605072079669.246094 out_system_timestamp = 1605072079675.809082 component_latency = 6.562988
Comp name = nvv4l2decoder25 in_system_timestamp = 1605072079536.737061 out_system_timestamp = 1605072079670.735107 component latency= 133.998047
Comp name = src_bin_muxer source_id = 18 pad_index = 18 frame_num = 444 in_system_timestamp = 1605072079670.820068 out_system_timestamp = 1605072079675.809082 component_latency = 4.989014
Comp name = nvv4l2decoder30 in_system_timestamp = 1605072079583.794922 out_system_timestamp = 1605072079671.798096 component latency= 88.003174
Comp name = src_bin_muxer source_id = 16 pad_index = 16 frame_num = 444 in_system_timestamp = 1605072079671.849121 out_system_timestamp = 1605072079675.809082 component_latency = 3.959961
Comp name = nvv4l2decoder25 in_system_timestamp = 1605072079609.876953 out_system_timestamp = 1605072079672.917969 component latency= 63.041016
Comp name = src_bin_muxer source_id = 18 pad_index = 18 frame_num = 445 in_system_timestamp = 1605072079672.946045 out_system_timestamp = 1605072079675.810059 component_latency = 2.864014
Comp name = nvv4l2decoder28 in_system_timestamp = 1605072079564.688965 out_system_timestamp = 1605072079674.497070 component latency= 109.808105
Comp name = src_bin_muxer source_id = 19 pad_index = 19 frame_num = 444 in_system_timestamp = 1605072079674.529053 out_system_timestamp = 1605072079675.810059 component_latency = 1.281006
Comp name = nvv4l2decoder8 in_system_timestamp = 1605072079608.271973 out_system_timestamp = 1605072079674.638916 component latency= 66.366943
Comp name = src_bin_muxer source_id = 10 pad_index = 10 frame_num = 445 in_system_timestamp = 1605072079674.687988 out_system_timestamp = 1605072079675.810059 component_latency = 1.122070
Comp name = nvv4l2decoder19 in_system_timestamp = 1605072079543.917969 out_system_timestamp = 1605072079675.674072 component latency= 131.756104
Comp name = src_bin_muxer source_id = 2 pad_index = 2 frame_num = 444 in_system_timestamp = 1605072079675.758057 out_system_timestamp = 1605072079675.810059 component_latency = 0.052002
Comp name = primary_gie in_system_timestamp = 1605072079675.894043 out_system_timestamp = 1605072079716.792969 component latency= 40.898926
Comp name = secondary_gie_0 in_system_timestamp = 1605072079716.834961 out_system_timestamp = 0.000000 config_infer_primary_yoloV3_tiny_.txt (3.4 KB) config_infer_resnet50_TONVIDIA.txt (2.4 KB) deepstream_test.txt (5.3 KB) component latency= -1605072079716.834961
Comp name = tiled_display_tiler in_system_timestamp = 1605072079720.131104 out_system_timestamp = 1605072079722.270020 component latency= 2.138916
Source id = 3 Frame_num = 444 Frame latency = 164.152100 (ms)
Source id = 4 Frame_num = 443 Frame latency = 215.430908 (ms)
Source id = 6 Frame_num = 444 Frame latency = 160.251953 (ms)
Source id = 7 Frame_num = 444 Frame latency = 181.043945 (ms)
Source id = 9 Frame_num = 443 Frame latency = 222.739014 (ms)
Source id = 12 Frame_num = 445 Frame latency = 169.642090 (ms)
Source id = 14 Frame_num = 444 Frame latency = 182.696045 (ms)
Source id = 15 Frame_num = 444 Frame latency = 191.122070 (ms)
Source id = 16 Frame_num = 443 Frame latency = 184.746094 (ms)
Source id = 17 Frame_num = 444 Frame latency = 207.596924 (ms)
Source id = 20 Frame_num = 443 Frame latency = 211.954102 (ms)
Source id = 22 Frame_num = 443 Frame latency = 196.800049 (ms)
Source id = 25 Frame_num = 444 Frame latency = 157.070068 (ms)
Source id = 26 Frame_num = 443 Frame latency = 207.909912 (ms)
Source id = 27 Frame_num = 444 Frame latency = 201.754883 (ms)
Source id = 29 Frame_num = 444 Frame latency = 192.418945 (ms)
Source id = 4 Frame_num = 444 Frame latency = 139.615967 (ms)
Source id = 9 Frame_num = 444 Frame latency = 141.933105 (ms)
Source id = 17 Frame_num = 445 Frame latency = 142.395996 (ms)
Source id = 20 Frame_num = 444 Frame latency = 137.725098 (ms)
Source id = 22 Frame_num = 444 Frame latency = 129.149902 (ms)
Source id = 26 Frame_num = 444 Frame latency = 148.498047 (ms)
Source id = 27 Frame_num = 445 Frame latency = 139.874023 (ms)
Source id = 29 Frame_num = 445 Frame latency = 119.635986 (ms)
Source id = 10 Frame_num = 444 Frame latency = 185.662109 (ms)
Source id = 15 Frame_num = 445 Frame latency = 117.322021 (ms)
Source id = 18 Frame_num = 444 Frame latency = 186.866943 (ms)
Source id = 16 Frame_num = 444 Frame latency = 139.809082 (ms)
Source id = 18 Frame_num = 445 Frame latency = 113.727051 (ms)
Source id = 19 Frame_num = 444 Frame latency = 158.915039 (ms)
Source id = 10 Frame_num = 445 Frame latency = 115.332031 (ms)
Source id = 2 Frame_num = 444 Frame latency = 179.686035 (ms)

I have uploaded the config files as well.

5

What does your “asynchronous mode” and “synchronous mode” mean?

6

It means classifier asynch mode is 1. Kindly look at above log and can you tell why this negative value occurs

7

In “asynchronous mode”, the output timestamp is got in another thread without thread mutual exclusion. Sometimes, batch is released in the main thread before output timestamp is got in another thread(so no timestamp is generated, the default value is 0). Please check nvinfer source codes.

8

Can you try the latest DS version?