Latency with python test3

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Using deepstream5.1-devel docker on dGPU.

I am using deepstream-test3 from deepstream_python_apps/apps/deepstream-test3 at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub and I am experiencing increasing latency when runnning multiple (6 right now) video streams.
I have set sink.set_property("sync", 0) and live source is also set since they are RTSP streams.

My model with the current hardware should be able to perform close to 200 fps and deepstream is currently running on 6 cameras with 15fps video streams, which is far less for detection to be a bottleneck. Either way my app requires to process newest frames, even if it means dropping frames before. As far as I read sync=0 and live source should do that, but somehow I am still experiencing latency. For testing purposes I tried to change sync to 1, result was the same.

I tested this parallely running opencv VideoCapture and then visually comparing deepstream tiler output and the VideoCapture output. When the program is running for over 24h latency is over couple of seconds.

The deepstream-test3 is using uridecodebin and I am passing h265 streams, maybe I should try rtspsrc?

What could cause the latency, how could I debug this?

3

Can you check whether the following link can help you? DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums

5

Should these work in Python
export NVDS_ENABLE_COMPONENT_LATENCY_MEASUREMENT=1 export NVDS_ENABLE_LATENCY_MEASUREMENT=1 ?

I have tried to create Python bindings according to the repositories in that link, but I am getting multiple declaration, syntax errors when trying to include nvds_latency_meta.h:

In file included from pyds_latency_meta.cpp:3:0:
/opt/nvidia/deepstream/deepstream-5.1/app/includes/nvds_latency_meta.h:107:47: error: ‘GstBuffer’ was not declared in this scope
NvDsUserMeta *nvds_set_input_system_timestamp(GstBuffer * buffer,
^~~~~~~~~
/opt/nvidia/deepstream/deepstream-5.1/app/includes/nvds_latency_meta.h:107:47: note: suggested alternative: ‘setbuffer’
NvDsUserMeta *nvds_set_input_system_timestamp(GstBuffer * buffer,
^~~~~~~~~
setbuffer
/opt/nvidia/deepstream/deepstream-5.1/app/includes/nvds_latency_meta.h:107:59: error: ‘buffer’ was not declared in this scope
NvDsUserMeta *nvds_set_input_system_timestamp(GstBuffer * buffer,
…

and the error output goes on, meanwhile I can include nvds_analytics_meta.h (as in one the examples, for binding creation) successfully without any errors

6

No.

For python bindings, you can refer to DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums

7

Yes, I was referring to these examples in my earlier comment, I can’t even include nvds_latency_meta.h in my code (#include "nvds_latency_meta.h"), while there are no problems including nvds_analytics_meta.h or nvds_tracker_meta.h as in the examples.

8

Please resolve build problem with yourself. Such ”error: ‘xxxxxx’ was not declared in this scope“ means the dependent packages(libraries and headers) are not included. Please find with google.

9

I’ve succesfully created bindings, but I am having another issue.

When running program all of user_meta_lists are None (batch_meta.batch_user_meta_list, frame_meta.frame_user_meta_list, obj_meta.obj_user_meta_list).

After I export these variables:
export NVDS_ENABLE_COMPONENT_LATENCY_MEASUREMENT=1 export NVDS_ENABLE_LATENCY_MEASUREMENT=1

batch_meta.batch_user_meta_list is not None anymore, but others are, and I need frame_user_meta_list to access NvDsFrameLatencyInfo.

Am I missing something?

10

What do you mean by “but others are”?

11

I meant other are None (frame and obj user_metas). However now I am able to extract frame latency from batch_user_meta_list but the values are gibberish:

source_id=880178798 frame_num=1701065324 comp_in_timestamp=2.79475250126614e-310 latency=0.0 
source_id=1701999699 frame_num=1831693665 comp_in_timestamp=9.48236774e-315 latency=0.0 
source_id=880178798 frame_num=1701065324 comp_in_timestamp=2.6861063167687e-310 latency=0.0 
source_id=1701999699 frame_num=1831693665 comp_in_timestamp=9.48236774e-315 latency=0.0 
source_id=880178798 frame_num=1701065324 comp_in_timestamp=2.63178322452e-310 latency=0.0 
source_id=1701999699 frame_num=1831693665 comp_in_timestamp=9.48236774e-315 latency=0.0 
source_id=880178798 frame_num=1701065324 comp_in_timestamp=2.74042940901743e-310 latency=0.0 
source_id=1701999699 frame_num=1831693665 comp_in_timestamp=9.48236774e-315 latency=0.0 
source_id=1835627120 frame_num=762933857 comp_in_timestamp=9.177152518567484e+170 latency=5e-322
12

Seems this interface dose not work with python.

13

Any tips how could I analyze my latency problem, or what else might help?

14

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks
Can you try c sample deepstream-test3-app to test the same case first?