Measuring Frame and Element wise latency

Please provide complete information as applicable to your setup.

Hardware Platform (Jetson / GPU):-Xavier
DeepStream Version:-6.0

I was trying to obtain element wise latency from my pipeline and I got it too, however I’m not able to interpret it.

see some logs


************BATCH-NUM = 8**************
Comp name = nvv4l2-decoder in_system_timestamp = 1684240026498.968018 out_system_timestamp = 1684240026517.566895               component latency= 18.598877
Comp name = nvstreammux-stream-muxer source_id = 0 pad_index = 0 frame_num = 8               in_system_timestamp = 1684240026517.625000 out_system_timestamp = 1684240026634.242920               component_latency = 116.617920
Comp name = primary-nvinference-engine in_system_timestamp = 1684240026634.278076 out_system_timestamp = 1684240026652.403076               component latency= 18.125000
Comp name = tracker in_system_timestamp = 1684240026652.424072 out_system_timestamp = 1684240026892.195068               component latency= 239.770996
Comp name = nvdsanalytics in_system_timestamp = 1684240026936.927979 out_system_timestamp = 1684240026937.008057               component latency= 0.080078
Comp name = secondary-1-nvinference-engine in_system_timestamp = 1684240026937.097900 out_system_timestamp = 1684240026948.909912               component latency= 11.812012
Comp name = secondary-2-nvinference-engine in_system_timestamp = 1684240026948.949951 out_system_timestamp = 1684240026978.732910               component latency= 29.782959
Comp name = nvvideo-converter in_system_timestamp = 1684240026979.748047 out_system_timestamp = 1684240026989.657959               component latency= 9.909912
Comp name = nv-onscreendisplay in_system_timestamp = 1684240026990.814941 out_system_timestamp = 1684240027012.312012               component latency= 21.497070
Src id=0 Frame_no=8 Frame latency=1684240027012.419922 (ms)

Questions :-
(1) component_latency :- what is the unit used for this ?? (nanoseconds or what)
(2) Frame latency :- what is the unit used for this ??
(3) under what circumstances, streammux can can be this slow ? Is this expected to I need to optimize it even further ?

The unit for latency is millisecond, similiar topic discussed here: Fame latencty.

For the 3rd question, does it happen on each frame or only on a specific frame?

it happens for each frame.

Did you run our demo or your own app? Could you attach the demo if you run your own app?

demo meaning the same logs for your apps ?

No. There are no such abnormal values in our demo app. So I want to know what demo are you running.

its not a demo, its my LPR app

Is it our lpr demo app or your own app?
https://github.com/NVIDIA-AI-IOT/deepstream_lpr_app

It is my own app

Maybe it’s a problem in your code. You can try it with our demo code or attach your own app. If we can reproduce in our enviroment, we will analyze problems better.

okay will try that