Trying to understand why Streammux and Nvv4l2decoder has higher latency relative to other elements in my pipeline

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) dGPU
• DeepStream Version 7.1

I have a simple pipeline that looks like this

uridecodebin-1 -> \
uridecodebin-2 ->  |  -> streammux -> pgie1 -> sgie1 -> sgie2 -> nvosd -> sink 
uridecodebin-n -> /

the decoder picked up by uridecodebin is nvv4l2decoder.

I am attaching a file that has per element latency.

latency.txt (900.8 KB)

Latency of nvv4l2decoder is not consistent across batche numebrs.
I see that initially it is very low. But for the later batch numbers, it goes very high.

my question,
Is streammux latency fine in my case ?
is Nvv4l2decoder latency fine in my case ?

my MUXER_BATCH_TIMEOUT is 150
and streammux Batch Size is = number of sources.

2

dmon_logs.txt (110.8 KB)
nvidia-smi dmon logs, if that helps anyhow.

4

This is not accurate. The following is the content from the log you provided:

Comp name = nvv4l2decoder7 in_system_timestamp = 1752826039350.527100 out_system_timestamp = 1752826039456.232910               component latency= 105.705811

But I think this is normal. I think your rtsp stream contains b frames. When nvv4l2decoder decodes these b frames, it needs to wait for the reference i frames and p frames to be received and decoded. This may result in delayed data accuracy

Please refer to this FAQ.

If you have multiple RTSP cameras, nvstreammux will be affected by the slowest stream when forming a batch.

5

allright.

6

I was using video files, not RTSP. still got irregularities in nvv4l2decoder latency across batches

7

Whether it is a video file or rtsp, if there is a b frame in the video stream, this will happen. This is determined by the h264 decoding algorithm.

8

allright.