Nvstreammux frame drops and jitter (24.9 fps) when enabling ntp-sync in rtspsrc

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 8.0
• TensorRT Version 10.9.0.34
• NVIDIA GPU Driver Version 575
• I am experiencing a slight but critical frame rate drop and “buffer dropped” warnings from old nvstreammux when enabling ntp-sync on rtspsrc.

  • The Issue:

  1. To achieve better multi-stream synchronization in nvstreammux, I enabled the ntp-sync=true property in rtspsrc.

  2. Without ntp-sync, the stream runs at a stable 25 FPS.

  3. With ntp-sync enabled, the frame rate fluctuates around 24.9 FPS.

  4. nvstreammux constantly outputs warnings about dropped frames.

  5.   batch-size: 36
  batched-push-timeout: 40000
  width: 3840
  height: 2160
  live-source: 1
  sync-inputs: 1
  max-latency: 3000000000
  attach-sys-ts: 0

  6. nvstreammux dropped frame printing

64:23:02.963293060 2036052 0x7ff0e802d6b0 WARN nvstreammux gstnvstreammux.cpp:2839:gst_nvstreammux_src_collect_buffers:<streammux_0> dropping late buffer pad id = 2 window_start=64:23:02.320000000src_buffer_pts=64:23:02.320234915 

64:23:03.003916246 2036052 0x7ff0e802d6b0 WARN nvstreammux gstnvstreammux.cpp:2839:gst_nvstreammux_src_collect_buffers:<streammux_0> dropping late buffer pad id = 8 window_start=64:23:02.360000000src_buffer_pts=64:23:02.361235433
  • Questions:

  1. set max-latency=3000000000 (3 seconds) in nvstreammux, but the issue persists

  2. Are there any specific properties in nvstreammux (new or old version) to relax the synchronization threshold to prevent these drops while maintaining NTP alignment?

6

Please refer to this faq for hwo to set nvstreammux parameters.

  1. How many sources are you testing? batch-size should be set to the number of sources. batched-push-timeout to the (1000000 us/maximum fps among the videos)
  2. do you mean there is no “dropping late buffer” issue if ntp-sync is disabled?
  3. ’ sync-inputs =1’ means sychronization of input frames using PTS. the late buffers will be dropped because the buffer is too late.
7

We are testing 36 sources

My understanding of this warning is: the current time is 64:23:03.003916246, and the current buffer’s PTS is 64:23:02.361235433. They are only about 0.6 seconds apart, so why isn’t my `max-latency` setting working? Perhaps my understanding is incorrect, but I’d like to know what parameters I can set to tolerate this 0.6-second difference.

8

‘max-latency’ means “Additional latency in live mode to allow upstream to take longer to produce buffers for the current position (in nanoseconds)”. the function is similar to fakesink’s ‘render-delay’. add a delay to process buffer. ‘max-latency’ is unrelated to ‘dropping late buffer’.
the reason of “dropping late buffer” is that buffer’s PTS is too late compared with current time. do you mean there is no “dropping late buffer” issue if ntp-sync is disabled? did you try ‘sync-inputs: 0’?