How to resolve latency issues when dynamically adding/removing RTSP streams in nvstreammux

[Fiona.Chen]

@polysong

My batched-push-timeout is set to 33 ms, meanings if a full batch cannot be formed, the muxer must wait 33 ms before pushing downstream – this is the source of the latency.

The dummy video FPS is 1. Have you follow DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums to set a proper “batched-push-timeout”?

If the input latency is the highest priority in your case, you may need to set nvstreammux batch size to 1 and process every stream in a separated pipeline to make every stream frames be batched ASAP. The side effect may be more loading to downstream inferencing processing.

@polysong

Should I dynamically update the batch-size of nvstreammux to match the actual number of active streams in the pipeline at runtime? nvstreammux is the core processing to generate batch from multiple streams, if the input streams are not synchronized, the latency is necessary for nvstreammux to collect as more frames as possible to generate the batch.

State change is needed to change the “batch-size” of “nvstreammux”.

@polysong

Or does nvstreammux provide a way to adaptively adjust batch-size automatically?

“batched-push-timeout” has already provided the way to adjust actual batch size of nvstreammux output