Deepstream RTSP streams fail over long run periods

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, , ,
1

Please provide complete information as applicable to your setup.

**• Hardware Platform GPU - 2 x A2
**• DeepStream Version 6.3
**• TensorRT Version 8.5.3.1
**• NVIDIA GPU Driver Version - 535.129.03
**• Issue Type - bug

I’m trying to debug an issue that I am struggling to find the route cause of.

I have a deepstream 6.3 pipeline (created with Python bindings) that roughly goes:

nvurisrcbin -> muxer -> pgie -> tracker -> analytics -> nvstreamdemux -> nvvideoconvert -> nvdsosd -> nvvideoconvert -> h264parse -> rtph264pay -> udpsink

I am then using GstRtspServer.RTSPServer to serve RTSP streams. My pipeline takes 7 input RTSP streams from live cameras on my local network and serves 7 annotated RTSP streams via a single GstRtspServer.

This works fine for a period of time (around 30 mins), after which the pipeline seems to falter, the streams die and the console output shows periodic “Resetting source” messages from the rtsp-reconnect-interval configured in nvurisrcbin.

The load on the GPU is a steady 60% or so, which drops to around 40% after it gets into this state.

Can anyone think of a good way to debug this? I have a feeling that it is something to do with time synchronisation between the streams in my pipeline. Are there elements or configuration changes that I can make to try to improve the robustness? I will also try upgrading to 6.4 I think.

3

please refer to this topic . you can modify batched-push-timeout to improve. please refer to this FAQ to improve. especially please set “export NVSTREAMMUX_ADAPTIVE_BATCHING=yes” .

4

Many thanks, I have modified the batched-push-timeout to accommodate our frame rate and set the environment variable as suggested. The streams seem to be holding fine now. Do you have documentation on that environment variable and will it take effect in DS 6.3 or will I need to upgrade our pipeline to 6.4? Cheers.

5

sorry for the late reply. it takes effects in DS6.3 and the latter version. please refer to the faq mentioned above. you don’ t need to upgrade to DS6.4 if only testing RTSP reconnectiong.