This problem has nothing to do with deepstream.
It should only be caused by the excessive network delay of the rtsp camera.
Or increase the value of the tcp-timeout property of rtspsrc, but this may not solve the problem
gst-launch-1.0 rtspsrc location=xxx tcp-timeout=80000000