Blurry and occasional pixelated videos when using multiple rtsp streams as input in my dockerized application

Shannon_Govekar · October 20, 2022, 5:52am

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.0.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

My application takes an input of 10 rtsp streams. I’ve noticed blurry images when my input consists of more than 3 stream occasionally (2/5 times). I’ve noticed a bunch of warnings when debugging, few with the decoder and some with the input udp-buffer-size . I’m using the uri-decode-bin element similar to example3 in DeepStream-test3 example. I’ve tested the validity of the input rtsp streams and there is no issue besides for few dropped frames which lower the frequency from 30 to around 25 hz. Any suggestions?

Decodebin child added: decodebin0
Decodebin child added: decodebin1
Decodebin child added: decodebin2
Decodebin child added: rtph264depay0
Decodebin child added: rtph264depay1
Decodebin child added: decodebin3
0:00:09.004104149 17394 0x7f060c01b4c0 WARN            uridecodebin gsturidecodebin.c:920:unknown_type_cb:<uri-decode-bin> warning: No decoder available for type 'application/x-rtp, media=(string)application, payload=(int)107, clock-rate=(int)90000, encoding-name=(string)VND.ONVIF.METADATA, decodertag=(string)"h3c-v3\ RTCP\=0", a-recvonly=(string)"", ssrc=(uint)134290772, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1'.
0:00:09.004281327 17394 0x7f060c01b4c0 WARN                 default descriptions.c:1233:gst_pb_utils_get_codec_description: No description available for media type: application/x-rtp
0:00:09.004366808 17394 0x7f060c01b4c0 WARN               decodebin gstdecodebin2.c:4678:gst_decode_bin_expose:<decodebin2> error: no suitable plugins found:
Missing decoder: application/x-rtp (application/x-rtp, media=(string)application, payload=(int)107, clock-rate=(int)90000, encoding-name=(string)VND.ONVIF.METADATA, decodertag=(string)"h3c-v3\ RTCP\=0", a-recvonly=(string)"", ssrc=(uint)134290772, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1)

Decodebin child added: h264parse0
Decodebin child added: h264parse1
Decodebin child added: capsfilter0
Decodebin child added: capsfilter1
0:00:09.006740201 17394 0x7f068802e1e0 WARN            uridecodebin gsturidecodebin.c:920:unknown_type_cb:<uri-decode-bin> warning: No decoder available for type 'application/x-rtp, media=(string)application, payload=(int)107, clock-rate=(int)90000, encoding-name=(string)VND.ONVIF.METADATA, decodertag=(string)"h3c-v3\ RTCP\=0", a-recvonly=(string)"", ssrc=(uint)33821588, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1'.
0:00:09.006856296 17394 0x7f068802e1e0 WARN                 default descriptions.c:1233:gst_pb_utils_get_codec_description: No description available for media type: application/x-rtp
0:00:09.006908915 17394 0x7f068802e1e0 WARN               decodebin gstdecodebin2.c:4678:gst_decode_bin_expose:<decodebin3> error: no suitable plugins found:
Missing decoder: application/x-rtp (application/x-rtp, media=(string)application, payload=(int)107, clock-rate=(int)90000, encoding-name=(string)VND.ONVIF.METADATA, decodertag=(string)"h3c-v3\ RTCP\=0", a-recvonly=(string)"", ssrc=(uint)33821588, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1)

Decodebin child added: nvv4l2decoder0
Setting drop frame interval
Decodebin child added: nvv4l2decoder1
Setting drop frame interval
Decodebin child added: decodebin4
0:00:09.013569139 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.013613355 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MJPG
0:00:09.013630405 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.013645825 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MJPG
0:00:09.013683906 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.013704202 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MPG4
0:00:09.013723525 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.013741267 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MPG4
0:00:09.013770969 17394 0x7f060001b460 WARN                    v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.013757408 17394 0x7f068002a580 WARN                    v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:<nvv4l2decoder1:sink> Unable to try format: Unknown error -1

yuweiw · October 20, 2022, 11:27am

It depends on your network and bandwidth environment. It drops some data before decoder plugin, so the quality of the picture is not very well.

Shannon_Govekar · October 20, 2022, 6:00pm

hmmm. Any thoughts on how to debug this on a remote system? Network throughput in my opinion seems fine. Any changes I can make to the Gstreamer pipeline?

I set the kernel receive max window size rmem_max to 52428800 on my server and updated the udp-buffer-size property to 2000000. Each stream is 1280 x 720 @30hz

What about this decoder plugin warning. Could this be the issue?

No decoder available for type 'application/x-rtp, media=(string)application, payload=(int)107, clock-rate=(int)90000, encoding-name=(string)VND.ONVIF.METADATA, decodertag=(string)"h3c-v3\ RTCP\=0", a-recvonly=(string)"", ssrc=(uint)134290772, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1'

yuweiw · October 21, 2022, 8:41am

The data transmission is basically implemented by gstreamer itself. You should find which plugin is used to transport data in your pipeline, like tcp,rtsp,etc… Then you can try to set some paras to improve the performance of the gstreamer plugins.

saransh661 · October 31, 2022, 7:38am

Can you please elaborate how and where exactly to tune the plugins to improve the performance?
Any specific resource you can refer to?

yuweiw · November 1, 2022, 1:50am

1.Don’t add any time consuming fuction in any of the probe function.
2.You can add the drop frame paras in the plugin which can drop frame.
3.Make sure the net is smooth.
4.Set the batch size para to your source number.

Shannon_Govekar · November 1, 2022, 1:58am

@yuweiw an interesting finding today. I changed the protocol to tcp (enum 4) instead of udp in my rtspsrc element and the blurry images seem to have disappeared. However, now the cameras keep reconnecting throwing a 500 internal server error saying that the last buffer timestamp exceeded the reconnect interval ( still looking into how to fix this, any idea? Not sure if the latency parameter helps this) . Regarding the rtp protocol. Could this be anything to do with udp packet size and DeepStream docker with multiple rtsp source? I seen a comment on one sample app which says which made me change the protocol/** Force the source (applicable only if RTSP) * to use TCP for RTP/RTCP channels. * forcing TCP to avoid problems with UDP port usage from within docker- * container. * The UDP RTCP channel when run within docker had issues receiving * RTCP Sender Reports from server */

This is what my current config file looks like

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri="test"
nvbuf-memory-type=0
latency=30000
rtsp-reconnect-interval-sec=10
select-rtp-protocol=4
num-sources=1
gpu-id=0
cudadec-memtype=0

in my rtspsrc element. I added the following

g_object_set (G_OBJECT (bin->src_elem), "location", config->uri, NULL);
g_object_set (G_OBJECT (bin->src_elem), "latency", config->latency, NULL);
g_object_set (G_OBJECT (bin->src_elem), "drop-on-latency", TRUE, NULL);
g_object_set (G_OBJECT (bin->src_elem), "tcp-timeout", 20000000, NULL);
g_object_set (G_OBJECT (bin->src_elem), "teardown-timeout", 20000000, NULL);
g_object_set (G_OBJECT (bin->src_elem), "do-rtsp-keep-alive", true, NULL);
g_object_set (G_OBJECT (bin->src_elem), "debug", true, NULL);

yuweiw · November 8, 2022, 12:24pm

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one.
Thanks

===》500 internal server error saying
It usually means your server’s error. You can check the stability of your server. Also the rtsp or tcp, ucp protocal is based on Gstreamer open source code. You can also pay attention to the ralated issues on Gstreamer forum.

system · December 6, 2022, 6:22am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.