Decode MJPEG based RTSP stream (IDS GigE camera RTSP streamer)

Hi,
I am trying to run inferencing on IDS GigE RTSP streamer which is provinding MJPEG encoded rtsp stream. I am using deepstream-test3 app. Here is the output.

Now playing: rtsp://192.168.1.200:8554/mystream,

Using winsys: x11
Creating LL OSD context new
Decodebin child added: source
Running…
ERROR from element source: Could not open resource for reading and writing.
Error details: gstrtspsrc.c(7469): gst_rtspsrc_retrieve_sdp (): /GstPipeline:dstest3-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin/GstRTSPSrc:source:
Failed to connect. (Generic error)
Returned, stopping playback
Deleting pipeline

Hi

Can you try with this pipeline?

gst-launch-1.0 uridecodebin uri=rtsp://192.168.1.200:8554/mystream ! queue ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM),format=NV12,width=640,height=480' !  queue ! \
nvstreammux0.sink_0 \
nvstreammux name=nvstreammux0 batch-size=1 batched-push-timeout=40000 width=640 height=480 live-source=TRUE !  queue ! \
nvvideoconvert !  queue ! \
nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/deepstream-test3/dstest3_pgie_config.txt !  queue ! \
nvdsosd process-mode=HW_MODE !  queue ! \
nvvideoconvert ! 'video/x-raw(memory:NVMM)' ! nvoverlaysink

Make sure that the config-file-path is ok. Let me know if you get any errors.

Hi Miguel,
Thanks for the reply. I tried executing the pipeline, but I am getting the following error.

===============================================
Setting pipeline to PAUSED …
Creating LL OSD context new
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.1.200:8554/mystream
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Creating LL OSD context new
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source/GstUDPSrc:udpsrc1: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source/GstUDPSrc:udpsrc1:
streaming stopped, reason error (-5)
Execution ended after 0:00:00.272164512
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Can you run it again adding GST_DEBUG=uridecodebin*:5 before gst-launch? To check if the uridecodebin logs a more helpful error message.

This is the debug log that i got for “uridecodebin:5”

Setting pipeline to PAUSED …
Creating LL OSD context new
0:00:03.219101088 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:2811:gst_uri_decode_bin_change_state: ready to paused
0:00:03.219210592 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:2184:setup_source: setup source
0:00:03.225772800 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:1362:gen_source_element: setting connection-speed=0 to source element
0:00:03.225863744 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:2238:setup_source: Source has dynamic output pads
0:00:03.226111392 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:899:do_async_done: posting ASYNC_DONE
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.1.200:8554/mystream
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:03.403147456 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:2110:source_new_pad: Found new pad source.recv_rtp_src_0_634769070_26 in source element source
0:00:03.403324320 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1444:has_all_raw_caps:source:recv_rtp_src_0_634769070_26 have caps application/x-rtp, media=(string)video, payload=(int)26, clock-rate=(int)90000, a-tool=(string)“LIVE555\ Streaming\ Media\ v2016.04.21”, a-type=(string)broadcast, x-qt-text-nam=(string)descr, x-qt-text-inf=(string)info, clock-base=(uint)404615500, seqnum-base=(uint)28217, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1, ssrc=(uint)634769070
0:00:03.404057728 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:2135:source_new_pad: linked decoder to new pad
0:00:03.404287840 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1013:source_no_more_pads: No more pads in source element source.
0:00:03.404361792 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:946:no_more_pads_full: no more pads, 2 pending
0:00:03.404697088 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1695:proxy_autoplug_continue_signal: autoplug-continue returned 1
0:00:03.404784768 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:336:gst_uri_decode_bin_autoplug_factories: finding factories
0:00:03.410028000 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:358:gst_uri_decode_bin_autoplug_factories: autoplug-factories returns 0x7ef405b0c0
0:00:03.410090784 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1710:proxy_autoplug_factories_signal: autoplug-factories returned 0x7ef405b120
0:00:03.410130624 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1725:proxy_autoplug_sort_signal: autoplug-sort returned (nil)
0:00:03.410269952 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:374:gst_uri_decode_bin_autoplug_select: default autoplug-select returns TRY
0:00:03.410291520 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1740:proxy_autoplug_select_signal: autoplug-select returned 0
0:00:03.410796288 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1695:proxy_autoplug_continue_signal: autoplug-continue returned 1
0:00:03.411200544 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1695:proxy_autoplug_continue_signal: autoplug-continue returned 1
0:00:03.411235712 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:336:gst_uri_decode_bin_autoplug_factories: finding factories
0:00:03.411464448 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:358:gst_uri_decode_bin_autoplug_factories: autoplug-factories returns 0x7ef40605a0
0:00:03.411486752 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1710:proxy_autoplug_factories_signal: autoplug-factories returned 0x7ef4060500
0:00:03.411513952 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1725:proxy_autoplug_sort_signal: autoplug-sort returned (nil)
0:00:03.411595872 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:374:gst_uri_decode_bin_autoplug_select: default autoplug-select returns TRY
0:00:03.411614464 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1740:proxy_autoplug_select_signal: autoplug-select returned 0
0:00:03.413621824 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1755:proxy_autoplug_query_signal: autoplug-query returned 0
0:00:03.413795904 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1755:proxy_autoplug_query_signal: autoplug-query returned 0
0:00:03.416740608 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1755:proxy_autoplug_query_signal: autoplug-query returned 0
0:00:03.417082112 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1140:new_decoded_pad_added_cb: new decoded pad, name: <src_0>
0:00:03.417334240 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1124:copy_sticky_events:<’’:src_0> store sticky event stream-start event: 0x7ef40051d0, time 99:99:99.999999999, seq-num 255, GstEventStreamStart, stream-id=(string)b9049c323800fa1dbf0c9c2f5d6dcf0e63b50fc2c5030d1c14e44a893d14e333/video:0:0:RTP:AVP:26, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE;
0:00:03.417407936 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:1124:copy_sticky_events:<’’:src_0> store sticky event caps event: 0x7ef4066ca0, time 99:99:99.999999999, seq-num 297, GstEventCaps, caps=(GstCaps)“video/x-raw(memory:NVMM),\ format=(string)I420,\ width=(int)1280,\ height=(int)720,\ interlace-mode=(string)progressive,\ multiview-mode=(string)mono,\ multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,\ pixel-aspect-ratio=(fraction)1/1,\ chroma-site=(string)mpeg2,\ colorimetry=(string)1:4:0:0,\ framerate=(fraction)0/1”;
0:00:03.418428992 15715 0x7f000031e0 DEBUG uridecodebin gsturidecodebin.c:946:no_more_pads_full: no more pads, 1 pending
Creating LL OSD context new
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source/GstUDPSrc:udpsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source/GstUDPSrc:udpsrc0:
streaming stopped, reason error (-5)
Execution ended after 0:00:00.272834336
Setting pipeline to PAUSED …
Setting pipeline to READY …
0:00:03.543058528 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:1206:pad_removed_cb: pad removed name: decodebin0:src_0
0:00:03.543325600 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:2837:gst_uri_decode_bin_change_state: paused to ready
0:00:03.543356192 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:1632:remove_decoders: removing old decoder element
0:00:03.543507392 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:2068:remove_source: removing old src element
Setting pipeline to NULL …
0:00:03.547070816 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:2846:gst_uri_decode_bin_change_state: ready to null
0:00:03.547488928 15715 0x7f2c0022c0 DEBUG uridecodebin gsturidecodebin.c:1206:pad_removed_cb: pad removed name: decodebin0:sink
Freeing pipeline …

Just as a long shot can you try replacing uridecodebin with uridecodebin3. I am clueless about why uridecodebin is failing with your stream. It should be able to handle mjpeg.

You can also try starting the pipeline with fakesink it should produce the same error:

GST_DEBUG=uridecodebin*:5 gst-launch-1.0 uridecodebin uri=rtsp://192.168.1.200:8554/mystream ! fakesink

Hi,
It is very likely the URI is invalid. Please try this:

$ gst-launch-1.0 uridecodebin uri=rtsp://freja.hiof.no:1935/rtplive/_definst_/hessdalen02.stream ! nvoverlaysink

If above pipeline can be run successfully, please replace with your URI and try.

This is the output when using fakesink

Setting pipeline to PAUSED …
0:00:00.044527392 9732 0x5560396ed0 DEBUG uridecodebin gsturidecodebin.c:2811:gst_uri_decode_bin_change_state: ready to paused
0:00:00.044614048 9732 0x5560396ed0 DEBUG uridecodebin gsturidecodebin.c:2184:setup_source: setup source
0:00:00.050215232 9732 0x5560396ed0 DEBUG uridecodebin gsturidecodebin.c:1362:gen_source_element: setting connection-speed=0 to source element
0:00:00.050313920 9732 0x5560396ed0 DEBUG uridecodebin gsturidecodebin.c:2238:setup_source: Source has dynamic output pads
0:00:00.050570528 9732 0x5560396ed0 DEBUG uridecodebin gsturidecodebin.c:899:do_async_done: posting ASYNC_DONE
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.1.200:8554/mystream
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:00.089595392 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:2110:source_new_pad: Found new pad source.recv_rtp_src_0_813079940_26 in source element source
0:00:00.089722144 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1444:has_all_raw_caps:source:recv_rtp_src_0_813079940_26 have caps application/x-rtp, media=(string)video, payload=(int)26, clock-rate=(int)90000, a-tool=(string)“LIVE555\ Streaming\ Media\ v2016.04.21”, a-type=(string)broadcast, x-qt-text-nam=(string)descr, x-qt-text-inf=(string)info, clock-base=(uint)4098655545, seqnum-base=(uint)57997, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1, ssrc=(uint)813079940
0:00:00.090616192 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:2135:source_new_pad: linked decoder to new pad
0:00:00.090877408 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1013:source_no_more_pads: No more pads in source element source.
0:00:00.090911392 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:946:no_more_pads_full: no more pads, 2 pending
0:00:00.091312512 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1695:proxy_autoplug_continue_signal: autoplug-continue returned 1
0:00:00.091362656 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:336:gst_uri_decode_bin_autoplug_factories: finding factories
0:00:00.097699008 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:358:gst_uri_decode_bin_autoplug_factories: autoplug-factories returns 0x7f4c058c60
0:00:00.097749664 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1710:proxy_autoplug_factories_signal: autoplug-factories returned 0x7f4c058cc0
0:00:00.097823232 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1725:proxy_autoplug_sort_signal: autoplug-sort returned (nil)
0:00:00.097980640 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:374:gst_uri_decode_bin_autoplug_select: default autoplug-select returns TRY
0:00:00.098047136 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1740:proxy_autoplug_select_signal: autoplug-select returned 0
0:00:00.098606144 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1695:proxy_autoplug_continue_signal: autoplug-continue returned 1
0:00:00.098991392 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1695:proxy_autoplug_continue_signal: autoplug-continue returned 1
0:00:00.099029984 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:336:gst_uri_decode_bin_autoplug_factories: finding factories
0:00:00.099411840 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:358:gst_uri_decode_bin_autoplug_factories: autoplug-factories returns 0x7f4c05db40
0:00:00.099441600 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1710:proxy_autoplug_factories_signal: autoplug-factories returned 0x7f4c05daa0
0:00:00.099479968 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1725:proxy_autoplug_sort_signal: autoplug-sort returned (nil)
0:00:00.099579104 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:374:gst_uri_decode_bin_autoplug_select: default autoplug-select returns TRY
0:00:00.099601760 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1740:proxy_autoplug_select_signal: autoplug-select returned 0
0:00:00.141704256 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1755:proxy_autoplug_query_signal: autoplug-query returned 0
0:00:00.142041248 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1755:proxy_autoplug_query_signal: autoplug-query returned 0
0:00:00.144843264 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1755:proxy_autoplug_query_signal: autoplug-query returned 0
0:00:00.145170400 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1140:new_decoded_pad_added_cb: new decoded pad, name: <src_0>
0:00:00.145484800 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1124:copy_sticky_events:<’’:src_0> store sticky event stream-start event: 0x7f4c005ac0, time 99:99:99.999999999, seq-num 157, GstEventStreamStart, stream-id=(string)b9049c323800fa1dbf0c9c2f5d6dcf0e63b50fc2c5030d1c14e44a893d14e333/video:0:0:RTP:AVP:26, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE;
0:00:00.145573344 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:1124:copy_sticky_events:<’’:src_0> store sticky event caps event: 0x7f4c099980, time 99:99:99.999999999, seq-num 199, GstEventCaps, caps=(GstCaps)“video/x-raw(memory:NVMM),\ format=(string)I420,\ width=(int)1280,\ height=(int)720,\ interlace-mode=(string)progressive,\ multiview-mode=(string)mono,\ multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,\ pixel-aspect-ratio=(fraction)1/1,\ chroma-site=(string)mpeg2,\ colorimetry=(string)1:4:0:0,\ framerate=(fraction)0/1”;
0:00:00.145746784 9732 0x7f64003f70 DEBUG uridecodebin gsturidecodebin.c:946:no_more_pads_full: no more pads, 1 pending

Hi,
It is not a URL issue. I am able to play the live stream in vlc player.

This is not a DeepStream problem because the pipeline without DS is failing in the same way.

I think it is a packetizing issue. It is likely that the camera is sending data that uridecodebin can’t handle. But this is very difficult to debug without analyzing the actual RTSP packages, and the solution may require modifying a GStreamer element.