I have been trying to make this pipeline work for a couple of days already and hit a wall…
I am trying to make a “server” that takes a rtsp stream and then re-streams it with gst/rtsp-server
The pipeline looks something like this:
rtspsrc location="rtsp://192.168.0.66"
! rtph264depay ! h264parse ! omxh264dec
! m.sink_0 nvstreammux name=m batch-size=1 live-source=true width=2688 height=1520
! nvinfer config-file-path=nvinfer.txt
! nvvidconv ! video/x-raw(memory:NVMM),format=RGBA ! nvdsosd ! nvvidconv ! video/x-raw(memory:NVMM),format=I420
! omxh264enc ! h264parse ! rtph264pay name=pay0 pt=96
I found that nvstreammux is required for nvinfer (Deepstream SDK 3.0)(please correct me if I am wrong). Now it looks like it complains about the number of surfaces. I have been looking through the examples and they do not setup anything else, what am I missing?
Removing lines 3 and 4 works and I am able to re-stream the video.
Here is the output:
0:00:05.834879477 17382 0x55b8d10d90 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/esteban/.config:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Creating LL OSD context new
0:00:10.215608720 17382 0x7ef0006770 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:10.215866017 17382 0x7f003ce190 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:10.320470770 17382 0x7ef00065e0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc2> error: Internal data stream error.
0:00:10.320880622 17382 0x7ef00065e0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc2> error: streaming stopped, reason not-linked (-1)
0:00:12.309996946 17382 0x7ec80034a0 FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec0> Sub-class should implement drain()
0:00:12.325193828 17382 0x7ec80034a0 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:<h264parse0:src> could not send sticky events
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.388: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_structure_get_string: assertion 'structure != NULL' failed
0:00:12.336783447 17382 0x7ec80034a0 ERROR omxvideodec gstomxvideodec.c:3274:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec0> Invalid caps: (NULL)
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:12.342775552 17382 0x7ec80034a0 FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec0> Sub-class should implement drain()
Allocating new output: 2688x1520 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 2688, nFrameHeight = 1520
Creating LL OSD context new
0:00:12.386941012 17382 0x7ef0006630 FIXME videoencoder gstvideoencoder.c:661:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> GstVideoEncoder::reset() is deprecated
Framerate set to : 20 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:12.388795010 17382 0x7ef0006630 WARN omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
H264: Profile = 66, Level = 40
0:00:12.500529652 17382 0x7ef0006630 WARN nvstreammux gstnvstreammux.c:308:gst_nvstreammux_chain:<m> error: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
0:00:12.500660488 17382 0x7ef0006630 WARN omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: Internal data stream error.
0:00:12.500689916 17382 0x7ef0006630 WARN omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: stream stopped, reason error
0:00:12.500763355 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
(/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(308): gst_nvstreammux_chain (): /GstPipeline:media-pipeline/GstBin:bin0/GstNvStreamMux:m)
0:00:12.501152061 17382 0x55b8d10d90 WARN rtspmedia rtsp-media.c:2994:wait_preroll: failed to preroll pipeline
0:00:12.501177686 17382 0x55b8d10d90 WARN rtspmedia rtsp-media.c:3298:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:12.502088747 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Internal data stream error. (/dvs/git/dirty/git-master_linux/3rdparty/gst/gst-omx/omx/gstomxvideodec.c(3005): gst_omx_video_dec_loop (): /GstPipeline:media-pipeline/GstBin:bin0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason error)
0:00:12.536356603 17382 0x7f3c005590 WARN rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc0> error: Unhandled error
0:00:12.536400250 17382 0x7f3c005590 WARN rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc0> error: Method Not Valid in This State (455)
0:00:12.536513117 17382 0x7f3c005590 WARN rtspsrc gstrtspsrc.c:8244:gst_rtspsrc_pause:<rtspsrc0> error: Could not send message. (Generic error)
0:00:12.536587025 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Unhandled error (gstrtspsrc.c(6161): gst_rtspsrc_send (): /GstPipeline:media-pipeline/GstBin:bin0/GstRTSPSrc:rtspsrc0:
Method Not Valid in This State (455))
0:00:12.536679683 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Could not write to resource. (gstrtspsrc.c(8244): gst_rtspsrc_pause (): /GstPipeline:media-pipeline/GstBin:bin0/GstRTSPSrc:rtspsrc0:
Could not send message. (Generic error))
0:00:17.547756971 17382 0x55b8d10d90 ERROR omx gstomx.c:2064:gst_omx_port_wait_buffers_released_unlocked:<omxh264dec-omxh264dec0> Timeout waiting for decode port 1 to release all buffers
0:00:17.578449960 17382 0x7f3c005590 WARN rtspsrc gstrtspsrc.c:5999:gst_rtspsrc_try_send:<rtspsrc0> send interrupted
0:00:17.578495951 17382 0x7f3c005590 WARN rtspsrc gstrtspsrc.c:7673:gst_rtspsrc_close:<rtspsrc0> TEARDOWN interrupted
0:00:17.581941752 17382 0x55b8d10d90 ERROR rtspclient rtsp-client.c:1054:find_media: client 0x55b8dba0f0: can't prepare media
0:00:17.582766613 17382 0x55b8d10d90 ERROR rtspclient rtsp-client.c:2910:handle_describe_request: client 0x55b8dba0f0: no media
Creating LL OSD context new
0:00:18.103361627 17382 0x7ec80034a0 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc2:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:18.104535922 17382 0x7ea4003400 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc3:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:18.190139181 17382 0x7eb4004b20 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc6> error: Internal data stream error.
0:00:18.190660026 17382 0x7eb4004b20 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc6> error: streaming stopped, reason not-linked (-1)
0:00:20.320330344 17382 0x7ed4002a30 FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec1> Sub-class should implement drain()
0:00:20.330312325 17382 0x7ed4002a30 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:<h264parse2:src> could not send sticky events
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_structure_get_string: assertion 'structure != NULL' failed
0:00:20.339492570 17382 0x7ed4002a30 ERROR omxvideodec gstomxvideodec.c:3274:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec1> Invalid caps: (NULL)
(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.392: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:20.346838766 17382 0x7ed4002a30 FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec1> Sub-class should implement drain()
Allocating new output: 2688x1520 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 2688, nFrameHeight = 1520
Creating LL OSD context new
0:00:20.387993121 17382 0x7ee0009e30 FIXME videoencoder gstvideoencoder.c:661:gst_video_encoder_setcaps:<omxh264enc-omxh264enc1> GstVideoEncoder::reset() is deprecated
Framerate set to : 20 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:20.390094988 17382 0x7ee0009e30 WARN omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc1> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
H264: Profile = 66, Level = 40
0:00:20.505743775 17382 0x7ee0009e30 WARN nvstreammux gstnvstreammux.c:308:gst_nvstreammux_chain:<m> error: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
0:00:20.506054198 17382 0x7ee0009e30 WARN omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec1> error: Internal data stream error.
0:00:20.506110085 17382 0x7ee0009e30 WARN omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec1> error: stream stopped, reason error
0:00:20.506118523 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
(/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(308): gst_nvstreammux_chain (): /GstPipeline:media-pipeline/GstBin:bin1/GstNvStreamMux:m)
0:00:20.506294047 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Internal data stream error. (/dvs/git/dirty/git-master_linux/3rdparty/gst/gst-omx/omx/gstomxvideodec.c(3005): gst_omx_video_dec_loop (): /GstPipeline:media-pipeline/GstBin:bin1/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec1:
stream stopped, reason error)
0:00:20.506337954 17382 0x55b8d10d90 WARN rtspmedia rtsp-media.c:2994:wait_preroll: failed to preroll pipeline
0:00:20.506455821 17382 0x55b8d10d90 WARN rtspmedia rtsp-media.c:3298:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:20.530217415 17382 0x7eb40048a0 WARN rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc1> error: Unhandled error
0:00:20.530264135 17382 0x7eb40048a0 WARN rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc1> error: Method Not Valid in This State (455)
0:00:20.530398773 17382 0x7eb40048a0 WARN rtspsrc gstrtspsrc.c:8244:gst_rtspsrc_pause:<rtspsrc1> error: Could not send message. (Generic error)
0:00:20.530422055 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Unhandled error (gstrtspsrc.c(6161): gst_rtspsrc_send (): /GstPipeline:media-pipeline/GstBin:bin1/GstRTSPSrc:rtspsrc1:
Method Not Valid in This State (455))
0:00:20.530523099 17382 0x7f7c44c590 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Could not write to resource. (gstrtspsrc.c(8244): gst_rtspsrc_pause (): /GstPipeline:media-pipeline/GstBin:bin1/GstRTSPSrc:rtspsrc1:
Could not send message. (Generic error))
0:00:25.563111767 17382 0x55b8d10d90 ERROR omx gstomx.c:2064:gst_omx_port_wait_buffers_released_unlocked:<omxh264dec-omxh264dec1> Timeout waiting for decode port 1 to release all buffers
0:00:25.592255609 17382 0x7eb40048a0 WARN rtspsrc gstrtspsrc.c:5999:gst_rtspsrc_try_send:<rtspsrc1> send interrupted
0:00:25.592290349 17382 0x7eb40048a0 WARN rtspsrc gstrtspsrc.c:7673:gst_rtspsrc_close:<rtspsrc1> TEARDOWN interrupted
0:00:25.596398352 17382 0x55b8d10d90 ERROR rtspclient rtsp-client.c:1054:find_media: client 0x55b8dba200: can't prepare media
0:00:25.597519053 17382 0x55b8d10d90 ERROR rtspclient rtsp-client.c:2639:handle_setup_request: client 0x55b8dba200: media '/test' not found
Thanks in advance,
Esteban