I needed to add a time overlay to an rtmp stream, and save to disk. The following pipeline, without the overlay, works fine:
gst-launch-1.0 -v \
rtmpsrc location=rtmp://192.168.x.x/live/0 do-timestamp=true ! queue2 ! flvdemux name=demux \
flvmux name=mux \
demux.video ! queue ! decodebin \
! nvvidconv \
! 'video/x-raw(memory:NVMM),width=1920,height=1080, format=(string)I420, framerate=50/1' \
! nvv4l2h264enc ! h264parse \
! mux.video \
demux.audio ! queue name="dmx_aud_q" ! mux.audio \
mux.src ! queue name="mux_q" ! filesink location="rtmp.flv"
Once I add the timeoverlay (or even clockoverlay), the pipeline doesn’t run:
gst-launch-1.0 -v \
rtmpsrc location=rtmp://192.168.0.168/x.x do-timestamp=true ! queue2 ! flvdemux name=demux \
flvmux name=mux \
demux.video ! queue ! decodebin \
! timeoverlay \
! nvvidconv \
! 'video/x-raw(memory:NVMM),width=1920,height=1080, format=(string)I420, framerate=50/1' \
! nvv4l2h264enc ! h264parse \
! mux.video \
demux.audio ! queue name="dmx_aud_q" ! mux.audio \
mux.src ! queue name="mux_q" ! filesink location="rtmp.flv"
The GST_DEBUG=3 logs are as added at the end. The Pipeline graph is as below.
From what I understand, at the point where the decodebin hands over to timeoverlay, there is some issue with caps negotiation. I’m not able to figure out how to make timeoverlay accept or output data in a way that the pipeline can continue to mux. Any help to understand what is happening here, and how to find a solution, would be great.
This is on an Nvidia Jetson Nano board, running Ubuntu 18.0. (The plugins starting with “nv” can usually be replaced with regular ones on other systems, I believe - nvvidconv with videoconvert, nvv4l2h264enc with omxh264enc etc.
Setting pipeline to PAUSED …
Opening in BLOCKING MODE 0:00:00.119491546 834 0x55b9d05600 WARN
v4l2
gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x55b9cf2360
Failed to determine interlace mode 0:00:00.119570298 834
0x55b9d05600 WARN v4l2
gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x55b9cf2360
Failed to determine interlace mode 0:00:00.119623164 834
0x55b9d05600 WARN v4l2
gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x55b9cf2360
Failed to determine interlace mode 0:00:00.119721552 834
0x55b9d05600 WARN v4l2
gstv4l2object.c:4408:gst_v4l2_object_probe_caps:nvv4l2h264enc0:src
Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error
-1 Pipeline is PREROLLING … 0:00:00.406155973 834 0x55b9a72370 WARN flvdemux
gstflvdemux.c:659:gst_flv_demux_parse_tag_script: failed
reading a tag, skipping
/GstPipeline:pipeline0/GstQueue:dmx_aud_q.GstPad:sink: caps =
audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, rate=(int)44100, channels=(int)2,
codec_data=(buffer)1210
/GstPipeline:pipeline0/GstQueue:dmx_aud_q.GstPad:src: caps =
audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, rate=(int)44100, channels=(int)2,
codec_data=(buffer)1210
/GstPipeline:pipeline0/GstFlvMux:mux.GstFlvMuxPad:sink_1: caps =
audio/mpeg, mpegversion=(int)4, framed=(boolean)true,
stream-format=(string)raw, rate=(int)44100, channels=(int)2,
codec_data=(buffer)1210
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps =
video/x-h264, stream-format=(string)avc, width=(int)1920,
height=(int)1080,
codec_data=(buffer)0142002affe100166742002a95a81e0089f961000003000100000300648401000468ce3c80
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps =
video/x-h264, stream-format=(string)avc, width=(int)1920,
height=(int)1080,
codec_data=(buffer)0142002affe100166742002a95a81e0089f961000003000100000300648401000468ce3c80
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink.GstProxyPad:proxypad0:
caps = video/x-h264, stream-format=(string)avc, width=(int)1920,
height=(int)1080,
codec_data=(buffer)0142002affe100166742002a95a81e0089f961000003000100000300648401000468ce3c80
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src:
caps = video/x-h264, stream-format=(string)avc, width=(int)1920,
height=(int)1080,
codec_data=(buffer)0142002affe100166742002a95a81e0089f961000003000100000300648401000468ce3c80
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse1.GstPad:sink:
caps = video/x-h264, stream-format=(string)avc, width=(int)1920,
height=(int)1080,
codec_data=(buffer)0142002affe100166742002a95a81e0089f961000003000100000300648401000468ce3c80
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:sink:
caps = video/x-h264, stream-format=(string)avc, width=(int)1920,
height=(int)1080,
codec_data=(buffer)0142002affe100166742002a95a81e0089f961000003000100000300648401000468ce3c80
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps
= video/x-h264, stream-format=(string)avc, width=(int)1920, height=(int)1080,
codec_data=(buffer)0142002affe100166742002a95a81e0089f961000003000100000300648401000468ce3c80
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse1.GstPad:src:
caps = video/x-h264, stream-format=(string)byte-stream,
width=(int)1920, height=(int)1080, framerate=(fraction)50/1,
interlace-mode=(string)progressive, chroma-format=(string)4:2:0,
bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8,
parsed=(boolean)true, alignment=(string)au, profile=(string)baseline,
level=(string)4.2 Opening in BLOCKING MODE 0:00:00.823485062 834
0x55b9ce8a30 WARN v4l2
gstv4l2object.c:4408:gst_v4l2_object_probe_caps:nvv4l2decoder0:src
Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error
-1 0:00:00.823543397 834 0x55b9ce8a30 WARN v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f6807a640
Failed to determine interlace mode NvMMLiteOpen : Block : BlockType =
261 NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink:
caps = video/x-h264, stream-format=(string)byte-stream,
width=(int)1920, height=(int)1080, framerate=(fraction)50/1,
interlace-mode=(string)progressive, chroma-format=(string)4:2:0,
bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8,
parsed=(boolean)true, alignment=(string)au, profile=(string)baseline,
level=(string)4.2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1.GstPad:src:
caps = video/x-h264, stream-format=(string)byte-stream,
width=(int)1920, height=(int)1080, framerate=(fraction)50/1,
interlace-mode=(string)progressive, chroma-format=(string)4:2:0,
bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8,
parsed=(boolean)true, alignment=(string)au, profile=(string)baseline,
level=(string)4.2
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1.GstPad:sink:
caps = video/x-h264, stream-format=(string)byte-stream,
width=(int)1920, height=(int)1080, framerate=(fraction)50/1,
interlace-mode=(string)progressive, chroma-format=(string)4:2:0,
bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8,
parsed=(boolean)true, alignment=(string)au, profile=(string)baseline,
level=(string)4.2 0:00:00.932089228 834 0x55b9ce8a30 WARN
v4l2
gstv4l2object.c:4408:gst_v4l2_object_probe_caps:nvv4l2decoder0:src
Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error
-1 0:00:00.932560124 834 0x55b9ce8a30 WARN v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f6807a640
Failed to determine interlace mode
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:src:
caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920,
height=(int)1080, interlace-mode=(string)progressive,
multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2,
colorimetry=(string)bt709, framerate=(fraction)50/1(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.065:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.066:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.066:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.066:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.066:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.066:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.067:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.067:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.068:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.068:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.068:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.068:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.068:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.069:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed
0:00:00.942959420 834 0x55b9ce8a30 WARN GST_PADS
gstpad.c:4226:gst_pad_peer_query:nvv4l2decoder0:src could not send
sticky events 0:00:00.943568965 834 0x55b9ce8a30 WARN
v4l2videodec
gstv4l2videodec.c:1433:gst_v4l2_video_dec_decide_allocation:
Duration invalid, not setting latency 0:00:00.944316482 834
0x55b9ce8a30 WARN v4l2bufferpool
gstv4l2bufferpool.c:1054:gst_v4l2_buffer_pool_start:nvv4l2decoder0:pool:src
Uncertain or not enough buffers, enabling copy threshold(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.073:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.074:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.074:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.074:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.074:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.074:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.074:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed
0:00:00.948613871 834 0x55b9ce8a30 WARN basetransform
gstbasetransform.c:1355:gst_base_transform_setcaps:
transform could not transform video/x-h264,
stream-format=(string)byte-stream, width=(int)1920, height=(int)1080,
framerate=(fraction)50/1, interlace-mode=(string)progressive,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au,
profile=(string)baseline, level=(string)4.2 in anything we support
0:00:00.948674601 834 0x55b9ce8a30 WARN basetransform
gstbasetransform.c:1415:gst_base_transform_reconfigure:
warning: not negotiated 0:00:00.948709446 834 0x55b9ce8a30 WARN
basetransform
gstbasetransform.c:1415:gst_base_transform_reconfigure:
warning: not negotiated WARNING: from element
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1:
not negotiated Additional debug info: gstbasetransform.c(1415):
gst_base_transform_reconfigure ():
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1:
not negotiated 0:00:00.971426937 834 0x7f70004a80 WARN
basesrc gstbasesrc.c:3055:gst_base_src_loop: error: Internal
data stream error. 0:00:00.971545793 834 0x7f70004a80 WARN
basesrc gstbasesrc.c:3055:gst_base_src_loop: error:
streaming stopped, reason not-negotiated (-4) 0:00:00.978535326 834
0x7f6807c8f0 WARN v4l2bufferpool
gstv4l2bufferpool.c:1518:gst_v4l2_buffer_pool_dqbuf:nvv4l2decoder0:pool:src
Driver should never set v4l2_buffer.field to ANY(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.105:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.105:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.105:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.105:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.105:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.105:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.105:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed
ERROR: from element /GstPipeline:pipeline0/GstRTMPSrc:rtmpsrc0:
Internal data stream error.(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.106:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.107:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.107:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.107:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed
Additional debug info: gstbasesrc.c(3055): gst_base_src_loop ():
/GstPipeline:pipeline0/GstRTMPSrc:rtmpsrc0: streaming stopped, reason
not-negotiated (-4) ERROR: pipeline doesn’t want to preroll. Setting
pipeline to NULL …(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.107:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_mini_object_copy: assertion ‘mini_object != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.108:
gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.109:
gst_structure_copy: assertion ‘structure != NULL’ failed(gst-launch-1.0:834): GStreamer-CRITICAL **: 14:19:35.109:
gst_caps_append_structure_full: assertion ‘GST_IS_CAPS (caps)’ failed
Freeing pipeline …