Use Case:
A USB Camera with YUY2 output connected to a Jetson Nano.
The stream is used as an input to a DeepStream 4.0.1 C-Code based GStreamer plugin chain.
Working example using videoconvert and nvvideoconvert to create the input for nvstreammux:
GST_DEBUG=3, gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, width=640, height=480, format=(string)YUY2" ! videoconvert ! 'video/x-raw,format=NV12' ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=NV12' ! m.sink_0 nvstreammux name=m width=640 height=480 batch-size=1 ! nvegltransform ! nveglglessink sync=False
Because videoconvert is not CUDA accelerated we like to use nvvidconv which does not work:
GST_DEBUG=3, gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, width=640, height=480, format=(string)YUY2" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! m.sink_0 nvstreammux name=m width=640 height=480 batch-size=1 ! nvegltransform ! nveglglessink sync=False
Setting pipeline to PAUSED ...
Using winsys: x11
Pipeline is live and does not need PREROLL ...
0:00:00.309557319 9712 0x7f68003d80 WARN structure gststructure.c:1832:priv_gst_structure_append_to_gstring: No value transform to serialize field 'display' of type 'GstEGLDisplay'
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.470290732 9712 0x55892042d0 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:02.305725101 9712 0x55892042d0 WARN nvstreammux gstnvstreammux.c:308:gst_nvstreammux_chain:<m> error: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
0:00:02.306312766 9712 0x55892042d0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:02.306389903 9712 0x55892042d0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason error (-5)
ERROR: from element /GstPipeline:pipeline0/GstNvStreamMux:m: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
Additional debug info:
/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(308): gst_nvstreammux_chain (): /GstPipeline:pipeline0/GstNvStreamMux:m
Execution ended after 0:00:01.996307263
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Adding nvvideoconvert does not solve the number of surfaces issue:
GST_DEBUG=3, gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, width=640, height=480, format=(string)YUY2" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=NV12' ! m.sink_0 nvstreammux name=m width=640 height=480 batch-size=1 ! nvegltransform ! nveglglessink sync=False
(gst-launch-1.0:9794): GLib-GObject-WARNING **: 09:58:18.091: cannot register existing type 'GstInterpolationMethod'
(gst-launch-1.0:9794): GLib-GObject-CRITICAL **: 09:58:18.091: g_param_spec_enum: assertion 'G_TYPE_IS_ENUM (enum_type)' failed
(gst-launch-1.0:9794): GLib-GObject-CRITICAL **: 09:58:18.091: validate_pspec_to_install: assertion 'G_IS_PARAM_SPEC (pspec)' failed
Setting pipeline to PAUSED ...
Using winsys: x11
Pipeline is live and does not need PREROLL ...
0:00:00.321576338 9794 0x559f6ea040 WARN structure gststructure.c:1832:priv_gst_structure_append_to_gstring: No value transform to serialize field 'display' of type 'GstEGLDisplay'
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.420532954 9794 0x559f2cc540 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:02.253261447 9794 0x559f2cc540 WARN nvstreammux gstnvstreammux.c:308:gst_nvstreammux_chain:<m> error: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
0:00:02.253569472 9794 0x559f2cc540 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:02.253692963 9794 0x559f2cc540 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason error (-5)
ERROR: from element /GstPipeline:pipeline0/GstNvStreamMux:m: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
Set nvstreammux property num-surfaces-per-frame appropriately
Additional debug info:
/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(308): gst_nvstreammux_chain (): /GstPipeline:pipeline0/GstNvStreamMux:m
Execution ended after 0:00:01.931606202
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Thank you for your support!