rtsp inference

I have been trying to make this pipeline work for a couple of days already and hit a wall…
I am trying to make a “server” that takes a rtsp stream and then re-streams it with gst/rtsp-server

The pipeline looks something like this:

rtspsrc location="rtsp://192.168.0.66"
   ! rtph264depay ! h264parse ! omxh264dec
   ! m.sink_0 nvstreammux name=m batch-size=1 live-source=true width=2688 height=1520
   ! nvinfer config-file-path=nvinfer.txt
   ! nvvidconv ! video/x-raw(memory:NVMM),format=RGBA ! nvdsosd ! nvvidconv ! video/x-raw(memory:NVMM),format=I420
   ! omxh264enc ! h264parse ! rtph264pay name=pay0 pt=96

I found that nvstreammux is required for nvinfer (Deepstream SDK 3.0)(please correct me if I am wrong). Now it looks like it complains about the number of surfaces. I have been looking through the examples and they do not setup anything else, what am I missing?
Removing lines 3 and 4 works and I am able to re-stream the video.
Here is the output:

0:00:05.834879477 17382   0x55b8d10d90 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/esteban/.config:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Creating LL OSD context new
0:00:10.215608720 17382   0x7ef0006770 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:10.215866017 17382   0x7f003ce190 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:10.320470770 17382   0x7ef00065e0 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc2> error: Internal data stream error.
0:00:10.320880622 17382   0x7ef00065e0 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc2> error: streaming stopped, reason not-linked (-1)
0:00:12.309996946 17382   0x7ec80034a0 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec0> Sub-class should implement drain()
0:00:12.325193828 17382   0x7ec80034a0 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<h264parse0:src> could not send sticky events

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.388: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_structure_get_string: assertion 'structure != NULL' failed
0:00:12.336783447 17382   0x7ec80034a0 ERROR            omxvideodec gstomxvideodec.c:3274:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec0> Invalid caps: (NULL)

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:40.389: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:12.342775552 17382   0x7ec80034a0 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec0> Sub-class should implement drain()
Allocating new output: 2688x1520 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 2688, nFrameHeight = 1520 
Creating LL OSD context new
0:00:12.386941012 17382   0x7ef0006630 FIXME           videoencoder gstvideoencoder.c:661:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> GstVideoEncoder::reset() is deprecated
Framerate set to : 20 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
0:00:12.388795010 17382   0x7ef0006630 WARN             omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
H264: Profile = 66, Level = 40 
0:00:12.500529652 17382   0x7ef0006630 WARN             nvstreammux gstnvstreammux.c:308:gst_nvstreammux_chain:<m> error: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
	Set nvstreammux property num-surfaces-per-frame appropriately

0:00:12.500660488 17382   0x7ef0006630 WARN             omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: Internal data stream error.
0:00:12.500689916 17382   0x7ef0006630 WARN             omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: stream stopped, reason error
0:00:12.500763355 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
	Set nvstreammux property num-surfaces-per-frame appropriately
 (/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(308): gst_nvstreammux_chain (): /GstPipeline:media-pipeline/GstBin:bin0/GstNvStreamMux:m)
0:00:12.501152061 17382   0x55b8d10d90 WARN               rtspmedia rtsp-media.c:2994:wait_preroll: failed to preroll pipeline
0:00:12.501177686 17382   0x55b8d10d90 WARN               rtspmedia rtsp-media.c:3298:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:12.502088747 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Internal data stream error. (/dvs/git/dirty/git-master_linux/3rdparty/gst/gst-omx/omx/gstomxvideodec.c(3005): gst_omx_video_dec_loop (): /GstPipeline:media-pipeline/GstBin:bin0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason error)
0:00:12.536356603 17382   0x7f3c005590 WARN                 rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc0> error: Unhandled error
0:00:12.536400250 17382   0x7f3c005590 WARN                 rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc0> error: Method Not Valid in This State (455)
0:00:12.536513117 17382   0x7f3c005590 WARN                 rtspsrc gstrtspsrc.c:8244:gst_rtspsrc_pause:<rtspsrc0> error: Could not send message. (Generic error)
0:00:12.536587025 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Unhandled error (gstrtspsrc.c(6161): gst_rtspsrc_send (): /GstPipeline:media-pipeline/GstBin:bin0/GstRTSPSrc:rtspsrc0:
Method Not Valid in This State (455))
0:00:12.536679683 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e210: got error Could not write to resource. (gstrtspsrc.c(8244): gst_rtspsrc_pause (): /GstPipeline:media-pipeline/GstBin:bin0/GstRTSPSrc:rtspsrc0:
Could not send message. (Generic error))
0:00:17.547756971 17382   0x55b8d10d90 ERROR                    omx gstomx.c:2064:gst_omx_port_wait_buffers_released_unlocked:<omxh264dec-omxh264dec0> Timeout waiting for decode port 1 to release all buffers
0:00:17.578449960 17382   0x7f3c005590 WARN                 rtspsrc gstrtspsrc.c:5999:gst_rtspsrc_try_send:<rtspsrc0> send interrupted
0:00:17.578495951 17382   0x7f3c005590 WARN                 rtspsrc gstrtspsrc.c:7673:gst_rtspsrc_close:<rtspsrc0> TEARDOWN interrupted
0:00:17.581941752 17382   0x55b8d10d90 ERROR             rtspclient rtsp-client.c:1054:find_media: client 0x55b8dba0f0: can't prepare media
0:00:17.582766613 17382   0x55b8d10d90 ERROR             rtspclient rtsp-client.c:2910:handle_describe_request: client 0x55b8dba0f0: no media
Creating LL OSD context new
0:00:18.103361627 17382   0x7ec80034a0 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc2:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:18.104535922 17382   0x7ea4003400 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc3:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:18.190139181 17382   0x7eb4004b20 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc6> error: Internal data stream error.
0:00:18.190660026 17382   0x7eb4004b20 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc6> error: streaming stopped, reason not-linked (-1)
0:00:20.320330344 17382   0x7ed4002a30 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec1> Sub-class should implement drain()
0:00:20.330312325 17382   0x7ed4002a30 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<h264parse2:src> could not send sticky events

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.391: gst_structure_get_string: assertion 'structure != NULL' failed
0:00:20.339492570 17382   0x7ed4002a30 ERROR            omxvideodec gstomxvideodec.c:3274:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec1> Invalid caps: (NULL)

(cameraserver:17382): GStreamer-CRITICAL **: 00:36:48.392: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:20.346838766 17382   0x7ed4002a30 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec1> Sub-class should implement drain()
Allocating new output: 2688x1520 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 2688, nFrameHeight = 1520 
Creating LL OSD context new
0:00:20.387993121 17382   0x7ee0009e30 FIXME           videoencoder gstvideoencoder.c:661:gst_video_encoder_setcaps:<omxh264enc-omxh264enc1> GstVideoEncoder::reset() is deprecated
Framerate set to : 20 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
0:00:20.390094988 17382   0x7ee0009e30 WARN             omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc1> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
H264: Profile = 66, Level = 40 
0:00:20.505743775 17382   0x7ee0009e30 WARN             nvstreammux gstnvstreammux.c:308:gst_nvstreammux_chain:<m> error: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
	Set nvstreammux property num-surfaces-per-frame appropriately

0:00:20.506054198 17382   0x7ee0009e30 WARN             omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec1> error: Internal data stream error.
0:00:20.506110085 17382   0x7ee0009e30 WARN             omxvideodec gstomxvideodec.c:3005:gst_omx_video_dec_loop:<omxh264dec-omxh264dec1> error: stream stopped, reason error
0:00:20.506118523 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)
	Set nvstreammux property num-surfaces-per-frame appropriately
 (/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(308): gst_nvstreammux_chain (): /GstPipeline:media-pipeline/GstBin:bin1/GstNvStreamMux:m)
0:00:20.506294047 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Internal data stream error. (/dvs/git/dirty/git-master_linux/3rdparty/gst/gst-omx/omx/gstomxvideodec.c(3005): gst_omx_video_dec_loop (): /GstPipeline:media-pipeline/GstBin:bin1/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec1:
stream stopped, reason error)
0:00:20.506337954 17382   0x55b8d10d90 WARN               rtspmedia rtsp-media.c:2994:wait_preroll: failed to preroll pipeline
0:00:20.506455821 17382   0x55b8d10d90 WARN               rtspmedia rtsp-media.c:3298:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:20.530217415 17382   0x7eb40048a0 WARN                 rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc1> error: Unhandled error
0:00:20.530264135 17382   0x7eb40048a0 WARN                 rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc1> error: Method Not Valid in This State (455)
0:00:20.530398773 17382   0x7eb40048a0 WARN                 rtspsrc gstrtspsrc.c:8244:gst_rtspsrc_pause:<rtspsrc1> error: Could not send message. (Generic error)
0:00:20.530422055 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Unhandled error (gstrtspsrc.c(6161): gst_rtspsrc_send (): /GstPipeline:media-pipeline/GstBin:bin1/GstRTSPSrc:rtspsrc1:
Method Not Valid in This State (455))
0:00:20.530523099 17382   0x7f7c44c590 WARN               rtspmedia rtsp-media.c:2725:default_handle_message: 0x7f7c45e3e0: got error Could not write to resource. (gstrtspsrc.c(8244): gst_rtspsrc_pause (): /GstPipeline:media-pipeline/GstBin:bin1/GstRTSPSrc:rtspsrc1:
Could not send message. (Generic error))
0:00:25.563111767 17382   0x55b8d10d90 ERROR                    omx gstomx.c:2064:gst_omx_port_wait_buffers_released_unlocked:<omxh264dec-omxh264dec1> Timeout waiting for decode port 1 to release all buffers
0:00:25.592255609 17382   0x7eb40048a0 WARN                 rtspsrc gstrtspsrc.c:5999:gst_rtspsrc_try_send:<rtspsrc1> send interrupted
0:00:25.592290349 17382   0x7eb40048a0 WARN                 rtspsrc gstrtspsrc.c:7673:gst_rtspsrc_close:<rtspsrc1> TEARDOWN interrupted
0:00:25.596398352 17382   0x55b8d10d90 ERROR             rtspclient rtsp-client.c:1054:find_media: client 0x55b8dba200: can't prepare media
0:00:25.597519053 17382   0x55b8d10d90 ERROR             rtspclient rtsp-client.c:2639:handle_setup_request: client 0x55b8dba200: media '/test' not found

Thanks in advance,
Esteban

Hi,
DS3.0 is not for Nano. Please upgrade to DS4.0. Your usecase is similar to deepstream-test3. Please take a look at the sample.

My mistake, I am on DS4.0

I looked at that example and it is not streaming to rtsp. The GstRTSPServer and GstRTSPMediaFactory play nice when you have a pipeline that is string-based. deepstream-test3 is not string based, it builds and links all the pipeline’s elements manually.

I was looking for someone to be able to identify what could be missing in my string-based pipeline.

Hi,
You may refer to the samples to construct the string-based pipeline. In running DS4.0, please use nvv4l2decoder instead of omxh264dec, and nvv4l2h264enc/nvv4l2h265enc instead of omxh264enc/omxh265enc. We have unified v4l2 plugins for desktop GPUs and Jetson platforms.

I tried using nvv4l2decoder/nvv4l2h264enc with a simpler pipeline and didnt work, here is the pipeline I used:

rtspsrc location="rtsp://192.168.0.66"
        ! rtph264depay ! h264parse ! nvv4l2decoder
        ! nvv4l2h264enc ! h264parse ! rtph264pay name=pay0 pt=96

Output:

Opening in BLOCKING MODE 
0:00:02.428636375  6359   0x7f9800e370 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f98089840 Failed to determine interlace mode
0:00:02.428725699  6359   0x7f9800e370 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f98089840 Failed to determine interlace mode
0:00:02.428778981  6359   0x7f9800e370 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f98089840 Failed to determine interlace mode
0:00:02.428895284  6359   0x7f9800e370 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE 
0:00:02.431346048  6359   0x7f9800e370 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:02.431455528  6359   0x7f9800e370 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f980851d0 Failed to determine interlace mode
0:00:02.614335042  6359   0x7f88005770 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:02.614714839  6359   0x7f90035d90 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:02.751880178  6359   0x7f88005450 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc3> error: Internal data stream error.
0:00:02.752073775  6359   0x7f88005450 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc3> error: streaming stopped, reason not-linked (-1)
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(cameraserver:6359): GStreamer-CRITICAL **: 06:55:37.463: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:04.867285385  6359   0x7f5c0038a0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:04.867696120  6359   0x7f5c0038a0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f980851d0 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
0:00:04.883582069  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1054:gst_v4l2_buffer_pool_start:<nvv4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:04.884831357  6359   0x7f5c0038a0 WARN            v4l2videodec gstv4l2videodec.c:1433:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:04.898659674  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:04.913335294  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:04.923871898  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:04.935054241  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:04.947964941  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:04.959175825  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:04.971623654  6359   0x7f88005630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1518:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:04.971644956  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
H264: Profile = 66, Level = 0 
0:00:04.981391237  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:04.994318187  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:05.003649410  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:05.008376090  6359   0x7f88005630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1463:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:sink> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:05.008429789  6359   0x7f88005630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1463:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:sink> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:05.008620365  6359   0x7f70009cf0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1518:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:05.011290714  6359   0x5562a03d90 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:05.012950789  6359   0x5562a03d90 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:05.012997561  6359   0x5562a03d90 WARN               rtspmedia rtsp-media.c:3867:gst_rtsp_media_suspend: media 0x7f980961b0 was not prepared
0:00:05.016192970  6359   0x7f70009cf0 FIXME               basesink gstbasesink.c:3145:gst_base_sink_default_event:<multiudpsink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:05.016499432  6359   0x5562a03d90 FIXME             rtspclient rtsp-client.c:1657:handle_play_request:<GstRTSPClient@0x5562aad0f0> Add support for seek style (null)
0:00:05.016663445  6359   0x5562a03d90 FIXME              rtspmedia rtsp-media.c:2437:gst_rtsp_media_seek_full:<GstRTSPMedia@0x7f980961b0> Handle going back to 0 for none live not seekable streams.
0:00:05.017464550  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.

(cameraserver:6359): GStreamer-CRITICAL **: 06:55:37.723: gst_structure_set: assertion 'IS_MUTABLE (structure) || field == NULL' failed

(cameraserver:6359): GStreamer-CRITICAL **: 06:55:37.723: gst_structure_set: assertion 'IS_MUTABLE (structure) || field == NULL' failed
0:00:05.022527797  6359   0x7f88005630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1463:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:sink> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:05.022589048  6359   0x7f88005630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1463:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:sink> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:05.027065464  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:05.027154788  6359   0x7f88005630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1463:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:sink> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:05.027190466  6359   0x7f88005630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1463:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:sink> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:05.035904441  6359   0x7f5c0038a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 976 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:15.577538706  6359   0x7f90035680 WARN                 rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc0> error: Unhandled error
0:00:15.577734490  6359   0x7f90035680 WARN                 rtspsrc gstrtspsrc.c:6161:gst_rtspsrc_send:<rtspsrc0> error: Method Not Valid in This State (455)
0:00:15.578113558  6359   0x7f90035680 WARN                 rtspsrc gstrtspsrc.c:8244:gst_rtspsrc_pause:<rtspsrc0> error: Could not send message. (Generic error)

The same pipeline with omxh264enc/omxh264dec works:

0:00:01.626888902  6620   0x55ba603d90 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/esteban/.config:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
0:00:01.849547896  6620   0x7f98006e30 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:01.849855450  6620   0x7f98006450 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:01.935200473  6620   0x7f98006ca0 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc3> error: Internal data stream error.
0:00:01.935467611  6620   0x7f98006ca0 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc3> error: streaming stopped, reason not-linked (-1)
0:00:04.155812663  6620   0x7f6c002b20 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec0> Sub-class should implement drain()
0:00:04.161363702  6620   0x7f6c002b20 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<h264parse0:src> could not send sticky events

(cameraserver:6620): GStreamer-CRITICAL **: 06:58:35.586: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:6620): GStreamer-CRITICAL **: 06:58:35.586: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:6620): GStreamer-CRITICAL **: 06:58:35.587: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:6620): GStreamer-CRITICAL **: 06:58:35.587: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed

(cameraserver:6620): GStreamer-CRITICAL **: 06:58:35.587: gst_structure_get_string: assertion 'structure != NULL' failed
0:00:04.172071609  6620   0x7f6c002b20 ERROR            omxvideodec gstomxvideodec.c:3274:gst_omx_video_dec_negotiate:<omxh264dec-omxh264dec0> Invalid caps: (NULL)

(cameraserver:6620): GStreamer-CRITICAL **: 06:58:35.587: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:04.179462873  6620   0x7f6c002b20 FIXME           videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out:<omxh264dec-omxh264dec0> Sub-class should implement drain()
Allocating new output: 2688x1520 (x 14), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 2688, nFrameHeight = 1520 
0:00:04.191168081  6620   0x7f98006cf0 FIXME           videoencoder gstvideoencoder.c:661:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> GstVideoEncoder::reset() is deprecated
Framerate set to : 20 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
0:00:04.192720282  6620   0x7f98006cf0 WARN             omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
H264: Profile = 66, Level = 40 
0:00:04.339988933  6620   0x7f80009c00 WARN               h264parse gsth264parse.c:2752:gst_h264_parse_set_caps:<h264parse1> H.264 AVC caps, but no codec_data
0:00:04.340108361  6620   0x7f80009c00 WARN               h264parse gsth264parse.c:2778:gst_h264_parse_set_caps:<h264parse1> refused caps video/x-h264, alignment=(string)au, profile=(string)baseline, level=(string)4, stream-format=(string)avc, width=(int)2688, height=(int)1520, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
0:00:04.340191071  6620   0x7f80009c00 WARN               h264parse gsth264parse.c:2752:gst_h264_parse_set_caps:<h264parse1> H.264 AVC caps, but no codec_data
0:00:04.340255498  6620   0x7f80009c00 WARN               h264parse gsth264parse.c:2778:gst_h264_parse_set_caps:<h264parse1> refused caps video/x-h264, alignment=(string)au, profile=(string)baseline, level=(string)4, stream-format=(string)avc, width=(int)2688, height=(int)1520, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
0:00:04.340294405  6620   0x7f80009c00 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<omxh264enc-omxh264enc0:src> could not send sticky events
0:00:04.342588696  6620   0x55ba603d90 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:04.343795530  6620   0x55ba603d90 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:04.343842978  6620   0x55ba603d90 WARN               rtspmedia rtsp-media.c:3867:gst_rtsp_media_suspend: media 0x7fa80992c0 was not prepared
0:00:04.347304571  6620   0x7f80009c00 FIXME               basesink gstbasesink.c:3145:gst_base_sink_default_event:<multiudpsink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:04.347721085  6620   0x55ba603d90 FIXME             rtspclient rtsp-client.c:1657:handle_play_request:<GstRTSPClient@0x55ba6ad0f0> Add support for seek style (null)
0:00:04.348070672  6620   0x55ba603d90 FIXME              rtspmedia rtsp-media.c:2437:gst_rtsp_media_seek_full:<GstRTSPMedia@0x7fa80992c0> Handle going back to 0 for none live not seekable streams.

The pipeline with inference and nvv4l2decoder/nvv4l2h264enc crashes:

Opening in BLOCKING MODE 
0:00:02.509370739  6754   0x7fac3e98f0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7fac444930 Failed to determine interlace mode
0:00:02.509462459  6754   0x7fac3e98f0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7fac444930 Failed to determine interlace mode
0:00:02.509506834  6754   0x7fac3e98f0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7fac444930 Failed to determine interlace mode
0:00:02.509627148  6754   0x7fac3e98f0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE 
0:00:02.555252173  6754   0x7fac3e98f0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:02.555351237  6754   0x7fac3e98f0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7fac0855a0 Failed to determine interlace mode
Creating LL OSD context new
0:00:06.705478309  6754   0x7f20005770 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:06.706353579  6754   0x7f683c4d90 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<fakesrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:06.795947263  6754   0x7f200055e0 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc4> error: Internal data stream error.
0:00:06.796536071  6754   0x7f200055e0 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc4> error: streaming stopped, reason not-linked (-1)
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(cameraserver:6754): GStreamer-CRITICAL **: 07:00:04.684: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:08.948152244  6754   0x7ef40034a0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:08.948535894  6754   0x7ef40034a0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7fac0855a0 Failed to determine interlace mode
Creating LL OSD context new
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
0:00:08.985785780  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1054:gst_v4l2_buffer_pool_start:<nvv4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:08.987402151  6754   0x7ef40034a0 WARN            v4l2videodec gstv4l2videodec.c:1433:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:08.997852630  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1054:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:09.011039545  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.023982707  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.035234861  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.044905540  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.053993608  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.064231792  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.075398425  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.075400977  6754   0x7ed800ce30 WARN          v4l2bufferpool gstv4l2bufferpool.c:1518:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:09.084878580  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.094237224  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.104446502  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.114526144  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:09.173881167  6754   0x7ef40034a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
Segmentation fault (core dumped)

Hi,
You may try deepstream-app and enable rtspsink in the config file.
Please refer to document and the post

I want to take the input from a web camera to my Jetson NANO and run yolo object detector on it and stream the result to a Jetson Xavier as RSTP. Is there a sample app I should specifically look at ? If not, can you help me to set the necessary configurations.

Hi,
For running Yolo, please check

deepstream_sdk_v4.0_jetson\sources\objectDetector_Yolo\README

Thanks, through the deepstream-app it works… now to play on getting a better detection…
In the future I will try to recreate the pipeline with gstream that deepstream-app is using… It would be helpful to figure out what was missing in the above pipeline

Hi,
Since there are bunch of parameters being configured into a deepstream pipeline, we design the config file as a quick start guide. All samples are open source and please refer to it to construct the pipeline.