Nvstreamdemux not work as expected with decodebin

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU)
GTX2080ti
• DeepStream Version
DeepStream 5.0.1
• TensorRT Version
TensorRT 7.1.3.4
• NVIDIA GPU Driver Version (valid for GPU only)
450.66

send rtp command :
gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.h264 ! h264parse ! rtph264pay ! udpsink host=“127.0.0.1” port=20000

rtmp streaming command: [this work]
gst-launch-1.0 -v udpsrc port=20000 do-timestamp=1 caps=“application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264,playload=(int)96” ! rtph264depay ! h264parse ! decodebin ! nvvideoconvert ! x264enc ! queue ! flvmux streamable=1 ! rtmpsink location=“rtmp://127.0.0.1/live/100000”

when I add the infer plugins with nvstreammux and nvstreamdemux .
this not work : it suddenly end the pipeline loop

gst-launch-1.0 -v udpsrc port=20000 do-timestamp=1 caps=“application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264,playload=(int)96” ! rtph264depay ! h264parse ! decodebin ! queue ! m.sink_0 nvstreammux name=m width=1280 height=720 batch-size=1 ! queue ! nvinfer config-file-path="/opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.txt
" ! nvstreamdemux name=d d.src_0 ! nvvideoconvert ! x264enc ! queue ! flvmux streamable=1 ! rtmpsink location=“rtmp://127.0.0.1/live/100000”

Also I successfully connect this by code, but it stuck at the nvstreamdemux plugin, seems no stream output.
ERROR:gst_pad_chain_data_unchecked:iva-streamdemux:sinke[00m called chainfunction &gst_nvstreamdemux_sink_chain_cuda_batch with buffer 0x7ffea0058350, returned error

EDIT: If I replace nvstramdemux with nvmultistreamtiler, then the pipeline worked, WHY?
It seems not only with the udpsrc or we use rtspsrc / filesrc.
both the same error!

Any advices? Thanks?

2

0:00:55.523144927 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00;01;33m GST_CLOCK gstclock.c:1046:gst_clock_get_internal_time:e[00m internal time 198:55:10.695785135
0:00:55.523193026 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00;01;33m GST_CLOCK gstclock.c:1091:gst_clock_get_time:e[00m adjusted time 198:55:10.695785135
0:00:55.523231230 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00m nvstreammux gstnvstreammux.c:2196:gst_nvstreammux_src_push_loop:e[00m Pushing buffer 0x7ffea0058350, batch size 1, PTS 0:00:51.089233330
0:00:55.523266556 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00m nvstreammux gstnvstreammux.c:2218:gst_nvstreammux_src_push_loop:e[00m STREAMMUX OUT BUFFER attached timestamp 0:00:51.089233330
0:00:55.523387171 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:<core_bin0:sink>e[00m calling chainfunction &gst_proxy_pad_chain_default with buffer buffer: 0x7ffea0058350, pts 0:00:51.089233330, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.523447248 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:<core_queue0:sink>e[00m calling chainfunction &gst_queue_chain with buffer buffer: 0x7ffea0058350, pts 0:00:51.089233330, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.523493579 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:<core_queue0:sink>e[00m called chainfunction &gst_queue_chain with buffer 0x7ffea0058350, returned ok
0:00:55.523506800 e[334m18951e[00m 0x7fff8c4b4770 e[37mDEBUG e[00m e[00m queue_dataflow gstqueue.c:1528:gst_queue_loop:<core_queue0>e[00m queue is not empty
0:00:55.523521436 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:<core_bin0:sink>e[00m called chainfunction &gst_proxy_pad_chain_default with buffer 0x7ffea0058350, returned ok
0:00:55.523572879 e[334m18951e[00m 0x7fff8c4b4770 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:<core_infer0:sink>e[00m calling chainfunction &gst_base_transform_chain with buffer buffer: 0x7ffea0058350, pts 0:00:51.089233330, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.523667920 e[334m18951e[00m 0x7ffe9c0051e0 e[37mDEBUG e[00m e[00;01;42m GST_BUFFER gstbuffer.c:2202:gst_buffer_add_meta:e[00m alloc metadata 0x7ffea0099308 (NvDsMeta) of size 72
0:00:55.523891472 e[334m18951e[00m 0x7fff8c4b4770 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:<core_infer0:sink>e[00m called chainfunction &gst_base_transform_chain with buffer 0x7ffea0058350, returned ok
0:00:55.523921128 e[334m18951e[00m 0x7fff8c4b4770 e[37mDEBUG e[00m e[00m queue_dataflow gstqueue.c:1516:gst_queue_loop:<core_queue0>e[00m queue is empty
0:00:55.524030815 e[334m18951e[00m 0x7ffe84002550 e[37mDEBUG e[00m e[00;01;42m GST_BUFFER gstbuffer.c:1375:gst_buffer_is_memory_range_writable:e[00m idx 0, length -1
0:00:55.531935725 e[334m18951e[00m 0x7fff8c4b4720 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:<core_tracker0:sink>e[00m calling chainfunction &gst_base_transform_chain with buffer buffer: 0x7ffea0058350, pts 0:00:51.089233330, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.532099146 e[334m18951e[00m 0x7fff8c4b4720 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:<core_tracker0:sink>e[00m called chainfunction &gst_base_transform_chain with buffer 0x7ffea0058350, returned ok
0:00:55.532261851 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:src:proxypad4e[00m calling chainfunction &gst_proxy_pad_chain_default with buffer buffer: 0x7ffea0058350, pts 0:00:51.089233330, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.532313436 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:iva-streamdemux:sinke[00m calling chainfunction &gst_nvstreamdemux_sink_chain_cuda_batch with buffer buffer: 0x7ffea0058350, pts 0:00:51.089233330, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.532379454 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;34m GST_CAPS gstpad.c:2705:gst_pad_has_current_caps:iva-streamdemux:src_0e[00m check current pad caps video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, framerate=(fraction)25/1, batch-size=(int)1, num-surfaces-per-frame=(int)1
0:00:55.532405557 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;42m GST_BUFFER gstbuffer.c:1148:gst_buffer_get_memory_range:e[00m idx 0, length 1
0:00:55.532433388 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;34m GST_MEMORY gstmemory.c:139:gst_memory_init:e[00m new memory 0x7ffea03edba0, maxsize:64 offset:0 size:64
0:00:55.532491196 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;42m GST_BUFFER gstbuffer.c:2202:gst_buffer_add_meta:e[00m alloc metadata 0x7ffe9002ed88 (NvDsMeta) of size 72
0:00:55.532536742 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:<sink_bin0:sink>e[00m calling chainfunction &gst_proxy_pad_chain_default with buffer buffer: 0x7ffea005e030, pts 0:00:50.848190110, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.532565869 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:<sink_queue0:sink>e[00m calling chainfunction &gst_queue_chain with buffer buffer: 0x7ffea005e030, pts 0:00:50.848190110, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:55.532603749 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;34m GST_MEMORY gstmemory.c:88:_gst_memory_free:e[00m free memory 0x7ffea03edba0
0:00:55.532628112 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:<sink_queue0:sink>e[00m called chainfunction &gst_queue_chain with buffer 0x7ffea005e030, returned error
0:00:55.532649542 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:<sink_bin0:sink>e[00m called chainfunction &gst_proxy_pad_chain_default with buffer 0x7ffea005e030, returned error
0:00:55.532670296 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;42m GST_BUFFER gstbuffer.c:2384:gst_buffer_foreach_meta:e[00m remove metadata 0x7ffe9002ec68 (NvDsMeta)
0:00:55.532710536 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00m nvdspool gstnvdsbufferpool.c:492:gst_nvds_buffer_pool_release_buffer:e[00m release_buffer
0:00:55.532729846 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;42m GST_BUFFER gstbuffer.c:1375:gst_buffer_is_memory_range_writable:e[00m idx 0, length -1
0:00:55.532749866 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;42m GST_BUFFER gstbuffer.c:1375:gst_buffer_is_memory_range_writable:e[00m idx 0, length -1
0:00:55.532770354 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:iva-streamdemux:sinke[00m called chainfunction &gst_nvstreamdemux_sink_chain_cuda_batch with buffer 0x7ffea0058350, returned error
0:00:55.532792235 e[334m18951e[00m 0x7fff8c5c10a0 e[37mDEBUG e[00m e[00;01;35m GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:src:proxypad4e[00m called chainfunction &gst_proxy_pad_chain_default with buffer 0x7ffea0058350, returned error

4

Also I try this command: (without nvinfer )

gst-launch-1.0 -v udpsrc port=20000 do-timestamp=1 caps=“application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264,playload=(int)96” ! rtph264depay ! h264parse ! decodebin ! queue ! m.sink_0 nvstreammux name=m width=1280 height=720 batch-size=1 ! queue ! nvstreamdemux name=d d.src_0 ! nvvideoconvert ! x264enc ! queue ! flvmux streamable=1 ! rtmpsink location=“rtmp://127.0.0.1/live/100000”

It just output decode signals ,seems no debug info from nvstreamdemux?

0:00:01.999112664 24488 0x55deea9422d0 INFO v4l2bufferpool gstv4l2bufferpool.c:841:gst_v4l2_buffer_pool_set_config:nvv4l2decoder0:pool:src can’t allocate, setting maximum to minimum
0:00:01.999194768 24488 0x55deea9422d0 WARN v4l2videodec gstv4l2videodec.c:1618:gst_v4l2_video_dec_decide_allocation: Duration invalid, not setting latency
0:00:01.999247671 24488 0x55deea9422d0 WARN v4l2bufferpool gstv4l2bufferpool.c:1057:gst_v4l2_buffer_pool_start:nvv4l2decoder0:pool:src Uncertain or not enough buffers, enabling copy threshold
0:00:01.999451525 24488 0x55deea9422d0 INFO task gsttask.c:457:gst_task_set_lock: setting stream lock 0x7fd05009e8f0 on task 0x55deea844b90
0:00:01.999479713 24488 0x55deea9422d0 INFO GST_PADS gstpad.c:6154:gst_pad_start_task:nvv4l2decoder0:src created task 0x55deea844b90
0:00:02.000138968 24488 0x55deea9422d0 INFO basetransform gstbasetransform.c:1308:gst_base_transform_setcaps: reuse caps
0:00:02.108121560 24488 0x7fd05005e370 WARN v4l2bufferpool gstv4l2bufferpool.c:1503:gst_v4l2_buffer_pool_dqbuf:nvv4l2decoder0:pool:src Driver should never set v4l2_buffer.field to ANY
0:00:02.108283827 24488 0x7fd05005e370 INFO videodecoder gstvideodecoder.c:3201:gst_video_decoder_clip_and_push_buf: First buffer since flush took 0:00:00.231158189 to produce
0:00:02.108398950 24488 0x55deea942280 INFO videoencoder gstvideoencoder.c:1144:gst_video_encoder_sink_event_default: upstream tags: taglist;
0:00:02.108459642 24488 0x55deea942280 INFO videoencoder gstvideoencoder.c:1144:gst_video_encoder_sink_event_default: upstream tags: taglist;
0:00:02.108516859 24488 0x55deea942280 INFO GST_EVENT gstevent.c:895:gst_event_new_segment: creating segment event time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999
0:00:02.108789838 24488 0x55deea942280 ERROR default video-frame.c:175:gst_video_frame_map_id: invalid buffer size 64 < 1382400
0:00:02.108815356 24488 0x55deea942280 ERROR x264enc gstx264enc.c:2447:gst_x264_enc_handle_frame: Failed to map frame
0:00:02.108866383 24488 0x55deea942280 INFO task gsttask.c:316:gst_task_func:queue1:src Task going to paused
0:00:02.187456032 24488 0x55deea942140 INFO task gsttask.c:316:gst_task_func:queue0:src Task going to paused
0:00:02.227574910 24488 0x55deea9422d0 INFO baseparse gstbaseparse.c:4777:gst_base_parse_set_upstream_tags: upstream tags: taglist, video-codec=(string)“H.264\ (High\ Profile)”, minimum-bitrate=(uint)776000, maximum-bitrate=(uint)776000, bitrate=(uint)2447680;
0:00:02.227646347 24488 0x55deea9422d0 INFO videodecoder gstvideodecoder.c:1330:gst_video_decoder_sink_event_default: upstream tags: taglist, video-codec=(string)“H.264\ (High\ Profile)”, minimum-bitrate=(uint)776000, maximum-bitrate=(uint)776000, bitrate=(uint)2447680;
0:00:02.227964864 24488 0x7fd05005e370 INFO task gsttask.c:316:gst_task_func:nvv4l2decoder0:src Task going to paused
0:00:02.267700787 24488 0x55deea9422d0 INFO baseparse gstbaseparse.c:4777:gst_base_parse_set_upstream_tags: upstream tags: taglist, video-codec=(string)“H.264\ (High\ Profile)”, minimum-bitrate=(uint)765600, maximum-bitrate=(uint)776000, bitrate=(uint)2294763;
0:00:02.267808932 24488 0x55deea9422d0 INFO videodecoder gstvideodecoder.c:1330:gst_video_decoder_sink_event_default: upstream tags: taglist, video-codec=(string)“H.264\ (High\ Profile)”, minimum-bitrate=(uint)765600, maximum-bitrate=(uint)776000, bitrate=(uint)2294763;

5

Hey customer, would you mind to share the pipeline which can work well?

6

I found the problem.
nvstreamdemux pad connection code changed
We can not use link_element_to_demux_src_pad API from deepstream_common.c
(which is the same as Deepstream 3)

I changed that refer to deepstream_app.c
then it worked!

7

Maybe deepstream5
you changed the pad connection code for nvstreamdemux
Which you should update the API makes that work

8

Cool, may I know which function are you referring?

9

Line 1105 from deepstream_app.c [ in function create_pipeline]

gchar pad_name[16];
g_snprintf(pad_name, 16, “src_%02d”, index);
demux_src_pad = gst_element_get_request_pad(demux, pad_name);
NVGSTDS_LINK_ELEMENT_FULL(demux, pad_name, elem, “sink”);