Some guidance on how to proceed after a 'streaming stopped, reason error (-5)'

• NVIDIA Jetson Nano (Developer Kit Version)
• DeepStream Version 6.0.1
• JetPack Version R32 (release), REVISION: 7.2
• TensorRT Version 8.2.1.8
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type question

jetsonUtilities (master) $ ./jetsonInfo.py
NVIDIA Jetson Nano (Developer Kit Version)
 L4T 32.7.2 [ JetPack UNKNOWN ]
   Ubuntu 18.04.6 LTS
   Kernel Version: 4.9.253-tegra
 CUDA 10.2.300
   CUDA Architecture: 5.3
 OpenCV version: 4.1.1
   OpenCV Cuda: NO
 CUDNN: 8.2.1.32
 TensorRT: 8.2.1.8
 Vision Works: 1.6.0.501
 VPI: ii libnvvpi1 1.2.3 arm64 NVIDIA Vision Programming Interface library
 Vulcan: 1.2.70

I have a rather complex pipeline like:

gst-launch-1.0 v4l2src io-mode=2 do-timestamp=TRUE device=/dev/video1 ! "video/x-raw,width=960,height=540,framerate=20/1" ! nvvidconv ! "video/x-raw(memory:NVMM)" ! mtdmetainject traffic-report-name=bob ! "video/x-raw(memory:NVMM)" ! mtdmetaoverlay qml=/etc/mtdata/qml/simple.qml ! "video/x-raw(memory:NVMM)" ! nvvidconv ! "video/x-raw(memory:NVMM),format=I420" ! queue ! nvvidconv ! video/x-raw ! mtddriversafety detect-drowsiness=false detect-distraction=true driver-location=right distraction-min-speed-kmh=10 distraction-min-calibration-speed-kmh=40 distraction-cal-sec=60 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=NV12" ! nvv4l2h264enc maxperf-enable=1 ! h264parse config-interval=1 ! capsfilter caps="video/x-h264,stream-format=avc,alignment=au,profile=baseline,mapping=/mtdstream1" ! rtspsink service=8554

which works.

  • mtdmetainject just injects some custom meta data into the stream
  • mtdmetaoverlay draws some of this meta data on the image
  • mtddriversafety is a bin which has 2 nvinfers to do face and facial landmark-detection, and draws these on the image

I would now like to move the mtdmetaoverlay to after the inference (otherwise it gets in the way of the face detection)
so i try :

gst-launch-1.0 v4l2src io-mode=2 do-timestamp=TRUE device=/dev/video1 ! "video/x-raw,width=960,height=540,framerate=20/1" ! mtdmetainject traffic-report-name=bob ! nvvidconv ! "video/x-raw(memory:NVMM),format=I420" ! queue ! nvvidconv ! video/x-raw ! mtddriversafety detect-drowsiness=false detect-distraction=true driver-location=right distraction-min-speed-kmh=10 distraction-min-calibration-speed-kmh=40 distraction-cal-sec=60 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mtdmetaoverlay qml=/etc/mtdata/qml/simple.qml ! "video/x-raw(memory:NVMM)" ! nvvidconv ! "video/x-raw(memory:NVMM),format=I420" ! nvv4l2h264enc maxperf-enable=1 ! h264parse config-interval=1 ! capsfilter caps="video/x-h264,stream-format=avc,alignment=au,profile=baseline,mapping=/mtdstream1" ! rtspsink service=8554

(moved mtdmetaoverlay towards the end) which keeps going inference starts to process (after about 5 seconds) , after which i see:

00:08.299583647 11390   0x55907895e0 WARN                 nvinfer gstnvinfer.cpp:2288:gst_nvinfer_output_loop:<nvinfer-second-10> error: Internal data stream error.
0:00:08.299619221 11390   0x55907895e0 WARN                 nvinfer gstnvinfer.cpp:2288:gst_nvinfer_output_loop:<nvinfer-second-10> error: streaming stopped, reason error (-5)
0:00:08.299684796 11390   0x55907895e0 INFO        GST_ERROR_SYSTEM gstelement.c:2145:gst_element_message_full_with_details:<nvinfer-second-10> posting message: Internal data stream error.
0:00:08.299796414 11390   0x55907895e0 INFO        GST_ERROR_SYSTEM gstelement.c:2172:gst_element_message_full_with_details:<nvinfer-second-10> posted error message: Internal data stream error.
ERROR: from element /GstPipeline:pipeline0/GstMtdDriverSafety:mtddriversafety0/GstNvInfer:nvinfer-second-10: Internal data stream error.
Additional debug info:
/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(2288): gst_nvinfer_output_loop (): /GstPipeline:pipeline0/GstMtdDriverSafety:mtddriversafety0/GstNvInfer:nvinfer-second-10:
streaming stopped, reason error (-5)
Execution ended after 0:00:01.066967747
Setting pipeline to PAUSED ...
0:00:08.300426592 11390   0x558ff5ea00 INFO              GST_STATES gstbin.c:2506:gst_bin_element_set_state:<rtspsink0> current READY pending PAUSED, desired next PAUSED

but i don’t have any idea on how to proceed from here. what is this telling me ? It looks like the second nvinfer in the mtddriversafety is triggering this, but in which direction should i look ? Is this a mismatch in the pipeline of caps ? is there any way to get more information. I tried GST_DEBUG=6 gst-launch-1.0 ...., but that killed me with logging and no more of the critical area that i have added above. any help is appreciated.

full log:

full.log (287.1 KB)

  1. gstnvinfer plugin is opensource. from the logs, gstnvinfer gst_pad_push failed, in the other words, downstream plugin handled data failed.
  2. from full.log, there are some abnormal printing,
    nvbuf_utils: nvbuffer Payload Type not supported
    NvBufferGetParams failed for src_dmabuf_fd
    nvbuffer_transform Failed
    gst_nvvconv_transform: NvBufferTransform Failed
  3. please add logs in your plugin to debug. please provide simple code to reproduce if found deepstream bug.
1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.