Using deepstream+gstreamer+opencv can not open a local video

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Please provide complete information as applicable to your setup.

• Hardware Platform ( Tesla T4)
• CUDA Version: 11.4
• Ubuntu Version: 18.04
• DeepStream Version: 6.0.1
• gstreamer Version: 1.14.5
• TensorRT Version: 8.0.1
• NVIDIA GPU Driver Version: 470.82.01
• Issue Type(bugs)
**• opencv version : 4.6.0 **

demo:

int main() {
string url = “/opt/nvidia/deepstream/deepstream6.0/samples/streams/sample_720p.mp4”;
cv::VideoCapture cap(url, cv::CAP_GSTREAMER);
if(!cap.isOpened()) {
cout << "Connect the URL failed: " << url.toStdString();
cap.release();
} else {
cv::Mat img;;
cap.read(img);
cv::imshow(“s”, img);
cv::waitKey(0);
cout <<URL is a valid address: " << url.toStdString();
cap.release();
}
return 0;
}

it would not open the video!

logs:

[ WARN:0@0.292] global /home/lgl/Downloads/opencv4.6/opencv/modules/videoio/src/cap_gstreamer.cpp (2402) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module qtdemux0 reported: Internal data stream error.
[ WARN:0@0.310] global /home/lgl/Downloads/opencv4.6/opencv/modules/videoio/src/cap_gstreamer.cpp (1356) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@0.310] global /home/lgl/Downloads/opencv4.6/opencv/modules/videoio/src/cap_gstreamer.cpp (862) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Connect the rtsp URL failed

details:

0:00:00.012687594 19582 0x55f6436b9a40 WARN basesrc gstbasesrc.c:3583:gst_base_src_start_complete: pad not activated yet
0:00:00.012923453 19582 0x55f6436b9a40 WARN basesrc gstbasesrc.c:3583:gst_base_src_start_complete: pad not activated yet
0:00:00.022294944 19582 0x7fb0b407bd40 WARN qtdemux qtdemux_types.c:233:qtdemux_type_get: unknown QuickTime node type sgpd
0:00:00.022340318 19582 0x7fb0b407bd40 WARN qtdemux qtdemux_types.c:233:qtdemux_type_get: unknown QuickTime node type sbgp
0:00:00.022390593 19582 0x7fb0b407bd40 WARN qtdemux qtdemux.c:3031:qtdemux_parse_trex: failed to find fragment defaults for stream 1
0:00:00.022585844 19582 0x7fb0b407bd40 WARN qtdemux qtdemux.c:3031:qtdemux_parse_trex: failed to find fragment defaults for stream 2
0:00:00.105382136 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105410663 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe minimum capture size for pixelformat MJPG
0:00:00.105420451 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105429079 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe maximum capture size for pixelformat MJPG
0:00:00.105456337 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105468192 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe minimum capture size for pixelformat MPG4
0:00:00.105482479 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105494294 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe maximum capture size for pixelformat MPG4
0:00:00.105517536 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105530404 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe minimum capture size for pixelformat MPG2
0:00:00.105540415 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105550814 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe maximum capture size for pixelformat MPG2
0:00:00.105570974 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105581634 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe minimum capture size for pixelformat H265
0:00:00.105591054 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105601936 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe maximum capture size for pixelformat H265
0:00:00.105619335 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105632922 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe minimum capture size for pixelformat VP90
0:00:00.105643045 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105653503 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe maximum capture size for pixelformat VP90
0:00:00.105669112 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105679182 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe minimum capture size for pixelformat VP80
0:00:00.105688961 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105698916 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe maximum capture size for pixelformat VP80
0:00:00.105717206 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105727938 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe minimum capture size for pixelformat H264
0:00:00.105738194 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:sink Unable to try format: Unknown error -1
0:00:00.105748295 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:sink Could not probe maximum capture size for pixelformat H264
0:00:00.106448216 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:src Unable to try format: Unknown error -1
0:00:00.106462865 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2937:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:src Could not probe minimum capture size for pixelformat NM12
0:00:00.106473818 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:3051:gst_v4l2_object_get_nearest_size:nvv4l2decoder0:src Unable to try format: Unknown error -1
0:00:00.106482302 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2943:gst_v4l2_object_probe_caps_for_format:nvv4l2decoder0:src Could not probe maximum capture size for pixelformat NM12
0:00:00.106496612 19582 0x7fb0ac014590 WARN v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7fb0b0021620 Failed to determine interlace mode
0:00:00.294482596 19582 0x7fb0ac014590 WARN basetransform gstbasetransform.c:1355:gst_base_transform_setcaps: transform could not transform video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 in anything we support
0:00:00.294601009 19582 0x7fb0ac014590 WARN basetransform gstbasetransform.c:1355:gst_base_transform_setcaps: transform could not transform video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 in anything we support
0:00:00.294617504 19582 0x7fb0ac014590 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:uridecodebin0:src_0 could not send sticky events
0:00:00.294853195 19582 0x7fb0ac014590 WARN v4l2videodec gstv4l2videodec.c:1685:gst_v4l2_video_dec_decide_allocation: Duration invalid, not setting latency
0:00:00.294901655 19582 0x7fb0ac014590 WARN v4l2bufferpool gstv4l2bufferpool.c:1065:gst_v4l2_buffer_pool_start:nvv4l2decoder0:pool:src Uncertain or not enough buffers, enabling copy threshold
0:00:00.295851144 19582 0x7fb0ac014590 WARN basetransform gstbasetransform.c:1355:gst_base_transform_setcaps: transform could not transform video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)3.1, profile=(string)high, width=(int)1280, height=(int)720, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true in anything we support
0:00:00.295872856 19582 0x7fb0ac014590 WARN basetransform gstbasetransform.c:1415:gst_base_transform_reconfigure: warning: not negotiated
0:00:00.295880336 19582 0x7fb0ac014590 WARN basetransform gstbasetransform.c:1415:gst_base_transform_reconfigure: warning: not negotiated
0:00:00.296076985 19582 0x7fb0b407bd40 WARN qtdemux qtdemux.c:6073:gst_qtdemux_loop: error: Internal data stream error.
0:00:00.296112899 19582 0x7fb0b407bd40 WARN qtdemux qtdemux.c:6073:gst_qtdemux_loop: error: streaming stopped, reason not-negotiated (-4)
[ WARN:0@0.297] global /home/lgl/Downloads/opencv4.6/opencv/modules/videoio/src/cap_gstreamer.cpp (2402) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module qtdemux0 reported: Internal data stream error.
[ WARN:0@0.315] global /home/lgl/Downloads/opencv4.6/opencv/modules/videoio/src/cap_gstreamer.cpp (1356) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@0.315] global /home/lgl/Downloads/opencv4.6/opencv/modules/videoio/src/cap_gstreamer.cpp (862) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Connect the rtsp URL failed

3

when I uninstall the deepstream, the demo would run ok. I think the nvv4l2decoder cause the this problem?

4

I think you are right.

Deepstream uses GPU decoding by default, so the rank of nvv4ldecoder is set to the highest level, but the memory of the GPU and the system memory cannot access each other.

Try the below code to disable nvv4ldecoder.

static guint disableElement(const char *name) {
  GstRegistry* pluginsRegister = gst_registry_get();
  GstPluginFeature* feature = gst_registry_lookup_feature(pluginsRegister, name);
  if (feature == NULL) {
    return GST_RANK_NONE;
  }
  guint rank = gst_plugin_feature_get_rank(feature);
  gst_plugin_feature_set_rank(feature, GST_RANK_NONE);
  gst_object_unref(feature);
  return rank;
}
5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.