decode the rtsp stream from a IP camera

jkjung13 · July 4, 2019, 8:46am

I think replacing ‘xvimagesink’ with ‘appsink’ would solve the problem.

cap = cv2.VideoCapture('rtspsrc location=rtsp://admin:admin@192.168.1.168 latency=0 ! '
                       'application/x-rtp,media=video ! decodebin ! videoconvert ! appsink',
                       cv2.CAP_GSTREAMER)

Hryy · September 12, 2019, 7:05am

Hi @jkjung13,
Your ip camera solution works well , but it’s too slow,very low frame rate, how to solve it?

jkjung13 · September 13, 2019, 12:56pm

@Hryy, if the TX2 H.264/H.265 hardware decoder works for your IP cam, I suggest you to just follow my original ‘tegra-cam.py’ code. Reference: [url]decode the rtsp stream from a IP camera - Jetson TX2 - NVIDIA Developer Forums

suruicong1991 · November 10, 2019, 5:18pm

@jkjung13 First of all, thank you very much for all your blogs and Community topics and replies about opencv on jetson serial devices. Those instructions save me a lot of time since I first saw your article.

However, I met the same problem about IP Camero capture or load stream as this topic post. it is the reason that I post this reply here, thought there are more logs of test result.
The story is:
My first thought was to reinstall on a clear tx2, when I failed loading ip camero with your python code tegra-cam.py. And I have just had time today to try again building and installing opencv on tx2 again follow your steps on web page:
https://jkjung-avt.github.io/opencv3-on-tx2/
and:
https://jkjung-avt.github.io/opencv-on-nano/

Then, after running the test instruction, I got some difference between my test result and yours. The test result might give some hit for the gstreamer problem.

The sumary is:

The following tests FAILED:
	  2 - opencv_test_core (Failed)
	  3 - opencv_test_cudaarithm (SEGFAULT)
	  6 - opencv_test_ml (Failed)
	 10 - opencv_test_cudaimgproc (Failed)
	 14 - opencv_test_photo (Failed)
	 <b>16 - opencv_test_videoio (Failed)</b> (different from your result)
         <s>20 - opencv_test_calib3d (Failed)</s> (passed on my tx2)
         <s>23 - opencv_test_objdetect (Failed)</s> (passed on my tx2)
	 24 - opencv_test_cudalegacy (Failed)
Errors while running CTest

The details of the test of videoio is:

16/29 Testing: opencv_test_videoio
16/29 Test: opencv_test_videoio
Command: "/home/nvidia/Documents/OpenCV/build/bin/opencv_test_videoio" "--gtest_output=xml:opencv_test_videoio.xml"
Directory: /home/nvidia/Documents/OpenCV/build/test-reports/accuracy
"opencv_test_videoio" start time: Nov 10 23:23 CST
Output:
----------------------------------------------------------
CTEST_FULL_OUTPUT
OpenCV version: 3.4.6
OpenCV VCS version: 3.4.6
Build type: RELEASE
WARNING: build value differs from runtime: Release
Compiler: /usr/bin/c++  (ver 5.4.0)
Parallel framework: pthreads
CPU features: NEON FP16
OpenCL is disabled
[==========] Running 94 tests from 7 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from videoio_avi
[ RUN      ] videoio_avi.good_MJPG
[       OK ] videoio_avi.good_MJPG (5 ms)
[ RUN      ] videoio_avi.bad_MJPG
[       OK ] videoio_avi.bad_MJPG (1 ms)
[ RUN      ] videoio_avi.basic
[       OK ] videoio_avi.basic (0 ms)
[----------] 3 tests from videoio_avi (7 ms total)

[----------] 2 tests from Videoio_Video
[ RUN      ] Videoio_Video.ffmpeg_writebig
OpenCV: FFMPEG: tag 0x34363248/'H264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x00000021/'!???'
[       OK ] Videoio_Video.ffmpeg_writebig (33458 ms)
[ RUN      ] Videoio_Video.ffmpeg_image
[       OK ] Videoio_Video.ffmpeg_image (10 ms)
[----------] 2 tests from Videoio_Video (33468 ms total)

[----------] 1 test from Videoio_Video_parallel_writers_and_readers
[ RUN      ] Videoio_Video_parallel_writers_and_readers.accuracy
[       OK ] Videoio_Video_parallel_writers_and_readers.accuracy (18743 ms)
[----------] 1 test from Videoio_Video_parallel_writers_and_readers (18743 ms total)

[----------] 1 test from Videoio_dshow
[ RUN      ] Videoio_dshow.fourcc_conversion
[       OK ] Videoio_dshow.fourcc_conversion (0 ms)
[----------] 1 test from Videoio_dshow (0 ms total)

[----------] 48 tests from videoio/Videoio_Bunny
[ RUN      ] videoio/Videoio_Bunny.read_position/0, where GetParam() = ("wmv", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.wmv[/i][/b]
[       OK ] videoio/Videoio_Bunny.read_position/0 (32 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/1, where GetParam() = ("wmv", FFMPEG)
[       OK ] videoio/Videoio_Bunny.read_position/1 (276 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/2, where GetParam() = ("wmv", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.wmv</i></b>
[       OK ] videoio/Videoio_Bunny.read_position/2 (1 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/3, where GetParam() = ("mov", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mov[/i][/b]
[       OK ] videoio/Videoio_Bunny.read_position/3 (23 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/4, where GetParam() = ("mov", FFMPEG)
[       OK ] videoio/Videoio_Bunny.read_position/4 (367 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/5, where GetParam() = ("mov", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mov</i></b>
[       OK ] videoio/Videoio_Bunny.read_position/5 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/6, where GetParam() = ("mp4", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mp4[/i][/b]
[       OK ] videoio/Videoio_Bunny.read_position/6 (12 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/7, where GetParam() = ("mp4", FFMPEG)
[       OK ] videoio/Videoio_Bunny.read_position/7 (367 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/8, where GetParam() = ("mp4", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mp4</i></b>
[       OK ] videoio/Videoio_Bunny.read_position/8 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/9, where GetParam() = ("mpg", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mpg[/i][/b]
[       OK ] videoio/Videoio_Bunny.read_position/9 (12 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/10, where GetParam() = ("mpg", FFMPEG)
[       OK ] videoio/Videoio_Bunny.read_position/10 (317 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/11, where GetParam() = ("mpg", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mpg</i></b>
[       OK ] videoio/Videoio_Bunny.read_position/11 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/12, where GetParam() = ("avi", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.avi[/i][/b]
[       OK ] videoio/Videoio_Bunny.read_position/12 (11 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/13, where GetParam() = ("avi", FFMPEG)
[       OK ] videoio/Videoio_Bunny.read_position/13 (4039 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/14, where GetParam() = ("avi", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.avi</i></b>
[       OK ] videoio/Videoio_Bunny.read_position/14 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/15, where GetParam() = ("h264", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h264[/i][/b]
[       OK ] videoio/Videoio_Bunny.read_position/15 (16 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/16, where GetParam() = ("h264", FFMPEG)
CAP_PROP_FRAME_COUNT is not supported by backend. Assume 50 frames.
[       OK ] videoio/Videoio_Bunny.read_position/16 (146 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/17, where GetParam() = ("h264", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h264</i></b>
[       OK ] videoio/Videoio_Bunny.read_position/17 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/18, where GetParam() = ("h265", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h265[/i][/b]
[       OK ] videoio/Videoio_Bunny.read_position/18 (12 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/19, where GetParam() = ("h265", FFMPEG)
CAP_PROP_FRAME_COUNT is not supported by backend. Assume 50 frames.
[       OK ] videoio/Videoio_Bunny.read_position/19 (317 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/20, where GetParam() = ("h265", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h265</i></b>
[       OK ] videoio/Videoio_Bunny.read_position/20 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/21, where GetParam() = ("mjpg.avi", GSTREAMER)

<b><i>(opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_query_set_position: assertion 'format == g_value_get_enum (gst_structure_id_get_value (s, GST_QUARK (FORMAT)))' failed</i></b>
Seek to frame '20' is not supported. SKIP.
Seek to frame '40' is not supported. SKIP.
Seek to frame '60' is not supported. SKIP.
Seek to frame '80' is not supported. SKIP.
Seek to frame '100' is not supported. SKIP.
Seek to frame '120' is not supported. SKIP.
Seek to frame '26' is not supported. SKIP.
Seek to frame '18' is not supported. SKIP.
Seek to frame '116' is not supported. SKIP.
Seek to frame '106' is not supported. SKIP.
Seek to frame '100' is not supported. SKIP.
Seek to frame '90' is not supported. SKIP.
Seek to frame '81' is not supported. SKIP.
Seek to frame '111' is not supported. SKIP.
Seek to frame '97' is not supported. SKIP.
Seek to frame '38' is not supported. SKIP.
[       OK ] videoio/Videoio_Bunny.read_position/21 (5251 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/22, where GetParam() = ("mjpg.avi", FFMPEG)
[       OK ] videoio/Videoio_Bunny.read_position/22 (877 ms)
[ RUN      ] videoio/Videoio_Bunny.read_position/23, where GetParam() = ("mjpg.avi", CV_MJPEG)
[       OK ] videoio/Videoio_Bunny.read_position/23 (436 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/0, where GetParam() = ("wmv", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.wmv[/i][/b]
[       OK ] videoio/Videoio_Bunny.frame_count/0 (7 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/1, where GetParam() = ("wmv", FFMPEG)
[       OK ] videoio/Videoio_Bunny.frame_count/1 (256 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/2, where GetParam() = ("wmv", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.wmv</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/2 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/3, where GetParam() = ("mov", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mov[/i][/b]
[       OK ] videoio/Videoio_Bunny.frame_count/3 (9 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/4, where GetParam() = ("mov", FFMPEG)
[       OK ] videoio/Videoio_Bunny.frame_count/4 (167 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/5, where GetParam() = ("mov", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mov</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/5 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/6, where GetParam() = ("mp4", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mp4[/i][/b]
[       OK ] videoio/Videoio_Bunny.frame_count/6 (9 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/7, where GetParam() = ("mp4", FFMPEG)
[       OK ] videoio/Videoio_Bunny.frame_count/7 (163 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/8, where GetParam() = ("mp4", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mp4</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/8 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/9, where GetParam() = ("mpg", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mpg[/i][/b]
[       OK ] videoio/Videoio_Bunny.frame_count/9 (6 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/10, where GetParam() = ("mpg", FFMPEG)
[       OK ] videoio/Videoio_Bunny.frame_count/10 (258 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/11, where GetParam() = ("mpg", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mpg</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/11 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/12, where GetParam() = ("avi", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.avi[/i][/b]
[       OK ] videoio/Videoio_Bunny.frame_count/12 (8 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/13, where GetParam() = ("avi", FFMPEG)
[       OK ] videoio/Videoio_Bunny.frame_count/13 (479 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/14, where GetParam() = ("avi", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.avi</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/14 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/15, where GetParam() = ("h264", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h264[/i][/b]
[       OK ] videoio/Videoio_Bunny.frame_count/15 (8 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/16, where GetParam() = ("h264", FFMPEG)
Frames counter is not available. Actual frames: 125. SKIP check.
[       OK ] videoio/Videoio_Bunny.frame_count/16 (331 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/17, where GetParam() = ("h264", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h264</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/17 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/18, where GetParam() = ("h265", GSTREAMER)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h265[/i][/b]
[       OK ] videoio/Videoio_Bunny.frame_count/18 (10 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/19, where GetParam() = ("h265", FFMPEG)
Frames counter is not available. Actual frames: 125. SKIP check.
[       OK ] videoio/Videoio_Bunny.frame_count/19 (503 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/20, where GetParam() = ("h265", CV_MJPEG)
<b><i>SKIP test: backend CV_MJPEG can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.h265</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/20 (0 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/21, where GetParam() = ("mjpg.avi", GSTREAMER)

<b><i>(opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_query_set_position: assertion 'format == g_value_get_enum (gst_structure_id_get_value (s, GST_QUARK (FORMAT)))' failed</i></b>
[       OK ] videoio/Videoio_Bunny.frame_count/21 (5196 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/22, where GetParam() = ("mjpg.avi", FFMPEG)
[       OK ] videoio/Videoio_Bunny.frame_count/22 (365 ms)
[ RUN      ] videoio/Videoio_Bunny.frame_count/23, where GetParam() = ("mjpg.avi", CV_MJPEG)
[       OK ] videoio/Videoio_Bunny.frame_count/23 (369 ms)
[----------] 48 tests from videoio/Videoio_Bunny (20658 ms total)

[----------] 28 tests from videoio/Videoio_Synthetic
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/0, where GetParam() = (640x480, FOURCC(XVID), .avi, FFMPEG, 30dB)
[       OK ] videoio/Videoio_Synthetic.write_read_position/0 (1435 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/1, where GetParam() = (640x480, FOURCC(MPEG), .avi, FFMPEG, 30dB)
[       OK ] videoio/Videoio_Synthetic.write_read_position/1 (2066 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/2, where GetParam() = (640x480, FOURCC(IYUV), .avi, FFMPEG, 30dB)
[       OK ] videoio/Videoio_Synthetic.write_read_position/2 (936 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/3, where GetParam() = (640x480, FOURCC(MJPG), .avi, FFMPEG, 30dB)
[       OK ] videoio/Videoio_Synthetic.write_read_position/3 (2467 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/4, where GetParam() = (640x480, FOURCC(XVID), .mkv, FFMPEG, 30dB)
[       OK ] videoio/Videoio_Synthetic.write_read_position/4 (1458 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/5, where GetParam() = (640x480, FOURCC(MPEG), .mkv, FFMPEG, 30dB)
[       OK ] videoio/Videoio_Synthetic.write_read_position/5 (2069 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/6, where GetParam() = (640x480, FOURCC(MJPG), .mkv, FFMPEG, 30dB)
[       OK ] videoio/Videoio_Synthetic.write_read_position/6 (2515 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/7, where GetParam() = (640x480, FOURCC(MPEG), .avi, GSTREAMER, 30dB)

[b][i](opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed

** (opencv_test_videoio:22376): CRITICAL **: gst_missing_encoder_message_new: assertion 'gst_caps_is_fixed (encode_caps)' failed

(opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_post_message: assertion 'message != NULL' failed

(opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /tmp/__opencv_temp.qg5UrR.MPEG.avi[/i][/b]
[       OK ] videoio/Videoio_Synthetic.write_read_position/7 (256 ms)
[ RUN      ] videoio/Videoio_Synthetic.write_read_position/8, where GetParam() = (640x480, FOURCC(MJPG), .avi, GSTREAMER, 30dB)

<b><i>(opencv_test_videoio:22376): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed</i></b>
JPEG parameter struct mismatch: library thinks size is 584, caller expects 712
<end of output>
Test time =  86.20 sec
----------------------------------------------------------
Test Failed.
"opencv_test_videoio" end time: Nov 10 23:25 CST
"opencv_test_videoio" time elapsed: 00:01:26
----------------------------------------------------------

As the log shows, the gstreamer did not work well on many test tasks. I hope to get some help from you about the gstreamer, and here are some info of my tx2 and my installation steps.

The os of my tx2 is Ubuntu 16.04, which is based on jetpack 3.3 installed by official run file "JetPack-L4T-3.3-linux-x64_b39.run";
I have compared the script file "install_opencv-3.4.6.sh" and the the scripts you paste in "How to Install OpenCV (3.4.0) on Jetson TX2";
After comparison, I notice that the difference of dependencies installed by apt-get is only libjasper-dev. And, I decide not to install jasper;
I only install python3 and needed dependencies by pip3, which I'll always use in the future. Python2 is the one installed by os default. No pip2 installed, no numpy or other dependencies for python2 installed;
Change "matplotlibrc" file;
Change "cuda_gl_interop.h" file. You gave two different ways in the two web page. But the result is same. I modified the file by vim, actually gedit;
Follow "How to Install OpenCV (3.4.0) on Jetson TX2" add soft link for libGL.so;
Checkout opencv 3.4.6 from git. Then mkdir build and cd into the folder;
Use the cmake settings post on "Installing OpenCV 3.4.6 on Jetson Nano", changed CUDA_ARCH_BIN="6.2", and change BUILD_TESTS=ON;
In ./configure, some problems was shown, a. Unexpected option: ENABLE_FAST_MATH (=ON) b. CBLAS/LAPACK headers are not found in
Test result was different from yours, but build well.

I haven’t tried the ip camera loading or ways showed here, I’ll reply here later.

suruicong1991 · November 10, 2019, 7:24pm

@jkjung13 my test result is:
With your suggest test command

gst-launch-1.0 rtspsrc latency=200 location=<rtsp-url> ! rtph264depay ! h264parse ! omxh264dec ! nveglglessink

it open the video and almost real-time.(about half-second delay)

Copy the string direct into cv2.VideoCapture:

vc = cv2.VideoCapture('gst-launch-1.0 rtspsrc latency=200 location=<rtsp-url> ! rtph264depay ! h264parse ! omxh264dec ! nveglglessink', cv2.CAP_GSTREAMER)

I got error as:

(python3:8944): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed

By only change nveglglessink to appsink:

vc = cv2.VideoCapture('gst-launch-1.0 rtspsrc latency=200 location=<rtsp-url> ! rtph264depay ! h264parse ! omxh264dec ! appsink', cv2.CAP_GSTREAMER)

NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1179: BeginSequence  1920x1088, bVPR = 0
TVMR: LowCorner Frequency = 180000 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 5, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1600: Display Resolution : (1920x1080) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1920x1080) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1674 ColorSpace = NvColorSpace_YCbCr709_ER
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 12, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1
Allocating new output: 1920x1088 (x 12), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 1920, nFrameHeight = 1088 
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 

(python3:8879): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed

By using your cv2.VideoCapture script and change the rtsp url:

vc = cv2.VideoCapture('rtspsrc location=<rtsp-url> ! '
                           'application/x-rtp,media=video ! decodebin ! videoconvert ! appsink',
                           cv2.CAP_GSTREAMER)

it works but about 5s delay.

I’m seeing the physical accelerate method.

suruicong1991 · November 11, 2019, 3:58am

@jkjung13 New progressing. I think I solve some part of the problem.
I compiled OpenCV again, just after install the dependencies for rtsp ip camera, as you wrote in “How to Capture and Display Camera Video with Python on Jetson TX2”:
https://jkjung-avt.github.io/tx2-camera-with-python/

sudo apt-get install gstreamer1.0-plugins-bad-faad \
                     gstreamer1.0-plugins-bad-videoparsers

Then, the make test still failed on videoio module, but with fewer failure task according to the log file than I last post here #24, I only post the remain failure here:

............
[----------] 2 tests from Videoio_Video
[ RUN      ] Videoio_Video.ffmpeg_writebig
OpenCV: FFMPEG: tag 0x34363248/'H264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x00000021/'!???'
[       OK ] Videoio_Video.ffmpeg_writebig (35043 ms)
[ RUN      ] Videoio_Video.ffmpeg_image
[       OK ] Videoio_Video.ffmpeg_image (11 ms)
[----------] 2 tests from Videoio_Video (35054 ms total)
............
[ RUN      ] videoio/Videoio_Bunny.read_position/9, where GetParam() = ("mpg", GSTREAMER)

(opencv_test_videoio:4673): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.mpg
[       OK ] videoio/Videoio_Bunny.read_position/9 (10 ms)
............
[ RUN      ] videoio/Videoio_Bunny.read_position/12, where GetParam() = ("avi", GSTREAMER)
NvMMLiteOpen : Block : BlockType = 260 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 260 
TVMR: cbBeginSequence: 1179: BeginSequence  672x384, bVPR = 0
TVMR: LowCorner Frequency = 100000 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 3, pnvsi->eCodec = 2, codec = 6 
TVMR: cbBeginSequence: 1600: Display Resolution : (672x384) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (672x384) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 0, TransferCharacteristics = 0, MatrixCoefficients = 0
Allocating new output: 672x384 (x 10), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 672, nFrameHeight = 384 

(opencv_test_videoio:4673): GStreamer-CRITICAL **: gst_query_set_position: assertion 'format == g_value_get_enum (gst_structure_id_get_value (s, GST_QUARK (FORMAT)))' failed
TVMR: FrameRate = 24 
TVMR: NVDEC LowCorner Freq = (80000 * 1024) 
---> TVMR: Video-conferencing detected !!!!!!!!!
TVMR: FrameRate = 17.245512 
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
NvMMLiteOpen : Block : BlockType = 260 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 260 
TVMR: cbBeginSequence: 1179: BeginSequence  672x384, bVPR = 0
TVMR: LowCorner Frequency = 100000 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 3, pnvsi->eCodec = 2, codec = 6 
TVMR: cbBeginSequence: 1600: Display Resolution : (672x384) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (672x384) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 0, TransferCharacteristics = 0, MatrixCoefficients = 0
Allocating new output: 672x384 (x 10), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 672, nFrameHeight = 384 
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
NvMMLiteOpen : Block : BlockType = 260 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 260 
TVMR: cbBeginSequence: 1179: BeginSequence  672x384, bVPR = 0
TVMR: LowCorner Frequency = 100000 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 3, pnvsi->eCodec = 2, codec = 6 
TVMR: cbBeginSequence: 1600: Display Resolution : (672x384) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (672x384) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 0, TransferCharacteristics = 0, MatrixCoefficients = 0
Allocating new output: 672x384 (x 10), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 672, nFrameHeight = 384 
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
NvMMLiteOpen : Block : BlockType = 260 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 260 
TVMR: cbBeginSequence: 1179: BeginSequence  672x384, bVPR = 0
TVMR: LowCorner Frequency = 100000 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 3, pnvsi->eCodec = 2, codec = 6 
TVMR: cbBeginSequence: 1600: Display Resolution : (672x384) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (672x384) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 0, TransferCharacteristics = 0, MatrixCoefficients = 0
Allocating new output: 672x384 (x 10), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 672, nFrameHeight = 384 
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
Seek to frame '20' is not supported. SKIP.
Seek to frame '40' is not supported. SKIP.
Seek to frame '60' is not supported. SKIP.
Seek to frame '80' is not supported. SKIP.
Seek to frame '100' is not supported. SKIP.
Seek to frame '120' is not supported. SKIP.
NvMMLiteOpen : Block : BlockType = 260 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 260 
TVMR: cbBeginSequence: 1179: BeginSequence  672x384, bVPR = 0
TVMR: LowCorner Frequency = 100000 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 3, pnvsi->eCodec = 2, codec = 6 
TVMR: cbBeginSequence: 1600: Display Resolution : (672x384) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (672x384) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1683 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 0, TransferCharacteristics = 0, MatrixCoefficients = 0
Allocating new output: 672x384 (x 10), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 672, nFrameHeight = 384 
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
Seek to frame '44' is not supported. SKIP.
Seek to frame '65' is not supported. SKIP.
Seek to frame '48' is not supported. SKIP.
Seek to frame '26' is not supported. SKIP.
Seek to frame '34' is not supported. SKIP.
Seek to frame '82' is not supported. SKIP.
Seek to frame '91' is not supported. SKIP.
Seek to frame '84' is not supported. SKIP.
Seek to frame '45' is not supported. SKIP.
Seek to frame '11' is not supported. SKIP.
[       OK ] videoio/Videoio_Bunny.read_position/12 (3872 ms)
............
[ RUN      ] videoio/Videoio_Bunny.read_position/21, where GetParam() = ("mjpg.avi", GSTREAMER)

(opencv_test_videoio:4673): GStreamer-CRITICAL **: gst_query_set_position: assertion 'format == g_value_get_enum (gst_structure_id_get_value (s, GST_QUARK (FORMAT)))' failed
Seek to frame '20' is not supported. SKIP.
Seek to frame '40' is not supported. SKIP.
Seek to frame '60' is not supported. SKIP.
Seek to frame '80' is not supported. SKIP.
Seek to frame '100' is not supported. SKIP.
Seek to frame '120' is not supported. SKIP.
Seek to frame '88' is not supported. SKIP.
Seek to frame '12' is not supported. SKIP.
Seek to frame '34' is not supported. SKIP.
Seek to frame '71' is not supported. SKIP.
Seek to frame '102' is not supported. SKIP.
Seek to frame '39' is not supported. SKIP.
Seek to frame '32' is not supported. SKIP.
Seek to frame '62' is not supported. SKIP.
Seek to frame '1' is not supported. SKIP.
[       OK ] videoio/Videoio_Bunny.read_position/21 (5241 ms)
............
[ RUN      ] videoio/Videoio_Bunny.frame_count/0, where GetParam() = ("wmv", GSTREAMER)

(opencv_test_videoio:4673): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
SKIP test: backend GSTREAMER can’t open the video: /home/nvidia/Documents/OpenCV-extra/testdata/highgui/video/big_buck_bunny.wmv
[       OK ] videoio/Videoio_Bunny.frame_count/0 (7 ms)
............
and so on.

Summary for the test report log, currently, gstreamer is still not capable for mpg, some avi, and wmv. But for rtsp with h264/h265 encoding, it should work well, as the report shows. Actually, it does work well when using OpenCV with the tegra_camera.py.

In conclusion for all, any required encoder and decoder for gstreamer, or even other plugins, should be installed before OpenCV compilation, post installation seems to be useless.