IP camera streaming on Jetson TX2

Jetson TX2
Jetpack 3.1
Opencv 3.3.0 Installation link: https://github.com/jetsonhacks/buildOpenCVTX2
Python 2.7.12

I have uninstalled OpenCV4Tegra before installing OpenCV 3.3.0

i’m using the following code::

import numpy as np
import cv2

uri=  "uridecodebin uri=rtsp://<ip_address>/onvif1 ! videoconvert ! videoscale ! appsink"
cap = cv2.VideoCapture(uri)
while(True):
    try:
	    ret, frame = cap.read()
	    cv2.imshow('frame',frame)
	    if cv2.waitKey(1) & 0xFF == ord('q'):
		break
    except:
	    print "Unable to open"

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

This code works fine most of the times.

Build Information

GStreamer:                      
      base:                      YES (ver 1.8.3)
      video:                     YES (ver 1.8.3)
      app:                       YES (ver 1.8.3)
      riff:                      YES (ver 1.8.3)
      pbutils:                   YES (ver 1.8.3)\n

But it gives following errors sometime :

OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file __/opencv/modules/videoio/src/cap_gstreamer.cpp, line 881
VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:

__/opencv/modules/videoio/src/cap_gstreamer.cpp:881: error: (-2) GStreamer: unable to start pipeline
 in function cvCaptureFromCAM_GStreamer

Error opening stream :  Error: Video file could not be opened (opencv reqs. non relative paths)
    at Error (native)
    at startLiveStreaming (/home/nvidia/aggregatorOpenCV/jetson-device-client/aggregatorjetson.js:346:16)
    at /home/nvidia/aggregatorOpenCV/jetson-device-client/aggregatorjetson.js:143:21
    at /home/nvidia/aggregatorOpenCV/jetson-device-client/aggregatorjetson.js:494:17
    at /home/nvidia/aggregatorOpenCV/jetson-device-client/node_modules/mkdirp/index.js:30:20
    at FSReqWrap.oncomplete (fs.js:123:15)

There are some more errors regarding gstreamer pipeline.
Can I know the solution?

Could you try to use

rtspsrc location= rtsp://255.255.255.255/test name=r latency=0 ! rtph264depay ! h264parse ! decodebin ! appsink

Thanks for the reply.
Yes. I tried this.
It’s also giving the same error.
Infact it never opens the stream.

Could you try these pipelines by using gst-launch-1.0 and see if any error first?

I used gst-launch-1.0 for both. Tt gives the same following errors for both

gst-launch-1.0 uridecodebin uri=rtsp://<ip_address>/etc ! videoconvert ! videoscale ! appsink

It gives the following same errors

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://<ip_address>/etc
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source: Could not open resource for reading.
Additional debug info:
gstrtspsrc.c(5384): gst_rtspsrc_setup_auth (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source:
No supported authentication protocol was found
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

And the other one

gst-launch-1.0 rtspsrc location=rtsp://<ip_address>/etc name=r latency=0 ! rtph264depay ! h264parse ! decodebin ! appsink

Its giving following error:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://<ip_address>/etc
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Could not open resource for reading.
Additional debug info:
gstrtspsrc.c(5384): gst_rtspsrc_setup_auth (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
No supported authentication protocol was found
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Could you try to use fakesink first?

gst-launch-1.0 rtspsrc location=rtsp://<ip_address>/etc name=r latency=0 ! rtph264depay ! h264parse ! decodebin ! fakesink

The error “No supported authentication protocol was found”. Are you sure your rtsp server is working? Could you use vlc or any client app to do streaming first?

I tried to do so.

I have two types of rtsp.
First I’m getting from DVR of the form

rtsp://user:password@<ip>:<port>/cam/realmonitor?channel=11&subtype=0

This URL gives that authentication error by using gst-launch-1.0
But this works fine with OpenCV 3.3.0 and gstreamer 1.8.3 on Jetson

And the other one from IP camera Sricam SP007 of the format

rtsp://<ip_address>/onvif1

Gives all the errors of OpenCV 3.3.0 mentioned except the authentication one.

  1. Using gst-launch-1.0 it gives following output for the command::
gst-launch-1.0 rtspsrc location="rtsp://<ip_address>/onvif1" name=r latency=0 ! rtph264depay ! h264parse ! decodebin ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://<ip_address>/onvif1
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7907: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1223: BeginSequence  1280x720, bVPR = 0
TVMR: LowCorner Frequency = 100000 
TVMR: cbBeginSequence: 1622: DecodeBuffers = 2, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1693: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 1694: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1762: ColorFormat : 5 
TVMR: cbBeginSequence:1776 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1904: SurfaceLayout = 3
TVMR: cbBeginSequence: 2005: NumOfSurfaces = 9, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 2007: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 1280x720 (x 9), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 1280, nFrameHeight = 720 
TVMR: FrameRate = 15 
TVMR: NVDEC LowCorner Freq = (50000 * 1024) 
---> TVMR: Video-conferencing detected !!!!!!!!!
TVMR: FrameRate = 14.995831
  1. And using OpenCV the error mentioned and the start.

  2. Using ffplay it works fine. Opens the stream. using ffmpeg also it works fine.

Just OpenCV isnt able to do so sometimes. I want reliable streaming from the IP camera.

Can I know if streaming with rtsp+onvif1 need some other plugins or some command line arguments?

Hi pharatekomal,

There are still something to clarify.

This URL gives that authentication error by using gst-launch-1.0
But this works fine with OpenCV 3.3.0 and gstreamer 1.8.3 on Jetson

What is the difference here between gst-launch-1.0 and gstreamer1.8.3? They are supposed to be the same thing. Do you mean some rtsp plugin from gstreamer 1.8.3?

OpenCV 3.3 works fine under this camera, right?

For second camera, looks like gstreamer can work. Could you try it with gst-launch-1.0 rtspsrc location=“rtsp://<ip_address>/onvif1” name=r latency=0 ! rtph264depay ! h264parse ! decodebin ! nvoverlaysink on Tegra to see if preview is shown?

This thread has successful example, could you take a look?
https://devtalk.nvidia.com/default/topic/1007962/that-proceesing-to-open-ip-camera-with-gstreamer-and-opencv-only-display-a-still-picture-how-to-solve-it-/

This URL gives that authentication error by using gst-launch-1.0
    But this works fine with OpenCV 3.3.0 and gstreamer 1.8.3 on Jetson

I meant OpenCV3.3.0 which is using gstreamer 1.8.3 internally is giving the issue
While using gst-launch-1.0 command line tool, streaming works fine.


Actually I’m able to stream IP camera. But like 3 out of 10 times it fails giving the gstreamer OpenCV issue mentioned at very first.
Using gst-launch and ffmpeg or ffplay these URLs works fine.
The issue is with OpenCV streaming Videocapture.

OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
    ) in cvCaptureFromCAM_GStreamer, file __/opencv/modules/videoio/src/cap_gstreamer.cpp, line 881
    VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:

    __/opencv/modules/videoio/src/cap_gstreamer.cpp:881: error: (-2) GStreamer: unable to start pipeline
     in function cvCaptureFromCAM_GStreamer

    Error opening stream :  Error: Video file could not be opened (opencv reqs. non relative paths)
        at Error (native)
        at startLiveStreaming (/home/nvidia/aggregatorOpenCV/jetson-device-client/aggregatorjetson.js:346:16)
        at /home/nvidia/aggregatorOpenCV/jetson-device-client/aggregatorjetson.js:143:21
        at /home/nvidia/aggregatorOpenCV/jetson-device-client/aggregatorjetson.js:494:17
        at /home/nvidia/aggregatorOpenCV/jetson-device-client/node_modules/mkdirp/index.js:30:20
        at FSReqWrap.oncomplete (fs.js:123:15)

Like if it’s works with gst-launch anf ffmpeg consistently, then why it isnt consistent with OpenCV.

And if it gets opened
Also while doing

cap.release()

It sometimes gives following error

(node:6523): GStreamer-CRITICAL **: 
Trying to dispose element videoconvert0, but it is in READY instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(node:6523): GStreamer-CRITICAL **: 
Trying to dispose element pipeline0, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(node:6523): GStreamer-CRITICAL **: 
Trying to dispose element videoscale0, but it is in READY instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.


(node:6523): GStreamer-CRITICAL **: 
Trying to dispose element pipeline0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

I tried to stream using suggested URL
1.
https://devtalk.nvidia.com/default/topic/1007962/that-proceesing-to-open-ip-camera-with-gstreamer-and-opencv-only-display-a-still-picture-how-to-solve-it-/

But its not able to open the stream
and gives some errors.

  1. The option mentioned by you.
gst-launch-1.0 rtspsrc location="rtsp://<ip_address>/onvif1" name=r latency=0 ! rtph264depay ! h264parse ! decodebin ! nvoverlaysink

Gives this output

WARNING: erroneous pipeline: no element "nvoverlaysink"

While
I tried ::

gst-launch-1.0 rtspsrc location=\"rtsp://<ip_adrees>/onvif1\" ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! xvimagesink

And

uri="rtspsrc location=\"rtsp://<ip_adrees>/onvif1\" ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! xvimagesink"
cam=cv2.VideoCapture(uri)
#do something
cam.release()

Works fine for me maximum times.

Just need to solve issues mentioned while opening and releasibg the stream(3 out of 10 times).

I hope I’m not confusing.
Thanks

Could you try to check cap status before read frame?

cap.isOpened()

Yes I do check.
The error occurs at VideoCapture only.

Is this issue resolved?

No.It’s not solved yet.

You may check https://devtalk.nvidia.com/default/topic/1030866/jetson-tx2/opencv-problem-with-capturing-rtsp-stream-using-gstreamer-on-tx2/post/5244649/#5244649