Gstreamer fail on Jetson AGX - Internal data error,

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
,
1

Hello all,
I’m using Jetson AGX Xavier and custom made driver camera which works fine with

gst-launch-1.0 nvarguscamerasrc sensor-id=1 sensor-mode=0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080’ ! nvvidconv flip-method=0 ! ‘video/x-raw, format=(string)I420’ ! xvimagesink

But, I want to use it in opencv with BGR format, so I’m trying to convert it in BGR format(if there is a way)
So my test pattern was like below, which says internal data error unfortunately.
But it seems nvarguscamerasrc is fine with pads, at least in debug output.

$ gst-launch-1.0 -vvvvv nvarguscamerasrc sensor-id=1 sensor-mode=0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12’ ! nvvidconv ! videoconvert ! ‘video/x-raw, width=(int)1920, height=(int)1080, format=(string)BGRA’ ! autovideosink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)NV12
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)BGRA
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)BGRA
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)BGRA
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)BGRA
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)BGRA
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)NV12
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 10000000000, max 20000000000000;
GST_ARGUS: 3856 x 2176 FR = 40.000000 fps Duration = 25000000 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 10000000000, max 20000000000000;
GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 3840 H = 2160
seconds to Run = 0
Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.663027346
Setting pipeline to PAUSED …
Setting pipeline to READY …
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Setting pipeline to NULL …
Freeing pipeline …

Is there any way to convert it in BGR?
Finally, I want to use it as

cv::VideoCapture cap("nvarguscamerasrc sensor-id=1 sensor-mode=0 ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! appsink ")

which is end with appsink.
Thank you.

3

Hi,
You may try this sample:
OpenCV Video Capture with GStreamer doesn't work on ROS-melodic - #3 by DaneLLL

4

Hi DaneLLL,
Thanks for the reply.
and don’t forget about this one below

Since I’m working on c++, so I just copied the inside of initialize.

“nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink”

The answer was error, and here’s the output below

$ ./test_app
Gtk-Message: 18:00:51.546: Failed to load module “canberra-gtk-module”
[ WARN:0] OpenCV | GStreamer warning: Error opening bin: no element “nvcamerasrc”
[ WARN:0] OpenCV | GStreamer warning: GStreamer: pipeline have not been created
VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels
[ERROR:0] VIDEOIO(cvCreateFileCapture_Images(filename.c_str())): raised OpenCV exception:
OpenCV(3.4.12) /data/work/NVIDIA-OpenSource/opencv/opencv-3.4.12/modules/videoio/src/cap_images.cpp:246: error: (-5:Bad argument) CAP_IMAGES: can’t find starting number (in the name of file): nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink in function ‘icvExtractPattern’

Looks like I don’t have nvcamerasrc.(Confirmed with gst-inspect-1.0 nvcamerasrc)

5

Hi,
You should use nvarguscamerasrc. The reference pipeline is

    cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert !  appsink")
6

When I tried this, it says nothing for a while in this state, so I pressed Ctrl+C.

Gtk-Message: 20:13:04.185: Failed to load module “canberra-gtk-module”
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3840 x 1918 FR = 40.000000 fps Duration = 25000000 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 100000000000, max 20000000000000;
GST_ARGUS: 3840 x 1920 FR = 40.000000 fps Duration = 25000000 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 10000000000, max 20000000000000;
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 1
Output Stream W = 3840 H = 1920
seconds to Run = 0
Frame Rate = 40.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
^C

After that, I can’t use my camera until I reboot, and no output when I try the very first one,

$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 sensor-mode=0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080’ ! nvvidconv flip-method=0 ! ‘video/x-raw, format=(string)I420’ ! xvimagesink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
(Argus) Error Timeout: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 215)
(Argus) Error Timeout: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:551 Failed to create CameraProvider
Got EOS from element “pipeline0”.
Execution ended after 0:01:00.009373407
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

7

Argus may break if you don’t release your opencv VideoCapture using nvarguscamerasrc properly. You may then restart nvargus-daemon service, or wait for one minute.
The workaround is to set a signal handler in your app that will release the VideoCapture when interrupted, such as here.

8

Hello Honey_Patouceul,
Thanks for your advise.
This time, I got the error message like this.

$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 sensor-mode=0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080’ ! nvvidconv flip-method=0 ! ‘video/x-raw, format=(string)I420’ ! xvimagesink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: Connection refused (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 201)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:551 Failed to create CameraProvider
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.002486971
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Is it a same reason?
I was working in docker when I got this error, but the camera worked outside of docker with same line.

9

Hi,
For running in docker, please refer to
Jetpack 4.3 CSI Camera Support from within Containers - #9 by dusty_nv

10

Hi DaneLLL,
Thanks for the advise.
I’m already running docker with –ipc=host and -v /tmp/argus_socket:/tmp/argus_socket.
I’ve found that I had to modify from you mentioned, then mine is working.

cv::VideoCapture cap(“nvarguscamerasrc sensor-id=1 sensor-mode=0 ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! appsink”)

This worked!