Gstreamer warnings with two camera operation

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hi Folks,

Our Tx1 board has two CSI cameras. They both are working with following code. When I run, I get few gstreamer warnings - which I am unable to fully understand. Could someone please guide about what is wrong in my Gstreamer pipeline setup which is leading to these warnings ?

Thanks,

My Code

VideoCapture cap1("nvcamerasrc sensor-id=0 fpsRange='30.0 30.0' ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

VideoCapture cap2("nvcamerasrc sensor-id=1 fpsRange='30.0 30.0' ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");


    if(!cap1.isOpened()) { // check if we succeeded
       cerr << "Fail to open camera " << endl;
       return -1;
    }
    for(;;)
    {
       Mat frame;
       cap1 >> frame; 
       imshow("cam1", frame);
       cap2 >> frame; 
       imshow("cam2", frame);
       waitKey(1);
    }
    // the camera will be deinitialized automatically in VideoCapture destructor
    cap1.release();
    cap2.release();

The Warnings

Available Sensor modes :
3840 x 2160 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1920 x 1080 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected 1280x720 FrameRate = 60.000000 …

(cv_hello:18934): GStreamer-CRITICAL **: gst_element_get_static_pad: assertion ‘GST_IS_ELEMENT (element)’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_pad_get_current_caps: assertion ‘GST_IS_PAD (pad)’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_structure_get_int: assertion ‘structure != NULL’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_structure_get_int: assertion ‘structure != NULL’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_structure_get_fraction: assertion ‘structure != NULL’ failed

Available Sensor modes :
3840 x 2160 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1920 x 1080 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected 1280x720 FrameRate = 60.000000 …

(cv_hello:18934): GStreamer-CRITICAL **: gst_element_get_static_pad: assertion ‘GST_IS_ELEMENT (element)’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_pad_get_current_caps: assertion ‘GST_IS_PAD (pad)’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_structure_get_int: assertion ‘structure != NULL’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_structure_get_int: assertion ‘structure != NULL’ failed

(cv_hello:18934): GStreamer-CRITICAL **: gst_structure_get_fraction: assertion ‘structure != NULL’ failed

Available Sensor modes :
3840 x 2160 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1920 x 1080 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected 1280x720 FrameRate = 60.000000 …

Available Sensor modes :
3840 x 2160 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1920 x 1080 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10

NvCameraSrc: Trying To Set Default Camera Resolution. Selected 1280x720 FrameRate = 60.000000 …

2

Hi,

Thanks for your question.

Could you check if all gstreamer plugins are well installed first?

sudo add-apt-repository universe
sudo add-apt-repository multiverse
sudo apt-get update
sudo apt-get install gstreamer1.0-tools gstreamer1.0-alsa gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav
sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-dev libgstreamer-plugins-bad1.0-dev   // for development
3

Hi AastaLLL,

It seems like I was not up-to-date. After your update instructions, I have few more things for fix - I just do not know where and what to fix ?

THanks

Socket read error. Camera Daemon stopped functioning…
gst_nvcamera_open() failed ret=0
GStreamer Plugin: Embedded video playback halted; module nvcamerasrc0 reported: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/ubuntu/opencv/modules/videoio/src/cap_gstreamer.cpp, line 818
terminate called after throwing an instance of ‘cv::Exception’
what(): /home/ubuntu/opencv/modules/videoio/src/cap_gstreamer.cpp:818: error: (-2) GStreamer: unable to start pipeline
in function cvCaptureFromCAM_GStreamer

4

I think, I need to reinstall everything. This is the output of nvcamerasrc.

gst-launch-1.0 nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvtee ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvoverlaysink -e
Setting pipeline to PAUSED …
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingSocket read error. Camera Daemon stopped functioning…
gst_nvcamera_open() failed ret=0
ERROR: Pipeline doesn’t want to pause.
ERROR: from element /GstPipeline:pipeline0/GstNvCameraSrc:nvcamerasrc0: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
Additional debug info:
gstbasesrc.c(3354): gst_base_src_start (): /GstPipeline:pipeline0/GstNvCameraSrc:nvcamerasrc0:
Failed to start
Setting pipeline to NULL …
Freeing pipeline …

5

Hi,

Thanks for your feedback.

Could you restart the device and run the pipeline again?
Thanks and please let us know the results.

6

Hi AastaLLL,
Yes after rebooting the device, I see the camera output again. But I see the same warnings too.
Thanks,

7

Hi,

Thanks for your feedback.

Could you open camera via gst-launch directly.
This helps us clarify the warning is from gstreamer+opencv or gstreamer itself.

Thanks.

8

It seems warning are not coming from gst command line. They seem to come when Gstreamer is used frm openCV

Thanks for such a quick help.

Thanks,

gst-launch-1.0 nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvtee ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvoverlaysink -e
Setting pipeline to PAUSED …
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and Mjstreaming
Available Sensor modes :
3840 x 2160 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1920 x 1080 FR=60.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock

NvCameraSrc: Trying To Set Default Camera Resolution. Selected 1920x1080 FrameRate = 30.000000 …

^Chandling interrupt.
Interrupt: Stopping pipeline …
EOS on shutdown enabled – Forcing EOS on the pipeline
Waiting for EOS…
Got EOS from element “pipeline0”.
EOS received - stopping pipeline…
Execution ended after 0:00:03.780956785
Setting pipeline to PAUSED …
Setting pipeline to READY …

9

I was told that our Leopard cameras only support 1080p and 4K resolutions.
Thanks,

10

Hi AastaLLL,
Any idea why we are getting warnings from OpenCV (when we are using the resolutions that driver supports ) and not from gst-launch command line ?
Thanks,

11

Hi,

Sorry for my late reply. I thought you already found the root cause in comment #9.

From comment #9,
Camera only support 1080p and 4K but your code launch gstreamer with 720p.

VideoCapture cap2("nvcamerasrc sensor-id=1 fpsRange='30.0 30.0' ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

Could you check if this cause the error?

Thanks and please let us know the results.

12

Hi AastaLLL

Sorry about the confusion. The camera supports only 1080p and 4k resolutions. Following code is giving warnings.

int main ()
{

  //open the default camera

    VideoCapture cap1("nvcamerasrc sensor-id=0 fpsRange='30.0 30.0' ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");


    if(!cap1.isOpened()) { // check if we succeeded
       cerr << "Fail to open camera " << endl;
       return -1;
    }
    for(;;)
    {
       Mat frame;
       cap1 >> frame; 
       imshow("sobel output", frame);

       waitKey(1);
    }
    // the camera will be deinitialized automatically in VideoCapture destructor
    cap1.release();
    
}

And this gst command line does not give warnings.

gst-launch-1.0 nvcamerasrc fpsRange=“30.0 30.0” sensor-id=0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)60/1’ ! nvtee ! nvvidconv flip-method=0 ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvoverlaysink -e

Thanks,

13

Hi,

Thanks for your feedback and patience.

I have tried your code with my dual CSI camera. It works properly.
Guess that this error is caused by opecv since your gst-launch works well.

Which opencv branch did you used?
Usually, we install opencv3.2.0 with ‘-D WITH_GSTREAMER=ON’ configuration.

Could you check if your opencv is well enabled with gstreamer1.0?
In cmake status:

--   Video I/O:
......
--     GStreamer:                   
--       base:                      YES (ver 1.8.3)
--       video:                     YES (ver 1.8.3)
--       app:                       YES (ver 1.8.3)
--       riff:                      YES (ver 1.8.3)
--       pbutils:                   YES (ver 1.8.3)

And may I know which camera do you use?
Thanks and please let us know the results.

14

Hi AastaLLL

I am using OpenCV 3.1.0. Following is my cmake command line to build opencv -

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PNG=OFF -DBUILD_TIFF=OFF -DBUILD_TBB=OFF -DBUILD_JPEG=OFF -DBUILD_JASPER=OFF -DBUILD_ZLIB=OFF -DBUILD_EXAMPLES=ON -DBUILD_opencv_java=OFF -DBUILD_opencv_python2=ON -DBUILD_opencv_python3=OFF -DENABLE_PRECOMPILED_HEADERS=OFF -DWITH_OPENCL=OFF -DWITH_OPENMP=OFF -DWITH_FFMPEG=ON -DWITH_GSTREAMER=ON -DWITH_GSTREAMER_0_10=OFF -DWITH_CUDA=ON -DWITH_GTK=ON -DWITH_VTK=OFF -DWITH_TBB=ON -DWITH_1394=OFF -DWITH_OPENEXR=OFF -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-8.0 -DCUDA_ARCH_BIN=5.3 -DCUDA_ARCH_PTX=“” -DINSTALL_C_EXAMPLES=ON -DINSTALL_TESTS=ON -DOPENCV_TEST_DATA_PATH=…/opencv_extra/testdata …/opencv

In the logs I do see -

– GStreamer:
– base: YES (ver 1.8.3)
– video: YES (ver 1.8.3)
– app: YES (ver 1.8.3)
– riff: YES (ver 1.8.3)
– pbutils: YES (ver 1.8.3)

We are using SOny IMX274M12-D

I am wondering should I reinstall jetpack ?

15

Hi,

For comment #12, this error also shows when opening ONE csi camera, right?

16

Yes that is right. When I open ONE csi camera (whether camera_id = 0 or 1) from openCV I get the warning. And then when I use from gst command line (with camera_id = 0 or 1) I do not get the warning.

Thanks,

17

Hi,

Could you use this command to open error log?

VideoCapture cap1("--gst-debug=3 nvcamerasrc sensor-id=0 fpsRange='30.0 30.0' ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

Thanks.

18

Where would error log be generated ? I am assuming current directory. But when I ran your code line I do not see new file getting generated with --gst-debug = 3?

19

it seems I had to set an environment variable.

export GST_DEBUG=4

0:00:00.000117551 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gst.c:511:init_pre:ESC[00m Initializing GStreamer Core Library version 1.8.3
0:00:00.000247029 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gst.c:512:init_pre:ESC[00m Using library installed in /usr/lib/aarch64-linux-gnu
0:00:00.000276247 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gst.c:523:init_pre:ESC[00m Linux tegra-ubuntu 3.10.96-test #1 SMP PREEMPT Sat Apr 1 04:17:11 PDT 2017 aarch64
0:00:00.000810930 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gstmessage.c:119:_priv_gst_message_initialize:ESC[00m init messages
0:00:00.001680610 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gstcontext.c:83:_priv_gst_context_initialize:ESC[00m init contexts
0:00:00.002015919 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;36m  GST_PLUGIN_LOADING gstplugin.c:316:_priv_gst_plugin_initialize:ESC[00m registering 0 static plugins
0:00:00.002220396 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;36m  GST_PLUGIN_LOADING gstplugin.c:224:gst_plugin_register_static:ESC[00m registered static plugin "staticelements"
0:00:00.002244094 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;36m  GST_PLUGIN_LOADING gstplugin.c:226:gst_plugin_register_static:ESC[00m added static plugin "staticelements", result: 1
0:00:00.002285969 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00m        GST_REGISTRY gstregistry.c:1723:ensure_current_registry:ESC[00m reading registry cache: /home/ubuntu/.cache/gstreamer-1.0/registry.aarch64.bin
0:00:00.033451362 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00m        GST_REGISTRY gstregistrybinary.c:619:priv_gst_registry_binary_read_cache:ESC[00m loaded /home/ubuntu/.cache/gstreamer-1.0/registry.aarch64.bin in 0.031126 seconds
0:00:00.033638183 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00m        GST_REGISTRY gstregistry.c:1579:scan_and_update_registry:ESC[00m Validating plugins from registry cache: /home/ubuntu/.cache/gstreamer-1.0/registry.aarch64.bin
0:00:00.035908267 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00m        GST_REGISTRY gstregistry.c:1681:scan_and_update_registry:ESC[00m Registry cache has not changed
0:00:00.035939777 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00m        GST_REGISTRY gstregistry.c:1758:ensure_current_registry:ESC[00m registry reading and updating done, result = 1
0:00:00.035959672 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gst.c:724:init_post:ESC[00m GLib runtime version: 2.48.1
0:00:00.035978839 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gst.c:726:init_post:ESC[00m GLib headers version: 2.48.1
0:00:00.035995453 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;31m            GST_INIT gst.c:727:init_post:ESC[00m initialized GStreamer successfully
0:00:00.036075140 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;37;41m        GST_PIPELINE gstparse.c:323:gst_parse_launch_full:ESC[00m parsing pipeline description '--gst-debug=3 nvcamerasrc sensor-id=1 fpsRange='60.0 60.0' ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink '
0:00:00.036123838 ESC[334m11697ESC[00m       0x45d400 ESC[33;01mWARN   ESC[00m ESC[00;04m             default grammar.y:1035:priv_gst_parse_yyerror:ESC[00m Error during parsing: syntax error, unexpected $undefined
0:00:00.036153889 ESC[334m11697ESC[00m       0x45d400 ESC[31;01mERROR  ESC[00m ESC[00;01;37;41m        GST_PIPELINE grammar.y:959:priv_gst_parse_yyparse:ESC[00m syntax error
0:00:00.036174983 ESC[334m11697ESC[00m       0x45d400 ESC[31;01mERROR  ESC[00m ESC[00;01;37;41m        GST_PIPELINE grammar.y:959:priv_gst_parse_yyparse:ESC[00m syntax error
0:00:00.036190399 ESC[334m11697ESC[00m       0x45d400 ESC[31;01mERROR  ESC[00m ESC[00;01;37;41m        GST_PIPELINE grammar.y:959:priv_gst_parse_yyparse:ESC[00m syntax error
0:00:00.036227326 ESC[334m11697ESC[00m       0x45d400 ESC[31;01mERROR  ESC[00m ESC[00;01;37;41m        GST_PIPELINE grammar.y:959:priv_gst_parse_yyparse:ESC[00m syntax error
0:00:00.038315693 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;36m  GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry:ESC[00m plugin "/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcamera.so" loaded
0:00:00.038358765 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;37;41m GST_ELEMENT_FACTORY gstelementfactory.c:364:gst_element_factory_create:ESC[00m creating element "nvcamerasrc"
0:00:00.038847771 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00;01;37;41m    GST_ELEMENT_PADS gstelement.c:659:gst_element_add_pad:<GstBaseSrc@0x641a00>ESC[00m adding pad 'src'
0:00:00.038902146 ESC[334m11697ESC[00m       0x45d400 ESC[36mINFO   ESC[00m ESC[00m         nvcamerasrc gstnvcamerasrc.cpp:3408:set_fps_range:<GstNvCameraSrc@0x641a00>ESC[00m NvCameraSrc: Setting FPSRange : (null)

It seems there is a syntax error. Could someone please check my code for GStreamer syntax errors ?

VideoCapture cap1("--gst-debug=3 nvcamerasrc sensor-id=1 fpsRange='60.0 60.0' ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink ");
20

Hi,

Sorry for the late reply.

Could you change to this command and test?

VideoCapture cap1("--gst-debug=3 nvcamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

Please also attach execution log for us debugging. Thanks.