Problem with nvcompositor + nvarguscamerasrc

andrey16 · August 18, 2021, 10:21am

Good afternoon. I have a Jetson AGS Xavier and 2 Leopard Imaging LI-IMX390 cameras. the task is to get an image from two cameras simultaneously using gstreamer.

I use the following command

gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920
! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420’, width=3840, height=1080 ! nv3dsink
nvarguscamerasrc sensor_id=1 ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1’
! nvvidconv flip-method=2 ! nvcomp.sink_0
nvarguscamerasrc sensor_id=0 ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1’
! nvvidconv flip-method=2 ! nvcomp.sink_1

But it gives an error

gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 ! nvvidconv ! “video/x-raw(memory:NVMM), format=(string)I420”, width=3840, height=1080 ! nv3dsink nvarguscamerasrc sensor_id=1 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1” ! nvvidconv flip-method=2 ! nvcomp.sink_0 nvarguscamerasrc sensor_id=0 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1” ! nvvidconv flip-method=2 ! nvcomp.sink_1
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1: TIMEOUT
Additional debug info:
Argus Error Status
Execution ended after 0:00:02.923376928
Setting pipeline to PAUSED …
Setting pipeline to READY …

(gst-launch-1.0:10234): GStreamer-CRITICAL **: 12:50:18.644: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
CONSUMER: Done Success
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success

(gst-launch-1.0:10234): GStreamer-CRITICAL **: 12:50:19.089: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
GST_ARGUS: Cleaning up
Setting pipeline to NULL …
Freeing pipeline …

What could be the problem?

I run one camera at a time normally and they work
For single camera? I use:

gst-launch-1.0 nvarguscamerasrc tnr-mode=0 sensor-id=0 ! “video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=30/1” ! nvvidconv flip-method=0 ! “video/x-raw, format=(string)I420” ! xvimagesink -e

Honey_Patouceul · August 18, 2021, 10:37am

There may be 2 problems:

Wrong quoting in caps before nv3dsink
I420 as input of nvcompositor. Try RGBA instead:

gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420, width=3840, height=1080' ! nv3dsink \
     nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1' ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvcomp.sink_1   \
     nvarguscamerasrc sensor_id=1 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1' ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvcomp.sink_0

andrey16 · August 18, 2021, 11:00am

I use command:

gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 ! nvvidconv ! "video/x-raw(memory:NVMM), format=RGBA, width=3840, height=1080" ! nv3dsink nvarguscamerasrc sensor_id=0 ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1" ! nvvidconv flip-method=2 ! "video/x-raw(memory:NVMM),format=RGBA" ! nvcomp.sink_1 nvarguscamerasrc sensor_id=1 ! "video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1" ! nvvidconv flip-method=2 ! "video/x-raw(memory:NVMM),format=RGBA" ! nvcomp.sink_0

And result:

jetson@jetson:~$ gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 ! nvvidconv ! “video/x-raw(memory:NVMM), format=RGBA, width=3840, height=1080” ! nv3dsink nvarguscamerasrc sensor_id=0 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1” ! nvvidconv flip-method=2 ! “video/x-raw(memory:NVMM),format=RGBA” ! nvcomp.sink_1 nvarguscamerasrc sensor_id=1 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1” ! nvvidconv flip-method=2 ! “video/x-raw(memory:NVMM),format=RGBA” ! nvcomp.sink_0
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:656 Failed to create CaptureSession
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:656 Failed to create CaptureSession

(gst-launch-1.0:11950): GStreamer-CRITICAL **: 13:56:30.101: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed

(gst-launch-1.0:11950): GStreamer-CRITICAL **: 13:56:30.101: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
Redistribute latency…
Caught SIGSEGV
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.011740192
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
#0 0x0000007fb2894400 in syscall ()
#1 0x0000007fb29e4b84 in g_cond_wait ()
#2 0x0000007fb16d4634 in ()
#3 0x0000007fb16d6054 in ()
#4 0x0000007fb2a73008 in g_object_unref ()
#5 0x0000007fb16dbb34 in ()
#6 0x0000007fb21b776c in () at /usr/lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#7 0x0000000000000001 in ()
Spinning. Please run ‘gdb gst-launch-1.0 11950’ to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

andrey16 · August 18, 2021, 11:09am

I restarted Jetson and repeated the command.
Here is the result

gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 ! nvvidconv ! “video/x-raw(memory:NVMM), format=I420, width=3840, height=1080” ! nv3dsink nvarguscamerasrc sensor_id=0 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1” ! nvvidconv flip-method=2 ! “video/x-raw(memory:NVMM),format=I420” ! nvcomp.sink_1 nvarguscamerasrc sensor_id=1 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1” ! nvvidconv flip-method=2 ! “video/x-raw(memory:NVMM),format=I420” ! nvcomp.sink_0
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1: TIMEOUT
Additional debug info:
Argus Error Status
Execution ended after 0:00:02.874837600
Setting pipeline to PAUSED …
Setting pipeline to READY …
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…

(gst-launch-1.0:9132): GStreamer-CRITICAL **: 14:07:30.997: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
CONSUMER: Done Success
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success

(gst-launch-1.0:9132): GStreamer-CRITICAL **: 14:07:31.405: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
GST_ARGUS: Cleaning up
Setting pipeline to NULL …
Freeing pipeline …

Honey_Patouceul · August 18, 2021, 11:14am

The command you’ve launched after reboot still uses I420 as nvcompositor inputs.
Can you reboot and launch exactly my command ?

andrey16 · August 18, 2021, 11:20am

Honey_Patouceul:

gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420, width=3840, height=1080' ! nv3dsink \
     nvarguscamerasrc sensor_id=0 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1' ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvcomp.sink_1   \
     nvarguscamerasrc sensor_id=1 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1' ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvcomp.sink_0

I restarted Jetson and executed your command

jetson@jetson:~$ gst-launch-1.0 nvcompositor name=nvcomp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=I420, width=3840, height=1080’ ! nv3dsink nvarguscamerasrc sensor_id=0 ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1’ ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM),format=RGBA’ ! nvcomp.sink_1 nvarguscamerasrc sensor_id=1 ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1’ ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM),format=RGBA’ ! nvcomp.sink_0
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1: CANCELLED
Additional debug info:
Argus Error Status
Execution ended after 0:00:03.539535424
Setting pipeline to PAUSED …
Setting pipeline to READY …

(gst-launch-1.0:8523): GStreamer-CRITICAL **: 14:17:59.887: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed

(gst-launch-1.0:8523): GStreamer-CRITICAL **: 14:18:00.886: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
CONSUMER: Done Success
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success
GST_ARGUS: Cleaning up
Setting pipeline to NULL …
Freeing pipeline …

ShaneCCC · August 19, 2021, 4:15am

Have a try below command.

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp. -e nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1" ! queue ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp. -e

andrey16 · August 19, 2021, 7:36am

I tried your command.

Result:

jetson@jetson:~$ gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! “video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1” ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! “video/x-raw(memory:NVMM), format=(string)RGBA” ! comp. -e nvarguscamerasrc sensor-id=1 ! “video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1” ! queue ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! “video/x-raw(memory:NVMM), format=(string)RGBA” ! comp. -e
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1: TIMEOUT
Additional debug info:
Argus Error Status
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…

(gst-launch-1.0:12798): GStreamer-CRITICAL **: 10:34:42.727: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed

andrey16 · August 19, 2021, 7:52am

Your command worked, but it only gives a picture from one camera, and the other half is just a black square

jetson@jetson:~$ gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! “video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1” ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! “video/x-raw(memory:NVMM), format=(string)RGBA” ! comp. -e nvarguscamerasrc sensor-id=1 ! “video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1” ! queue ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! “video/x-raw(memory:NVMM), format=(string)RGBA” ! comp. -e
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:649 Invalid camera device specified 1 specified, 0 max index

(gst-launch-1.0:9318): GStreamer-CRITICAL **: 10:50:27.482: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Redistribute latency…

andrey16 · August 19, 2021, 4:01pm

ShaneCCC:

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp. -e nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1" ! queue ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp. -e

jetson@jetson:~$ gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! “video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1” ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! “video/x-raw(memory:NVMM), format=(string)RGBA” ! comp. -e nvarguscamerasrc sensor-id=1 ! “video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1” ! queue ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! “video/x-raw(memory:NVMM), format=(string)RGBA” ! comp. -e
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

CONSUMER: Waiting until producer is connected…
GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1: TIMEOUT
Additional debug info:
Argus Error Status
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…

(gst-launch-1.0:9241): GStreamer-CRITICAL **: 18:59:58.932: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed

Honey_Patouceul · August 19, 2021, 5:22pm

My guess would be that both are sent into input0 of compositor. You may add sink number property:

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp.sink_1 -e      nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1" ! queue ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp.sink_0 -e

ShaneCCC · August 19, 2021, 5:53pm

Also have a check below command.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_0  nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_1  nvcompositor name=comp sink_0::alpha=0.5 sink_1::alpha=0.5 ! nvoverlaysink sync=false -e

andrey16 · August 20, 2021, 6:12am

I tried it. The output is black on the screen, that is, there is no picture. and this is what he writes to the terminal

jetson@jetson:~$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080’ ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=RGBA’ ! comp.sink_0 nvarguscamerasrc sensor-id=1 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080’ ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=RGBA’ ! comp.sink_1 nvcompositor name=comp sink_0::alpha=0.5 sink_1::alpha=0.5 ! nvoverlaysink sync=false -e
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: TIMEOUT
Additional debug info:
Argus Error Status
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…

(gst-launch-1.0:10896): GStreamer-CRITICAL **: 09:09:37.795: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
Redistribute latency…

andrey16 · August 20, 2021, 6:16am

Honey_Patouceul:

My guess would be that both are sent into input0 of compositor. You may add sink number property:

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp.sink_1 -e      nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1" ! queue ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! comp.sink_0 -e

jetson@jetson:~$ gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 ! nvoverlaysink nvarguscamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1’ ! queue ! nvvidconv ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! ‘video/x-raw(memory:NVMM), format=(string)RGBA’ ! comp.sink_1 -e nvarguscamerasrc sensor-id=1 ! “video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080,framerate=(fraction)30/1” ! queue ! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so” ! ‘video/x-raw(memory:NVMM), format=(string)RGBA’ ! comp.sink_0 -e
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1: TIMEOUT
Additional debug info:
Argus Error Status
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…

(gst-launch-1.0:11056): GStreamer-CRITICAL **: 09:15:36.793: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed

Honey_Patouceul · August 20, 2021, 6:23pm

Does both cameras work independently and concurrently ? (Note this assumes your local monitor can do 1080p)

gst-launch-1.0 -v nvarguscamerasrc sensor_id=0 num-buffers=150 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! nvoverlaysink

gst-launch-1.0 -v nvarguscamerasrc sensor_id=1 num-buffers=150 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! nvoverlaysink

gst-launch-1.0 -v nvarguscamerasrc sensor_id=0 num-buffers=150 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! nvoverlaysink overlay=1    nvarguscamerasrc sensor_id=1 num-buffers=150 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! nvoverlaysink overlay=2

The last command may only show second camera overlaying the first one, but this is just for checking if both nvarguscamersrc work before using nvcompositor.

andrey16 · August 23, 2021, 5:49am

The first and second command are working. One by one, the cameras are launched and give a picture.

The third command does not work and outputs this log:

jetson@jetson:~$ gst-launch-1.0 -v nvarguscamerasrc sensor_id=0 num-buffers=150 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1” ! nvvidconv flip-method=2 ! nvoverlaysink overlay=1 nvarguscamerasrc sensor_id=1 num-buffers=150 ! “video/x-raw(memory:NVMM), width=1920, height=1080, format=(string)NV12, framerate=(fraction)30/1” ! nvvidconv flip-method=2 ! nvoverlaysink overlay=2
nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 31,622776; Exposure Range min 59000, max 33333000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29,999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD… Exiting…
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc1: TIMEOUT
Additional debug info:
Argus Error Status
Execution ended after 0:00:02.777882912
Setting pipeline to PAUSED …
Setting pipeline to READY …

(gst-launch-1.0:9817): GStreamer-CRITICAL **: 08:44:41.916: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success
Setting pipeline to NULL …
Freeing pipeline …

The monitor supports a resolution of 1920 by 1080

ShaneCCC · August 23, 2021, 5:53am

What’s your BSP version?

andrey16 · August 23, 2021, 6:03am

JetPack 4.5
L4T 32.5

R32 (release), REVISION: 5.0 (/etc/nv_tegra_release)
CTI version AGX-32.5-V004 (/etc/cti/CTI-L4T.version)

ShaneCCC · August 23, 2021, 6:04am

Have try this patch

andrey16 · August 23, 2021, 6:18am

thank you it helped. 2 cameras are displayed on the screen, but the image is superimposed on one another.