RTSP streaming of two CSI cameras by using Gstreamer

I would like to do RTSP streaming of two CSI cameras by using GStreamer.
I tried the following code;

GST_DEBUG=3 ./test-launch "nvcompositor name=comp sink_1::xpos=1920 ! nvvidconv ! video/x-raw(memory:NVMM) ! nvv4l2h264enc insert-sps-pps=true ! h264parse ! rtph264pay name=pay0 pt=96 nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1 ! queue ! comp. nvarguscamerasrc sensor-id=1 ! video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1 ! queue ! comp."

But it does not work. The console output is the following;

stream ready at rtsp://127.0.0.1:8554/test
Opening in BLOCKING MODE 
0:00:07.051047929 12037   0x7f84056990 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f840744f0 Failed to determine interlace mode
0:00:07.051135066 12037   0x7f84056990 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f840744f0 Failed to determine interlace mode
0:00:07.051177046 12037   0x7f84056990 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f840744f0 Failed to determine interlace mode
0:00:07.051289705 12037   0x7f84056990 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:07.053214701 12037   0x7f7c008f20 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<nvarguscamerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:07.053312360 12037   0x7f7c008f70 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<nvarguscamerasrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
CONSUMER: Waiting until producer is connected...
GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: Running with following settings:
   Camera index = 1 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
CONSUMER: Producer has connected; continuing.
GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
0:00:08.657838301 12037   0x7f7c008e80 WARN          v4l2bufferpool gstv4l2bufferpool.c:1057:gst_v4l2_buffer_pool_start:<nvv4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold
H264: Profile = 66, Level = 0 
0:00:08.761583527 12037   0x7f742505e0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1503:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:08.764393596 12037   0x5582017d90 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:08.781064791 12037   0x5582017d90 FIXME              rtspmedia rtsp-media.c:3841:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:08.781113386 12037   0x5582017d90 WARN               rtspmedia rtsp-media.c:3867:gst_rtsp_media_suspend: media 0x7f840b81d0 was not prepared
0:00:08.817053078 12037   0x7f84056b20 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<appsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:08.818270348 12037   0x7f742505e0 FIXME               basesink gstbasesink.c:3145:gst_base_sink_default_event:<appsink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:08.818597439 12037   0x5582017d90 FIXME             rtspclient rtsp-client.c:1657:handle_play_request:<GstRTSPClient@0x5582130110> Add support for seek style (null)
0:00:08.818734057 12037   0x5582017d90 FIXME              rtspmedia rtsp-media.c:2437:gst_rtsp_media_seek_full:<GstRTSPMedia@0x7f840b81d0> Handle going back to 0 for none live not seekable streams.
0:00:08.819970129 12037   0x7f7c008e80 WARN            nvcompositor gstnvcompositor.c:923:gst_nvcompositor_negotiated_caps:<comp> Release old pool
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:568 Failed to create CaptureSession
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:568 Failed to create CaptureSession
CONSUMER: Done Success
CONSUMER: Done Success
0:00:09.327488013 12037   0x7f742505e0 WARN           v4l2allocator gstv4l2allocator.c:1498:gst_v4l2_allocator_dqbuf:<nvv4l2h264enc0:pool:src:allocator> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success
0:00:10.721821040 12037   0x5582017d90 WARN               rtspmedia rtsp-media.c:4156:gst_rtsp_media_set_state: media 0x7f840b81d0 was not prepared

I use Jetson Nano 4GB with JetPack 4.4.1.

Can anyone help me?
Thanks in advance.

Hi,
Could you try with nvoverlaysink? Would like to know if it works with other sinks.

Thank you for your comment.
I confirmed that the following commands work well;

  1. RTSP streaming of one CSI camera.

     GST_DEBUG=3 ./test-launch "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080 ! nvv4l2h264enc insert-sps-pps=true ! h264parse ! rtph264pay name=pay0 pt=96"
    
  2. nvcompositor of two CSI cameras + nv3dsink

     GST_DEBUG=3 gst-launch-1.0 nvcompositor name=comp sink_1::xpos=1920 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=1920, height=1080,  framerate=30/1' ! queue ! comp. nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1' ! queue ! comp. -e
    
  3. RTSP streaming of nvcompositor of two videotestsrcs

     GST_DEBUG=3 ./test-launch "nvcompositor name=comp sink_1::xpos=1920 ! nvvidconv ! video/x-raw(memory:NVMM) ! nvv4l2h264enc insert-sps-pps=true ! h264parse ! rtph264pay name=pay0 pt=96 videotestsrc is-live=true ! video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! comp. videotestsrc is-live=true pattern=ball ! video/x-raw, width=1920, height=1080, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! comp. "
    

Are there any hints in these to solve the problem?

test-launch + nvcompositor + nvarguscamerasrc == Failed to create CaptureSession?

I would appreciate any advice.

Hi,
Please download the source code of gst-nvarguscamerasrc from:
https://developer.nvidia.com/embedded/L4T/r32_Release_v4.4/r32_Release_v4.4-GMC3/Sources/T210/public_sources.tbz2

and apply the patch:

diff --git a/gst-nvarguscamera/gstnvarguscamerasrc.cpp b/gst-nvarguscamera/gstnvarguscamerasrc.cpp
index 4ccb21b..0f3d7bc 100644
--- a/gst-nvarguscamera/gstnvarguscamerasrc.cpp
+++ b/gst-nvarguscamera/gstnvarguscamerasrc.cpp
@@ -1204,6 +1204,14 @@ static gboolean gst_nv_argus_camera_set_caps (GstBaseSrc *base, GstCaps *caps)
     GST_ERROR_OBJECT (src, "caps invalid");
     return FALSE;
   }
+  if (src->hasSetCaps &&
+      src->width == info.width &&
+      src->height == info.height &&
+      src->fps_n  == info.fps_n &&
+      src->fps_d  == info.fps_d) {
+      g_print("caps is equal \n");
+      return TRUE;
+  }
 
   src->width  = info.width;
   src->height = info.height;
@@ -1260,6 +1268,7 @@ static gboolean gst_nv_argus_camera_set_caps (GstBaseSrc *base, GstCaps *caps)
     return FALSE;
   }
 
+  src->hasSetCaps = TRUE;
   return TRUE;
 }
 
diff --git a/gst-nvarguscamera/gstnvarguscamerasrc.hpp b/gst-nvarguscamera/gstnvarguscamerasrc.hpp
index 799a522..94b33e0 100644
--- a/gst-nvarguscamera/gstnvarguscamerasrc.hpp
+++ b/gst-nvarguscamera/gstnvarguscamerasrc.hpp
@@ -186,6 +186,8 @@ struct _GstNvArgusCameraSrc
   void *iEeSettings_ptr;
   void *iRequestSourceSettings_ptr;
   NvArgusFrameInfo *frameInfo;
+
+  gboolean hasSetCaps;
 };
 
 struct _GstNvArgusCameraSrcClass

Please also apply this patch to fix a known memory leak:
https://elinux.org/Jetson/L4T/r32.4.x_patches
[nvarguscamerasrc] patch for fixing memory leak

Thank you for your advice.

I followed your advice and replaced libgstnvarguscamerasrc.so, and now I can do RTSP streaming with two CSI cameras.

test-launch + nvcompositor + nvarguscamerasrc == Good job !