IMX390 cameras for RTSP with file store and show pictures

Hirofumi.Tamori · July 19, 2024, 3:39am

Hello,

I’m trying to use IMX390 cameras with Sony’s SerDes board by gstreamer.
At now, I have confirmed that the following script displays live video from the cameras and simultaneously saves it as an MP4 file (with four cameras running simultaneously).

#/bin/bash

SENSOR_ID=${1:-"0"}

echo "SENSOR_ID: $SENSOR_ID"

gst-launch-1.0 -e -v nvarguscamerasrc sensor-id="$SENSOR_ID" sensor_mode=0 exposuretimerange='11000000 11000
000' ee-mode='0' ispdigitalgainrange='1 8' ! 'video/x-raw(memory:NVMM), width=1936, height=1220, framerate=(fraction)50/1, format=NV12'  !  tee name=t \
    t. ! queue ! nvvidconv flip-method=0 ! fpsdisplaysink video-sink=xvimagesink text-overlay=false sync=fal
se \
    t. ! queue ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=/media/fmp/t503/output_"$SENSOR_ID".mp4

(Suppose this script name as showandrec.sh, use showandrec.sh 0-3)

And now, I’m trying to add RTSP streaming to this script but RTSP does not work at all.

First, I installed gst-rtsp-server (gst-rtsp-server) version 1.16.3.

To keep the situation simpler, as the first step, I’m writing nvargcamerasrc RTSP script like this.

GST_DEBUG=3 test-launch "nvarguscamerasrc sensor-id=0 sensor_mode=0 exposuretimerange='11000000 11000000' ee
-mode=0 ispdigitalgainrange='1 8' ! video/x-raw(memory:NVMM), width=1936, height=1220, framerate=50/1, forma
t=NV12 ! tee name=t \
       t. ! queue ! nvvidconv ! video/x-raw(memory:NVMM), width=480, height=300, format=NV12 ! nvv4l2h264en
c ! h264parse ! rtph264pay name=pay0 pt=96"

But nothing is displayed on local VLC (with rtsp://localhost:8554/test). The log of test-launch is

stream ready at rtsp://127.0.0.1:8554/test
GST_ARGUS: NvArgusCameraSrc: Setting Exposure Time Range : '11000000 11000000'
GST_ARGUS: NvArgusCameraSrc: Setting ISP Digital Gain Range : '1 8'
Opening in BLOCKING MODE
0:00:28.187827488  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881b8470 Failed to determine interlace mode
0:00:28.187877824  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881b8470 Failed to determine interlace mode
0:00:28.187895488  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881b8470 Failed to determine interlace mode
0:00:28.187910144  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881b8470 Failed to determine interlace mode
0:00:28.187980320  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:4561:gst_v4l2_object_probe_caps:<nvv4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:28.188745536  3183 0xffff5800be40 FIXME                default gstutils.c:3980:gst_pad_create_stream_id_internal:<nvarguscamerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
NvMMLiteOpen : Block : BlockType = 4
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:28.197121568  3183 0xffff5800bde0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1114:gst_v4l2_buffer_pool_start:<nvv4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold
GST_ARGUS: Creating output stream
(Argus) Error BadParameter:  (propagating from src/eglstream/FrameConsumerImpl.cpp, function initialize(), line 89)
(Argus) Error BadParameter:  (propagating from src/eglstream/FrameConsumerImpl.cpp, function create(), line
44)
Error generated. gstnvarguscamerasrc.cpp, threadInitialize:493 Failed to create FrameConsumer
Error generated. gstnvarguscamerasrc.cpp, threadFunction:409 (propagating)
Error generated. gstnvarguscamerasrc.cpp, waitRunning:371 Invalid thread state 3
Error generated. gstnvarguscamerasrc.cpp, execute:1569 (propagating)
0:00:48.141854271  3183 0xaaab15062860 WARN               rtspmedia rtsp-media.c:3272:wait_preroll: failed to preroll pipeline
0:00:48.141930270  3183 0xaaab15062860 WARN               rtspmedia rtsp-media.c:3652:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:48.146679413  3183 0xaaab15062860 ERROR             rtspclient rtsp-client.c:1077:find_media: client 0xaaab14f30ac0: can't prepare media
0:00:48.147414537  3183 0xaaab15062860 ERROR             rtspclient rtsp-client.c:2963:handle_describe_request: client 0xaaab14f30ac0: no media
GST_ARGUS: NvArgusCameraSrc: Setting Exposure Time Range : '11000000 11000000'
GST_ARGUS: NvArgusCameraSrc: Setting ISP Digital Gain Range : '1 8'
Opening in BLOCKING MODE
0:00:48.188222868  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881eb510 Failed to determine interlace mode
0:00:48.188280723  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881eb510 Failed to determine interlace mode
0:00:48.188312466  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881eb510 Failed to determine interlace mode
0:00:48.188339762  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffff881eb510 Failed to determine interlace mode
0:00:48.188399761  3183 0xaaab15062b60 WARN                    v4l2 gstv4l2object.c:4561:gst_v4l2_object_probe_caps:<nvv4l2h264enc1:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:48.189017032  3183 0xffff5800be40 FIXME                default gstutils.c:3980:gst_pad_create_stream_id_internal:<nvarguscamerasrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
NvMMLiteOpen : Block : BlockType = 4
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:48.191390147  3183 0xffff5800bde0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1114:gst_v4l2_buffer_pool_start:<nvv4l2h264enc1:pool:src> Uncertain or not enough buffers, enabling copy threshold
GST_ARGUS: Creating output stream
(Argus) Error BadParameter:  (propagating from src/eglstream/FrameConsumerImpl.cpp, function initialize(), line 89)
(Argus) Error BadParameter:  (propagating from src/eglstream/FrameConsumerImpl.cpp, function create(), line
44)
Error generated. gstnvarguscamerasrc.cpp, threadInitialize:493 Failed to create FrameConsumer
Error generated. gstnvarguscamerasrc.cpp, threadFunction:409 (propagating)
Error generated. gstnvarguscamerasrc.cpp, execute:1567 (propagating)
0:01:08.150286924  3183 0xaaab15062860 WARN               rtspmedia rtsp-media.c:3272:wait_preroll: failed to preroll pipeline
0:01:08.150359564  3183 0xaaab15062860 WARN               rtspmedia rtsp-media.c:3652:gst_rtsp_media_prepare: failed to preroll pipeline
0:01:08.156084564  3183 0xaaab15062860 ERROR             rtspclient rtsp-client.c:1077:find_media: client 0xffff5800d350: can't prepare media
0:01:08.156702321  3183 0xaaab15062860 ERROR             rtspclient rtsp-client.c:2692:handle_setup_request: client 0xffff5800d350: media '/test' not found

Why it does not work?

Regards,

JerryChang · July 19, 2024, 6:01am

please see-also Jetson AGX Orin FAQ for an example of running RTSP streaming.

Hirofumi.Tamori · July 22, 2024, 6:42am

Thanks,

Is the link you mensioned https://forums.developer.nvidia.com/t/jetson-agx-orin-faq/237459#q-is-there-any-example-of-running-rtsp-streaming-6 ?
I have already tried and confirmed the test-launch videotestsrc… pattern.
Now I’m reading https://forums.developer.nvidia.com/t/jetson-nano-faq/82953.
Which part of the FAQ can I refer? As far as I read, there are no descriptions about nvarguscamerasrc.

DaneLLL · July 22, 2024, 6:46am

Hi,
Please run the sample setup and ensure it works first. And then replace videotestsrc with nvarguscamerasrc. It supposed to work well if videotestsrc works.

Hirofumi.Tamori · July 22, 2024, 8:26am

As I said, videotestsrc works.

After that,

GST_DEBUG=3 ./test-launch "nvarguscamerasrc sensor-id=0 sensor_mode=0 exposuretimerange='11000000 11000000' ee-mode=0 ispdigitalgainrange='1 8' ! video/x-raw(memory:NVMM), width=1936, height=1220, framerate=50/1, format=NV12 ! nvvidconv ! nvv4l2h264enc ! h264parse ! rtph264pay name=pay0 pt=96"

does not work.

stream ready at rtsp://127.0.0.1:8554/test
GST_ARGUS: NvArgusCameraSrc: Setting Exposure Time Range : '11000000 11000000'
GST_ARGUS: NvArgusCameraSrc: Setting ISP Digital Gain Range : '1 8'
Opening in BLOCKING MODE
0:00:04.334269710 71022 0xaaaad8c7df00 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffffac143ee0
Failed to determine interlace mode
0:00:04.334319406 71022 0xaaaad8c7df00 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffffac143ee0
Failed to determine interlace mode
0:00:04.334336558 71022 0xaaaad8c7df00 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffffac143ee0
Failed to determine interlace mode
0:00:04.334350414 71022 0xaaaad8c7df00 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xffffac143ee0
Failed to determine interlace mode
0:00:04.334409998 71022 0xaaaad8c7df00 WARN                    v4l2 gstv4l2object.c:4561:gst_v4l2_object_probe_caps:<nvv4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:04.335106062 71022 0xffff8000f5e0 FIXME                default gstutils.c:3980:gst_pad_create_stream_id_internal:<nvarguscamerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
NvMMLiteOpen : Block : BlockType = 4
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:04.338780656 71022 0xffff8000f5e0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1114:gst_v4l2_buffer_pool_start:<nvv4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1936 x 1220 FR = 50.000000 fps Duration = 20000000 ; Analog Gain range min 1.000000, max 125.892517; Exposure Range min 80000, max 19936000;

GST_ARGUS: 1936 x 1220 FR = 36.000000 fps Duration = 27777778 ; Analog Gain range min 1.000000, max 125.892517; Exposure Range min 81000, max 27714000;

GST_ARGUS: 1936 x 1220 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 125.892517; Exposure Range min 134000,
max 33227000;

GST_ARGUS: 1936 x 1100 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 125.892517; Exposure Range min 75000, max 16608000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 0
   Output Stream W = 1936 H = 1220
   seconds to Run    = 0
   Frame Rate = 50.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
H264: Profile = 66, Level = 0
NVMEDIA: Need to set EMC bandwidth : 4786666
NvVideo: bBlitMode is set to TRUE
0:00:05.266910844 71022 0xffff8401dc00 WARN          v4l2bufferpool gstv4l2bufferpool.c:1565:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:05.268034141 71022 0xaaaad8c7dc60 FIXME              rtspmedia rtsp-media.c:4201:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:05.269379230 71022 0xaaaad8c7dc60 FIXME              rtspmedia rtsp-media.c:4201:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:05.269404350 71022 0xaaaad8c7dc60 WARN               rtspmedia rtsp-media.c:4227:gst_rtsp_media_suspend: media 0xffffac169240 was not
prepared
0:00:05.272753951 71022 0xaaaad8c7dc60 FIXME             rtspclient rtsp-client.c:1818:handle_play_request:<GstRTSPClient@0xaaaad8c77bf0> Add support for seek style (null)
0:00:05.272785183 71022 0xffff8401e640 WARN                basesink gstbasesink.c:1209:gst_base_sink_query_latency:<multiudpsink0> warning: Pipeline construction is invalid, please add queues.
0:00:05.272807679 71022 0xffff8401e640 WARN                basesink gstbasesink.c:1209:gst_base_sink_query_latency:<multiudpsink0> warning: Not enough buffering available for  the processing deadline of 0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional data. Shortening processing latency to 0:00:00.000000000.
0:00:05.272847871 71022 0xaaaad8c7dc60 FIXME              rtspmedia rtsp-media.c:2711:gst_rtsp_media_seek_full:<GstRTSPMedia@0xffffac169240> Handle going back to 0 for none live not seekable streams.
0:00:05.273116032 71022 0xaaaad8c7dc60 WARN                basesink gstbasesink.c:1209:gst_base_sink_query_latency:<multiudpsink0> warning: Pipeline construction is invalid, please add queues.
0:00:05.273137568 71022 0xaaaad8c7dc60 WARN                basesink gstbasesink.c:1209:gst_base_sink_query_latency:<multiudpsink0> warning: Not enough buffering available for  the processing deadline of 0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional data. Shortening processing latency to 0:00:00.000000000.
0:00:05.273273312 71022 0xaaaad8c7df00 WARN               rtspmedia rtsp-media.c:3014:default_handle_message: 0xffffac169240: got warning Pipeline construction is invalid, please add queues. (gstbasesink.c(1209): gst_base_sink_query_latency (): /GstPipeline:media-pipeline/GstMultiUDPSink:multiudpsink0:
Not enough buffering available for  the processing deadline of 0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional data. Shortening processing latency to 0:00:00.000000000.)
0:00:05.273352544 71022 0xaaaad8c7df00 WARN               rtspmedia rtsp-media.c:3014:default_handle_message: 0xffffac169240: got warning Pipeline construction is invalid, please add queues. (gstbasesink.c(1209): gst_base_sink_query_latency (): /GstPipeline:media-pipeline/GstMultiUDPSink:multiudpsink0:
Not enough buffering available for  the processing deadline of 0:00:00.020000000, add enough queues to buffer  0:00:00.020000000 additional data. Shortening processing latency to 0:00:00.000000000.)

It shows some FIXME and WARN messages but I cannot find out the recovery way.

JerryChang · July 22, 2024, 9:11am

hello Hirofumi.Tamori,

you may dig into below for details of FIXME…

github.com

albfan/gst-rtsp-server-wfd/blob/master/gst/rtsp-server/rtsp-media.c#L2589


      
            /* ERRORS */
          no_setup_sdp:
            {
              g_rec_mutex_unlock (&priv->state_lock);
              GST_ERROR ("no setup_sdp function");
              g_critical ("no setup_sdp vmethod function set");
              return FALSE;
            }
          }
          
          /**
           * gst_rtsp_media_suspend:
           * @media: a #GstRTSPMedia
           *
           * Suspend @media. The state of the pipeline managed by @media is set to
           * GST_STATE_NULL but all streams are kept. @media can be prepared again
           * with gst_rtsp_media_undo_reset()
           *
           * @media must be prepared with gst_rtsp_media_prepare();
           *
           * Returns: %TRUE on success.

and…
the following WARN has suggest you add queues to the pipeline, please give it a try.

system · August 14, 2024, 5:54am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Argus stability on Orin NX Jetson Orin NX camera , gstreamer	28	1327	August 28, 2023
Error connecting CSI Camera Jetpack 6.0 - nvbuf_utils: dmabuf_fd mapped entry NOT found and a lot more errors Jetson Orin Nano camera	6	539	March 23, 2024
Orin Nano error with IMX219 Camera Jetson Orin Nano camera , gstreamer	47	2908	June 21, 2023
Errors starting 7 nvarguscamersrc pipeline in R32.7.1(Jetpack 4.6.1) Jetson AGX Xavier gstreamer , sony-imx390 , deepstream	31	922	August 17, 2023
IMX378 camera freezes on Jetpack 4.2 with jetson nano Jetson Nano	20	2120	October 18, 2021
After the agx orin is powered on, the first rtmp push multiplexes are stuck Jetson AGX Orin gstreamer	12	683	September 27, 2023
Unable to stream video from IMX296 sensor on Orin NX Jetson Orin NX camera , gstreamer , sensor	26	1397	January 3, 2024
Problem with nvcompositor + nvarguscamerasrc Jetson AGX Xavier camera , gstreamer , sony-imx390	27	4358	October 18, 2021
AGX Orin: nvargus-daemon: Error InvalidState Jetson AGX Orin camera	13	1353	February 9, 2023
Argus errors on some boots Jetson TX2 camera , gstreamer	8	956	March 28, 2023

IMX390 cameras for RTSP with file store and show pictures

Related topics