GStreamer pipeline to convert MJPEG to RTSP server stream with adaption of framerate

nvidia808 · November 15, 2021, 9:45am

I created a pipeline to convert MJPEG http source to a rtsp server which is basically working like this:

/usr/local/bin/gst-rtsp-cli -p 8009 -b 0.0.0.0 "/video" "souphttpsrc location=http://0.0.0.0:8082 do-timestamp=true is_live=true ! multipartdemux ! image/jpeg,framerate=10/1,width=1920,height=1080 ! nvjpegdec ! omxh264enc profile=main bitrate=4000000 ! h264parse ! rtph264pay name=pay0 pt=96 config-interval=1

The source probed via ffprobe has the following characteristics:

Input #0, mpjpeg, from 'http://0.0.0.0:8082':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 25 tbr, 25 tbn, 25 tb

Now when I try to ffprobe the output as first client I get:

[rtsp @ 0x5592c36a30] max delay reached. need to consume packet
[rtsp @ 0x5592c36a30] RTP: missed 28 packets
[h264 @ 0x5592c39910] corrupted macroblock 60 60 (total_coeff=-1)
[h264 @ 0x5592c39910] error while decoding MB 60 60
[h264 @ 0x5592c39910] concealing 949 DC, 949 AC, 949 MV errors in I frame
Input #0, rtsp, from 'rtsp://0.0.0.0:8009/video':
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
  Duration: N/A, start: 0.218756, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080, 90k tbr, 90k tbn, 180k tbc

But when other clients consume the stream, ffprobe on same host shows this:

[rtsp @ 0x55a6ca5a30] method PLAY failed: 503 Service Unavailable
rtsp://0.0.0.0:8009/video: Server returned 5XX Server Error reply

or

[rtsp @ 0x55bb50da30] method DESCRIBE failed: 503 Service Unavailable
rtsp://0.0.0.0:8009/video: Server returned 5XX Server Error reply

With GST_DEBUG=2 I get the following:

0:00:08.484084758 28379   0x559cb3b230 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /root/.config:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Framerate set to : 10 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:10.339949291 28379   0x7f8c005d40 WARN             omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
H264: Profile = 77, Level = 40
NVMEDIA_ENC: bBlitMode is set to TRUE
0:00:10.375822480 28379   0x7f845ea8a0 WARN               h264parse gsth264parse.c:2752:gst_h264_parse_set_caps:<h264parse0> H.264 AVC caps, but no codec_data
0:00:10.375970481 28379   0x7f845ea8a0 WARN               h264parse gsth264parse.c:2778:gst_h264_parse_set_caps:<h264parse0> refused caps video/x-h264, alignment=(string)au, profile=(string)main, level=(string)4, stream-format=(string)avc, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)10/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:0:0, chroma-site=(string)mpeg2, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
0:00:10.376079057 28379   0x7f845ea8a0 WARN               h264parse gsth264parse.c:2752:gst_h264_parse_set_caps:<h264parse0> H.264 AVC caps, but no codec_data
0:00:10.376157361 28379   0x7f845ea8a0 WARN               h264parse gsth264parse.c:2778:gst_h264_parse_set_caps:<h264parse0> refused caps video/x-h264, alignment=(string)au, profile=(string)main, level=(string)4, stream-format=(string)avc, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)10/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:0:0, chroma-site=(string)mpeg2, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
0:00:10.376205137 28379   0x7f845ea8a0 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<omxh264enc-omxh264enc0:src> could not send sticky events
0:00:10.403876551 28379   0x559cb3b230 WARN               rtspmedia rtsp-media.c:3867:gst_rtsp_media_suspend: media 0x7f940a42c0 was not prepared

When a client like VLC consumes the stream at first, it works for VLC, but the framerate is 29.97 fps and not 10 fps.

So how to fix the framerate?
How to fix the access from different clients?
How to fix the caps of h264parse?

Many thanks in advance.

Kind regards,
Markus

DaneLLL · November 16, 2021, 2:35am

Hi,
Looks like your source is in 29.97 fps. For a live source, you would need to check if the source can be set to 10fps.

And we suggest use v4l2 plugins since omx plugins are deprecated. Please try the command like:

... ! nvv4l2decoder mjpeg=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvv4l2h264enc ! ...

nvidia808 · November 16, 2021, 7:55pm

Hi DaneLLL,

thanks for the quick reply.

I tried with

/usr/local/bin/gst-rtsp-cli -p 8009 -b 0.0.0.0 "/video" "souphttpsrc location=http://192.168.14.103:8082 do-timestamp=true is_live=true ! multipartdemux ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! nvv4l2h264enc profile=2 ! h264parse ! rtph264pay name=pay0 pt=96 config-interval=1"

but this leads to many warnings and on vlc side it takes 5-8 seconds before the stream starts:

Opening in BLOCKING MODE
0:00:03.024320129 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f840854e0 Failed to determine interlace mode
0:00:03.024472130 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f840854e0 Failed to determine interlace mode
0:00:03.024550658 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f840854e0 Failed to determine interlace mode
0:00:03.024624962 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f840854e0 Failed to determine interlace mode
0:00:03.024899267 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:4476:gst_v4l2_object_probe_caps:<nvv4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
0:00:03.074100016 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:4476:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:03.074198097 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f84079240 Failed to determine interlace mode
0:00:03.074298481 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f84079240 Failed to determine interlace mode
0:00:03.074382641 30677   0x7f84026720 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f84079240 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 277
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 277
0:00:04.477970040 30677   0x7f7807dc50 WARN                    v4l2 gstv4l2object.c:4476:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:04.478077656 30677   0x7f7807dc50 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f84079240 Failed to determine interlace mode
0:00:04.478145400 30677   0x7f7807dc50 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f84079240 Failed to determine interlace mode
0:00:04.478196056 30677   0x7f7807dc50 WARN                    v4l2 gstv4l2object.c:2388:gst_v4l2_object_add_interlace_mode:0x7f84079240 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
0:00:04.486347856 30677   0x7f7807dc50 WARN                    v4l2 gstv4l2object.c:4043:gst_v4l2_object_set_format_full:<nvv4l2h264enc0:sink> Reuse caps framerate 0/1 - fix v4l2 output driver
0:00:04.487486547 30677   0x7f7807dc50 WARN          v4l2bufferpool gstv4l2bufferpool.c:1065:gst_v4l2_buffer_pool_start:<nvv4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:04.488239285 30677   0x7f7807dc50 WARN            v4l2videodec gstv4l2videodec.c:1685:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:04.493027011 30677   0x7f7807dc50 WARN          v4l2bufferpool gstv4l2bufferpool.c:1065:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:04.523731611 30677   0x7f585ec720 WARN          v4l2bufferpool gstv4l2bufferpool.c:1512:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
H264: Profile = 77, Level = 0
NVMEDIA_ENC: bBlitMode is set to TRUE
0:00:04.548351810 30677   0x7f58574cf0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1512:gst_v4l2_buffer_pool_dqbuf:<nvv4l2h264enc0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:04.612667195 30677   0x558a18e590 WARN               rtspmedia rtsp-media.c:3867:gst_rtsp_media_suspend: media 0x7f840a9280 was not prepared
0:00:15.472696102 30677   0x558a18e590 WARN               rtspmedia rtsp-media.c:4156:gst_rtsp_media_set_state: media 0x7f840a9280 was not prepared
Opening in BLOCKING MODE

How to get rid of the warnings?

The source is motion-based mjpeg stream, settings in MotionEye are set to 10 fps. So I wonder where the 29.97 fps come from.

Thanks for your support.

DaneLLL · November 25, 2021, 10:27am

Hi,
For 5-8 second latency, please set idrinterval=15 and check if there is improvement.

May refer to this post:
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

system · December 15, 2021, 6:41am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Problems about the image resolution of rtsp stream DeepStream SDK deepstream	17	27	December 14, 2024
Failed to stream GPU encoded rtsp-server using gstreamer DeepStream SDK rtsp , cuda , gstreamer , encoder	12	102	November 28, 2024
GStreamer Output with Hardware Decoding/Encoding Sped Up Jetson Nano gstreamer	3	675	October 18, 2021
using gstreamer to transcode/downscale 1080i videos on Jetson TX2 Jetson TX2	19	3772	October 18, 2021
RTSP stream not work "Stream format not found, dropping the frame" DeepStream SDK rtsp , gstreamer , nvbugs	24	1559	January 3, 2024
Problems with gstreamer video decoding pipeline Jetson AGX Xavier	11	5283	October 18, 2021
gStreamer - high delay, low fps Jetson AGX Orin gstreamer	12	919	March 26, 2024
Gst_rtsp_server error and VLC not playing stream Jetson TX2 rtsp , gstreamer	4	3461	October 18, 2021
Gstreamer RTMPsrc -> filesink stops after few seconds Jetson Nano	2	1961	October 15, 2021
Gstreamer ERROR: gstDecoder -- failed to retrieve next image buffer Jetson AGX Xavier camera , gstreamer	2	1630	March 31, 2022

GStreamer pipeline to convert MJPEG to RTSP server stream with adaption of framerate

Related topics