Limit framerate using omxh264enc

benky58un · February 10, 2020, 8:17am

Hi,

I am trying to hardware decode an RTSP stream and hardware re-encode the result into mpegts. I am eventually planning to use the hlssink however for the sake of simplicity for this question I will just encode into a mpegts file.

The RTSP source is 3840x2160 at 10 FPS and I wish to encode to a 3840x2160 at 10 FPS mpegts file. I am finding that using the omxh264enc encoder, I am getting a mpegts file at 30 FPS using the following command.

$ gst-launch-1.0 uridecodebin uri="rtsp://192.168.0.103:554/av0_0" ! omxh264enc ! mpegtsmux ! filesink location=test.ts

I have tried to use the “temporal-tradeoff” parameter to drop 1 in 3 frames thereby reducing the 30 FPS output to 10 FPS however when I do this I do not get a valid video file.

My only objective is to produce valid 10 FPS mpegts files in the most efficient means possible. I would be happy to use any other filter such as “videorate” (which I tried unsuccessfully) if it works.

I have included step by step commands that I have used below. Any help or advice that you can offer would be greatly appreciated.

Regards,
Ben

The RTSP source: 3840x2160 at 10FPS.

$ gst-discoverer-1.0 rtsp://192.168.0.103:554/av0_0 -v
Analyzing rtsp://192.168.0.103:554/av0_0
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Done discovering rtsp://192.168.0.103:554/av0_0

Topology:
  unknown: application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)H264, profile-level-id=(string)420033, packetization-mode=(string)1, sprop-parameter-sets=(string)"Z0IAM5Y1QHgAh9NwEBAQIA\=\=\,aM48gA\=\=", a-framerate=(string)10, a-Media_header=(string)"MEDIAINFO\=494D4B48010100000400010000000000000000000000000000000000000000000000000000000000", ssrc=(uint)454956448, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)5.1, profile=(string)baseline, width=(int)3840, height=(int)2160, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Tags:
        video codec: H.264 (Baseline Profile)

      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)5.1, profile=(string)baseline, width=(int)3840, height=(int)2160, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Additional info:
        None
      Stream ID: b9049c323800fa1dbf0c9c2f5d6dcf0e63b50fc2c5030d1c14e44a893d14e333/video:0:0:RTP:AVP:96
      Width: 3840
      Height: 2160
      Depth: 24
      Frame rate: 0/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

Properties:
  Duration: 99:99:99.999999999
  Seekable: no
  Live: yes
  Tags:
      video codec: H.264 (Baseline Profile)

Encode using implied temporal-tradeoff of 0. This will produce a 30 FPS file (following code block).

$ gst-launch-1.0 uridecodebin uri="rtsp://192.168.0.103:554/av0_0" ! omxh264enc ! mpegtsmux ! filesink location=test.ts
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.0.103:554/av0_0
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 40
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.417040579
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Resultant file is 30 FPS

gst-discoverer-1.0 test.ts -v
Analyzing file:///home/benky58un/Workspace/encode_profiling/test.ts
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Done discovering file:///home/benky58un/Workspace/encode_profiling/test.ts

Topology:
  container: video/mpegts, systemstream=(boolean)true, packetsize=(int)188
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)5.1
      Tags:
        video codec: H.264

      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)5.1
      Additional info:
        None
      Stream ID: 306d950db7183edb65cdb7dd29854469f5d899883212858cef5d39779e714a47:1/00000041
      Width: 3840
      Height: 2160
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

Properties:
  Duration: 0:00:07.819162925
  Seekable: yes
  Live: no
  Tags:
      video codec: H.264

When using temporal-tradeoff=2, the resultant file is invalid.

$ gst-launch-1.0 uridecodebin uri="rtsp://192.168.0.103:554/av0_0" ! omxh264enc temporal-tradeoff=2 ! mpegtsmux ! filesink location=test.ts
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.0.103:554/av0_0
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 40
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:10.118982911
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Resultant file is invalid.

$ gst-discoverer-1.0 test.ts
Analyzing file:///home/benky58un/Workspace/encode_profiling/test.ts
Done discovering file:///home/benky58un/Workspace/encode_profiling/test.ts
An error was encountered while discovering the file
 No valid frames found before end of stream

DaneLLL · February 10, 2020, 10:57am

Hi,
The following is a reference pipeline with video file source:

$ gst-launch-1.0 filesrc location= jellyfish-5-mbps-hd-h264.mkv ! matroskademux ! h264parse ! nvv4l2decoder drop-frame-interval=3 ! videorate ! 'video/x-raw(memory:NVMM),framerate=10/1' ! nvv4l2h264enc ! h264parse ! mpegtsmux ! filesink location=a.ts

It should work if you apply the RTS source. Please give it a try.

benky58un · February 10, 2020, 11:50am

Hi DaneLLL,

Thank you for your quick response. I can confirm that your pipeline works. I used the modified version as follows

gst-launch-1.0 rtspsrc location="rtsp://192.168.0.103:554/av0_0" ! rtph264depay ! h264parse ! nvv4l2decoder drop-frame-interval=3 ! videorate ! 'video/x-raw(memory:NVMM),framerate=10/1' ! nvv4l2h264enc ! h264parse ! mpegtsmux ! filesink location=a.ts

Is there any advantage/disadvantage to using nvv4l2h264enc as opposed to omxh264enc?

Out of curiosity I noticed that I could remove “drop-frame-interval=3” and “videorate” and I would get a 30 FPS file. Where do the extra frames come from and why?

Regards,
Ben

SarathSSC · February 10, 2020, 12:33pm

Hello Ben

     As Dane suggested,

videorate ! 'video/x-raw(memory:NVMM),framerate=10/1'

should do the trick.
For Reference https://gstreamer.freedesktop.org/documentation/tutorials/basic/handy-elements.html?gi-language=c#videorate

Topic		Replies	Views
H264 encoding from RGB source Jetson Xavier NX encoder	13	3509	December 22, 2021
GStreamer pipeline to convert MJPEG to RTSP server stream with adaption of framerate Jetson Xavier NX gstreamer	4	4076	December 15, 2021
omxh264 encoded data bitrate bigger than set Jetson AGX Xavier	4	651	October 18, 2021
omxh264dec nvvidconv performance problem. Jetson TX1	1	2299	May 27, 2016
Mpeg2 tts Jetson Nano gstreamer	12	953	August 30, 2023
TX2 H264 RTSP Stream decoding issues Jetson TX2	27	11553	October 18, 2021
JX2, ZedMINI GStreamer, RTSP server using Android phone Jetson TX2	8	2086	August 7, 2018
omxh264enc with gst-rtsp-server-1.8.3.tar.xz Jetson TX2	4	2606	October 18, 2021
omxh264dec vs nv_omx_h264dec Jetson TK1	12	7733	October 18, 2021
Memory leaks detected in GStreamer elements (omxh264 or nvvidconv) Jetson TX2	14	5039	October 18, 2021

Limit framerate using omxh264enc

Related topics