Limit framerate using omxh264enc

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hi,

I am trying to hardware decode an RTSP stream and hardware re-encode the result into mpegts. I am eventually planning to use the hlssink however for the sake of simplicity for this question I will just encode into a mpegts file.

The RTSP source is 3840x2160 at 10 FPS and I wish to encode to a 3840x2160 at 10 FPS mpegts file. I am finding that using the omxh264enc encoder, I am getting a mpegts file at 30 FPS using the following command.

$ gst-launch-1.0 uridecodebin uri="rtsp://192.168.0.103:554/av0_0" ! omxh264enc ! mpegtsmux ! filesink location=test.ts

I have tried to use the “temporal-tradeoff” parameter to drop 1 in 3 frames thereby reducing the 30 FPS output to 10 FPS however when I do this I do not get a valid video file.

My only objective is to produce valid 10 FPS mpegts files in the most efficient means possible. I would be happy to use any other filter such as “videorate” (which I tried unsuccessfully) if it works.

I have included step by step commands that I have used below. Any help or advice that you can offer would be greatly appreciated.

Regards,
Ben

The RTSP source: 3840x2160 at 10FPS.

$ gst-discoverer-1.0 rtsp://192.168.0.103:554/av0_0 -v
Analyzing rtsp://192.168.0.103:554/av0_0
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Done discovering rtsp://192.168.0.103:554/av0_0

Topology:
  unknown: application/x-rtp, media=(string)video, payload=(int)96, clock-rate=(int)90000, encoding-name=(string)H264, profile-level-id=(string)420033, packetization-mode=(string)1, sprop-parameter-sets=(string)"Z0IAM5Y1QHgAh9NwEBAQIA\=\=\,aM48gA\=\=", a-framerate=(string)10, a-Media_header=(string)"MEDIAINFO\=494D4B48010100000400010000000000000000000000000000000000000000000000000000000000", ssrc=(uint)454956448, npt-start=(guint64)0, play-speed=(double)1, play-scale=(double)1
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)5.1, profile=(string)baseline, width=(int)3840, height=(int)2160, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Tags:
        video codec: H.264 (Baseline Profile)

      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)5.1, profile=(string)baseline, width=(int)3840, height=(int)2160, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
      Additional info:
        None
      Stream ID: b9049c323800fa1dbf0c9c2f5d6dcf0e63b50fc2c5030d1c14e44a893d14e333/video:0:0:RTP:AVP:96
      Width: 3840
      Height: 2160
      Depth: 24
      Frame rate: 0/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

Properties:
  Duration: 99:99:99.999999999
  Seekable: no
  Live: yes
  Tags:
      video codec: H.264 (Baseline Profile)

Encode using implied temporal-tradeoff of 0. This will produce a 30 FPS file (following code block).

$ gst-launch-1.0 uridecodebin uri="rtsp://192.168.0.103:554/av0_0" ! omxh264enc ! mpegtsmux ! filesink location=test.ts
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.0.103:554/av0_0
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 40
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.417040579
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Resultant file is 30 FPS

gst-discoverer-1.0 test.ts -v
Analyzing file:///home/benky58un/Workspace/encode_profiling/test.ts
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Done discovering file:///home/benky58un/Workspace/encode_profiling/test.ts

Topology:
  container: video/mpegts, systemstream=(boolean)true, packetsize=(int)188
    video: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)5.1
      Tags:
        video codec: H.264

      Codec:
        video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)5.1
      Additional info:
        None
      Stream ID: 306d950db7183edb65cdb7dd29854469f5d899883212858cef5d39779e714a47:1/00000041
      Width: 3840
      Height: 2160
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 0
      Max bitrate: 0

Properties:
  Duration: 0:00:07.819162925
  Seekable: yes
  Live: no
  Tags:
      video codec: H.264

When using temporal-tradeoff=2, the resultant file is invalid.

$ gst-launch-1.0 uridecodebin uri="rtsp://192.168.0.103:554/av0_0" ! omxh264enc temporal-tradeoff=2 ! mpegtsmux ! filesink location=test.ts
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.0.103:554/av0_0
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 40
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:10.118982911
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Resultant file is invalid.

$ gst-discoverer-1.0 test.ts
Analyzing file:///home/benky58un/Workspace/encode_profiling/test.ts
Done discovering file:///home/benky58un/Workspace/encode_profiling/test.ts
An error was encountered while discovering the file
 No valid frames found before end of stream
2

Hi,
The following is a reference pipeline with video file source:

$ gst-launch-1.0 filesrc location= jellyfish-5-mbps-hd-h264.mkv ! matroskademux ! h264parse ! nvv4l2decoder drop-frame-interval=3 ! videorate ! 'video/x-raw(memory:NVMM),framerate=10/1' ! nvv4l2h264enc ! h264parse ! mpegtsmux ! filesink location=a.ts

It should work if you apply the RTS source. Please give it a try.

1 Like
3

Hi DaneLLL,

Thank you for your quick response. I can confirm that your pipeline works. I used the modified version as follows

gst-launch-1.0 rtspsrc location="rtsp://192.168.0.103:554/av0_0" ! rtph264depay ! h264parse ! nvv4l2decoder drop-frame-interval=3 ! videorate ! 'video/x-raw(memory:NVMM),framerate=10/1' ! nvv4l2h264enc ! h264parse ! mpegtsmux ! filesink location=a.ts

Is there any advantage/disadvantage to using nvv4l2h264enc as opposed to omxh264enc?

Out of curiosity I noticed that I could remove “drop-frame-interval=3” and “videorate” and I would get a 30 FPS file. Where do the extra frames come from and why?

Regards,
Ben

4

Hello Ben

     As Dane suggested,  

videorate ! 'video/x-raw(memory:NVMM),framerate=10/1'

should do the trick.
For Reference https://gstreamer.freedesktop.org/documentation/tutorials/basic/handy-elements.html?gi-language=c#videorate