How to use H.264/5 control-rate with GStreamer?

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
#1

I have a script that generates different combinations of H.264/265, bitrate and control-rate (0=VBR,1=CBR), e.g.:

==> h265 br=1000000
gst-launch-1.0 filesrc location=../src.mjpeg ! 'image/jpeg,framerate=200/1,width=192,height=192' ! nvjpegdec idct-method=float ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! nvv4l2h265enc bitrate=1000000 control-rate=0 ! filesink location=nv_h265_br1000000_ctrl0_patchedgstjpg.h265
==> h265 br=1000000
gst-launch-1.0 filesrc location=../src.mjpeg ! 'image/jpeg,framerate=200/1,width=192,height=192' ! nvjpegdec idct-method=float ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! nvv4l2h265enc bitrate=1000000 control-rate=1 ! filesink location=nv_h265_br1000000_ctrl1_patchedgstjpg.h265

However, the control-rate parameter has no effect, the produced files are identical.

How do I enable variable bitrate in the H.264/H.265 GStreamer encoder?

I found this three-year-old semi-related topic TX1 H265 encode rate-control mode VBR/CBR, which indicates that this should be supported by now.

ls -l
total 389496
-rw-r--r--  1 coudy  staff  36951567  8 Apr 15:01 nv_h264_br1000000_ctrl0_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff  36951567  8 Apr 15:01 nv_h264_br1000000_ctrl1_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff   3763402  8 Apr 15:01 nv_h264_br100000_ctrl0_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff   3763402  8 Apr 15:01 nv_h264_br100000_ctrl1_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff   7466901  8 Apr 15:01 nv_h264_br200000_ctrl0_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff   7466901  8 Apr 15:01 nv_h264_br200000_ctrl1_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff   1780226  8 Apr 15:01 nv_h264_br50000_ctrl0_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff   1780226  8 Apr 15:01 nv_h264_br50000_ctrl1_patchedgstjpg.h264
-rw-r--r--  1 coudy  staff  36885668  8 Apr 15:01 nv_h265_br1000000_ctrl0_patchedgstjpg.h265
-rw-r--r--  1 coudy  staff  36885668  8 Apr 15:01 nv_h265_br1000000_ctrl1_patchedgstjpg.h265
-rw-r--r--  1 coudy  staff   3714848  8 Apr 15:01 nv_h265_br100000_ctrl0_patchedgstjpg.h265
-rw-r--r--  1 coudy  staff   3714848  8 Apr 15:01 nv_h265_br100000_ctrl1_patchedgstjpg.h265
-rw-r--r--  1 coudy  staff   7403316  8 Apr 15:01 nv_h265_br200000_ctrl0_patchedgstjpg.h265
-rw-r--r--  1 coudy  staff   7403316  8 Apr 15:01 nv_h265_br200000_ctrl1_patchedgstjpg.h265
-rw-r--r--  1 coudy  staff   1729854  8 Apr 15:01 nv_h265_br50000_ctrl0_patchedgstjpg.h265
-rw-r--r--  1 coudy  staff   1729854  8 Apr 15:01 nv_h265_br50000_ctrl1_patchedgstjpg.h265
#3

Hi,
1Mbps looks to be a large value for 192x192 200fps. You may try with some public y4m files such as

#4

As you’ll see above I tried going as low as 50kbps. The quality was noticeably worse and yet the output files were still byte-for-byte identical.

Still nothing with the park_joy video:

coudy@nanog:~$ gst-launch-1.0 filesrc location= park_joy_1080p50.y4m ! y4mdec ! nvvidconv ! nvv4l2h264enc bitrate=14000000 control-rate=0 ! h264parse ! filesink location=park_joy0.h264
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is PREROLLING ...
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:08.289144732
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
coudy@nanog:~$ gst-launch-1.0 filesrc location= park_joy_1080p50.y4m ! y4mdec ! nvvidconv ! nvv4l2h264enc bitrate=14000000 control-rate=1 ! h264parse ! filesink location=park_joy1.h264
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is PREROLLING ...
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:08.312710034
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
coudy@nanog:~$ shasum park_joy*.h264
468a914fd8bd25da8b9e7da71d1e89669a1f6795  park_joy0.h264
468a914fd8bd25da8b9e7da71d1e89669a1f6795  park_joy1.h264

Again, lowering the bitrate doesn’t make VBR work here.

#6

Hi,
We can reproduce the issue. Will check and update.

#7

Hi,
Please apply the attached patch to gst-v4l2 and rebuild/replace libgstnvvideo4linux2.so. The source of gst-v4l2 is in
https://developer.nvidia.com/embedded/L4T/r32_Release_v4.2/Sources/T210/public_sources.tbz2

r32_3_1_TEST_0001-gst-v4l2-set-encoder-property-after-S_FMT.zip (1.0 KB)

TX2i video encoder frequency
#9

I’m running into the same issue with the vp9 encoder nvv4l2vp9enc (on JetPack 4.3).

You can quickly test with:
gst-launch-1.0 -v videotestsrc num-buffers=1000 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2vp9enc control-rate=0 bitrate=1000000 ! webmmux ! filesink location=test0.webm
and:
gst-launch-1.0 -v videotestsrc num-buffers=1000 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2vp9enc control-rate=1 bitrate=1000000 ! webmmux ! filesink location=test1.webm
When I run these two commands, both video files are of identical size.

#10

Hi,
Please follow this post to apply the fix.

#11

@DaneLLL it looks like that patch only applies to H264/H265, not VP9, is that correct?

#12

Hi,
The patch is general for all encoders, not specific to H264/H265. Please give it a try.

#13

@DaneLLL Has this patch been merged in the official release yet?

#14

Hi,

The patch is in Jetpack 4.4.1(r32.4.4).