TK1 omxh264enc not respecting bitrate during scene changes

Autonomous Machines Jetson & Embedded Systems Jetson TK1
1

We are streaming video over a changing wireless channel. And every 2s, we will estimate wireless channel video bandwidth and re-configure omxh264enc bitrate accordingly. We have code to profile omxh264enc bitrate every 5s. What we found is every time we update bitrate or there is large scene changes, the output bitrate will overshoot the requested bitrate quite a bit. The issue is especially bad for high res like 1080p, we set 1Mbps, encoder could output 3Mbps. This issue is making us have a hard time to maintain good streaming video quality in a wireless channel.

Softare: R21.5
Omxh264enc settings:

  • iframeinterval = Fixed at 24
  • bitrate = Changing every 2s
  • control-rate=4" /constant-skip-frames/
  • low-latency=false
  • profile=high

Note: we are already doing CBR.

Also, I still have issue described in
https://devtalk.nvidia.com/default/topic/930977/tk1-h264-encoder-bitrate-profile-level-issue/

Any other folks have similar issue?

2

dk1900,
Thanks for your submission …I am wondering if you could use local clips, one with high bitrate and the other low bitrate to perform similar switching operation and see the encoder behavior. This test is to remove the factor of wireless channel changing factor.

At the same time, we will look into this from our side. IF you have further information on how your streaming environment for us to replicate the issue, that will be much helpful.

3

Hi dk1900,
Please give more information about the issues:
1 Please share the gst command you run to reproduce the sudden-high bitrate case.
2 Another issue is that you cannot configure to high profile?
3 Any other issue?

4

The issue can be reproduced with any real world 1080p video clips (with lots of scene changes). All you need to is to send bitrate change command every 2s and profile the average bitrate after encoder. I can reproduce the issue without wireless channel.

The issue cannot be reproduced with a simple gstlaunch string. I can come up with a code for you debugging.

5

Hi DaneLL and chijen,

The issue can be easily reproduced with a standard 1080p SMPT test pattern:
Please get the test C++ code from below github link:
https://github.com/darknight1900/TK1

This code will send a random bitrate change request to omxh264enc (or x264enc) every 10s and will profile average bitrate after encoder every 2s.

To test on TK1: uncomment ‘//#define TK1’

By default, I am using x264enc and you can see it will respect bitrate quite well. But if you test with omxh264enc, it will have a really hard time to respect the bitrate.

6

Hi dk1900,

ubuntu@tegra-ubuntu:~/TK1$ ./bitrate
Using launch string: videotestsrc is-live=true ! video/x-raw, format=UYVY, width=1920, height=1080, framerate=24/1 ! videoconvert ! queue ! video/x-raw, format=I420, width=1920, height=1080,framerate=24/1 ! omxh264enc bitrate=1000000 name=video_enc iframeinterval=15 profile=high low-latency=false control-rate=4 ! queue ! video/x-h264, stream-format=byte-stream  ! filesink location=nv.h264
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingFramerate set to : 24 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
===== MSENC blits (mode: 1) into tiled surfaces =====
framerate_avg:23.66, bitrate_avg:1139012.38, bitrate_set 1000000
framerate_avg:24.01, bitrate_avg:965648.19, bitrate_set 1000000
framerate_avg:24.01, bitrate_avg:982487.00, bitrate_set 1000000
framerate_avg:23.99, bitrate_avg:991460.25, bitrate_set 1000000
framerate_avg:24.03, bitrate_avg:1046579.69, bitrate_set 1000000
timeout_change_bitrate: 1000000 -> 828797
framerate_avg:24.01, bitrate_avg:1071490.50, bitrate_set 828797
framerate_avg:24.01, bitrate_avg:810500.75, bitrate_set 828797
framerate_avg:24.01, bitrate_avg:807031.88, bitrate_set 828797
framerate_avg:23.92, bitrate_avg:873985.06, bitrate_set 828797
framerate_avg:24.10, bitrate_avg:810888.31, bitrate_set 828797
timeout_change_bitrate: 828797 -> 3512590
framerate_avg:24.01, bitrate_avg:2968972.00, bitrate_set 3512590
framerate_avg:24.01, bitrate_avg:3512889.75, bitrate_set 3512590
framerate_avg:23.98, bitrate_avg:3498267.25, bitrate_set 3512590
framerate_avg:24.03, bitrate_avg:3520961.25, bitrate_set 3512590
framerate_avg:24.00, bitrate_avg:3509892.25, bitrate_set 3512590
timeout_change_bitrate: 3512590 -> 2498192
framerate_avg:24.02, bitrate_avg:2702153.00, bitrate_set 2498192

Please put is-live=true in command to have accurate frame rate.

And also please tune parameters in /etc/enctune.conf. The default values are for 6M-8Mbps.

Settings_1080P
IMinQP 40
IMaxQP 45
PMinQP 40
PMaxQP 51
7

Hi DaneLLL,

Tuning Min/Max QP for I/P frames in this hidden config file is definitely helpful to achieve lower bitrate.

Would you explain a little bit about those QP settings.
Are they hard time QP threshold, so if I set IMinQP to be 40, QP of all MBs on intra frame will not go lower than this value? Or there is some AQ adjustment on top of those frame level QP?

Also on the same system, we have other omx encoders running to record live video. We are using high bitrate for those encoders, so changing this config file might be an issue for us. I guess I could just hack the gstomxsrc code to have two config file one for high bitrate encoder one for low bitrate encoder.

8

Hi dk,
It is quantization parameter. The value is within min and max. The big value brings less encoded bits but low quality.

And having two config files is practical. Please refer to

OMX_INDEXTYPE eIndex;
    NVX_PARAM_TEMPFILEPATH enc_conf_loc;
    gchar *location = g_strdup (ENCODER_CONF_LOCATION);

    GST_DEBUG_OBJECT (self, "Setting encoder to use default configurations");

    err = gst_omx_component_get_index (self->enc,
        (char *) NVX_INDEX_PARAM_TEMPFILEPATH, &eIndex);

    if (err == OMX_ErrorNone) {
      if (location != NULL) {
        GST_OMX_INIT_STRUCT (&enc_conf_loc);
        enc_conf_loc.pTempPath = location;
        err =
            gst_omx_component_set_parameter (self->enc, eIndex, &enc_conf_loc);
      }

In gstomxvideoenc.c
Source code is at https://developer.nvidia.com/embedded/dlc/l4t-Jetson-TK1-gstreamer-1-R21-5

9

Hi DaneLLL,

We have been testing/playing around this enctune file and it works fine for video test pattern we for real word clip we found that it is hard to find one set of QP settings which will make encoder to respect bitrate for both low(200kbps) and high(4Mbps) region.

Does NVIDIA have a pre-tuned set of QPs apply to this [200k, 4Mbps] region? If necessary, I can share you the clip I am currently using to tune the encoder. It is about 450M.

Thanks

10

Hi dk,
We don’t have the pre-tuned set for the case. I think 200kbps is very strict to 1080p30. We have seen the bottom rate at 1.5Mbps at high motion case:
[url]Constant Bitrate help - Jetson TX1 - NVIDIA Developer Forums

By setting [control-rate=4" /constant-skip-frames/], the frame should be dropped once it exceeds the target bitrate. Does it help the case?