H264 slice intra refresh not working

[michael_gruner]

Hello

I’m trying to enable H.264 slice intra refresh in a Jetson platform but can’t get the desired result. I’m expecting the IDR frames to be replaced by a periodic “refresh wave”, where the sequence of P-frames contain columns of intra-coded macroblocks that scans the frame from left to right, periodically.

For example, if I use x264enc and enable the intra-refresh with an interval of 5 frames:

gst-launch-1.0 videotestsrc pattern=ball num-buffers=15 ! x264enc intra-refresh=true key-int-max=5 ! video/x-h264,stream-format=byte-stream,profile=baseline ! filesink location=test.h264

I get the following:
x264enc.log (37.8 KB)

The very first frame is an IDR, followed by 4 P frames with mostly inter-coded macroblocks. However, starting from the second period, you can clearly see the “refresh wave” scanning the frame as shown in the following image (see log for full details):

Now, if I attempt to enable the same intra-refresh configuration using omxh264enc as in the following pipeline:

gst-launch-1.0 videotestsrc num-buffers=20 pattern=ball ! nvvidconv ! omxh264enc SliceIntraRefreshInterval=5 SliceIntraRefreshEnable=true ! video/x-h264,stream-format=avc ! h264parse ! video/x-h264,stream-format=byte-stream ! filesink location=jetson.h264

I get the following:
omxh264enc.log (44.2 KB)

It can be seen that, unfortunately, there is no refresh wave as one would expect (see log for full details):

Exactly the same happens if I attempt to use nvv4l2h264enc instead.

Is slice intra refresh currently supported? If so, how should I configure the encoder to get the proper result?

[DaneLLL]

Hi,
Do you need the function like this:
Jetson Nano multimedia API example 01_encoder with gdr

Not sure if you mean GDR. If not, please share more detail about what kind of h264(or h265) stream you need in your use-case. The function looks advancing and we would need to check with our teams.

[michael_gruner]

Thanks @DaneLLL I’ll give the example a try. BTW, I forgot my environment details:

Jetson Nano 4GB + SD card

cat /etc/nv_tegra_release
# R32 (release), REVISION: 4.4, GCID: 23942405, BOARD: t210ref, EABI: aarch64, DATE: Fri Oct 16 19:44:43 UTC 2020

If anyone is interested, the analysis was made with these commands:

H264 Analysis Tools

[michael_gruner]

@DaneLLL thank you, the example does work closely as expected. Here’s the associated analysis:
ma_video_encode.log (35.2 KB)

And I can see a top-down refresh wave as the following:

We are trying to configure an H264 bitstream for better UDP streaming error resiliency. Among other strategies, we’re looking to have GDR to avoid network spikes on IDR frames and potentially faster recovery.

• Is the GDR setting exposed in any of the GStreamer encoders?
• How is SliceIntraRefresh different from GDR?
• Are there any other recommended encoder configurations to make the stream more robust against network losses?

[DaneLLL]

Hi,
The SliceIntraRefresh property works similar to iframeinterval. It is to encode frames like:

IDR P P P ... I P P P ... I P P P ...

It is not same as illustrated in x264enc_intra_refresh. Please try GDR and check if you can apply it to your use-case.

Another solution is to enable RPS control. It is demonstrated in 01_video_encode :

        --erps                Enable External RPS [Default = disabled]

        --svc                 Enable RPS Three Layer SVC [Default = disabled]

So that when network condition gets worse, you can drop frames in sender and the h264 stream still can be decoded in receiver.

The functions are advancing so these are not included in gst-v4l2. Would need to do porting if you require it in gstreamer.

[michael_gruner]

Thanks @DaneLLL you’ve been of great help. I’ve tagged the solution accordingly, please feel free to close the topic.