Mismatch between stss and key frames

Hi! We found an odd problem with nvv4l2h264enc on r35.4.1: result mp4 has no parity between key frames and stss samples. From what I understand, in valid mp4 file, stss should point to key frames, but in our case there is just 1 key frame in beginning, while stss set to each 30 frames (15 fps video). That results in odd behavior while trying to seek through video in certain players, particularly in chrome browser. The problem was not observed on older version, like r32.4.4, where key frames and stss are always matched, so I assume the bug was introduced somewhere between that two versions (r32.4.4 - r35.4.1). Also note that h265 encoder is not affected by this bug.

Here you can find example of such video, produced by command:

gst-launch-1.0 videotestsrc num-buffers=151 ! video/x-raw, width=1920, height=1080, framerate=15/1 ! nvvidconv ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc maxperf-enable=1 control-rate=0 bitrate=8000000 ! h264parse ! mp4mux ! filesink location=/tmp/out.mp4

Here is ffprobe that provides info on the problem:

ffprobe -loglevel error -select_streams v:0 -show_packets -show_entries frame=pts_time,key_frame,pict_type -show_entries packet=pts_time,flags out.mp4

when applied to the video it shows that there are 6 packets with flag K__, but only one frame that marked as a key_frame. Also MP4Box (MP4Box -std -diso out.mp4 | grep stss) shows that there is clearly 6 stss samples:

<SyncSampleBox Size="40" Type="stss" Version="0" Flags="0" Specification="p12" Container="stbl" EntryCount="6">

So, is there any options or etc that we can use to avoid that problem?

Hi,
You may try to configure the properties in nvv4l2h264enc for a try:

  disable-cabac       : Set Entropy Coding Type CAVLC(TRUE) or CABAC(FALSE)
                        flags: readable, writable
                        Boolean. Default: false
  insert-aud          : Insert H.264 Access Unit Delimiter(AUD)
                        flags: readable, writable
                        Boolean. Default: false
  insert-sps-pps      : Insert H.264 SPS, PPS at every IDR frame
                        flags: readable, writable
                        Boolean. Default: false
  insert-vui          : Insert H.264 VUI(Video Usability Information) in SPS
                        flags: readable, writable
                        Boolean. Default: false
  poc-type            : Set Picture Order Count type value
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 2 Default: 0
  profile             : Set profile for v4l2 encode
                        flags: readable, writable, changeable only in NULL or READY state
                        Enum "GstV4l2VideoEncProfileType" Default: 0, "Baseline"
                           (0): Baseline         - GST_V4L2_H264_VIDENC_BASELINE_PROFILE
                           (1): Constrained-Baseline - GST_V4L2_H264_VIDENC_CONSTRAINED_BASELINE_PROFILE
                           (2): Main             - GST_V4L2_H264_VIDENC_MAIN_PROFILE
                           (4): High             - GST_V4L2_H264_VIDENC_HIGH_PROFILE
                           (17): Constrained-High - GST_V4L2_H264_VIDENC_CONSTRAINED_HIGH_PROFILE
                           (7): High444          - GST_V4L2_H264_VIDENC_HIGH_444_PREDICTIVE_PROFILE

And may try qtmux plugin.

unfortunately none of the settings didn’t make impact on the problem. Using qtmux also didn’t help

Hi,
Please also set the properties:

  idrinterval         : Encoding IDR Frame occurance frequency
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 256
  iframeinterval      : Encoding Intra Frame occurance frequency
                        flags: readable, writable, changeable only in NULL or READY state
                        Unsigned Integer. Range: 0 - 4294967295 Default: 30

By default IDR interval is 256 frames and it seems like you need it to be 30. You can set iframeinterval=0 to enable only IDR frames.

Thanks! Looks like disabling iframeinterval fixes the issue. I just have last three questions:

  1. is it a bug or intended behavior? Seems like iframes should not be marked by stss
  2. would that harm performance if we disable iframes and leave only IDR? Older version has no IDR interval setting and looks like it just hardcoded to be 30, what is default value for iframe interval
  3. why h265 videos have no that problem? The IDR and iframe defaults are identical and stss covers them both, but the video being played just fine. However looks like that is player problem

Hi,
For further check you may download source code of mp4mux from
Index of /src/gst-plugins-good

Not sure but probably IDR/I frames are handled differently between H264 and H265 streams.