H264 VUI changes after L4T 28.2.1->32.1 upgrade

Hi,

I have an issue with the VUI generated by v4l tegra H264 encoder in L4T 32.1. Here is an output of h264_parse utility for H264 bitstream generated in L4T 32.1:

Nal length 26 start code 4 bytes
 ref 3 type 7 Sequence parameter set
   profile: 100
   constaint_set0_flag: 0
   constaint_set1_flag: 0
   constaint_set2_flag: 0
   constaint_set3_flag: 0
field error - 4 bits should be 0 is c
   level_idc: 51
   seq parameter set id: 0
   chroma format idx: 1
   bit depth luma minus8: 0
   bit depth chroma minus8: 0
   Qpprime Y Zero Transform Bypass flag: 0
   Seq Scaling Matrix Present Flag: 0
   log2_max_frame_num_minus4: 4
   pic_order_cnt_type: 0
    log2_max_pic_order_cnt_lsb_minus4: 4
   num_ref_frames: 1
   gaps_in_frame_num_value_allowed_flag: 0
   pic_width_in_mbs_minus1: 119 (1920)
   pic_height_in_map_minus1: 67
   frame_mbs_only_flag: 1
     derived height: 1088
   direct_8x8_inference_flag: 1
   frame_cropping_flag: 1
     frame_crop_left_offset: 0
     frame_crop_right_offset: 0
     frame_crop_top_offset: 0
     frame_crop_bottom_offset: 4
   vui_parameters_present_flag: 1
    aspect_ratio_info_present_flag: 0
    overscan_info_present_flag: 0
    video_signal_info_present_flag: 1
     video_format: 5
     video_full_range_flag: 0
     colour_description_present_flag: 0
    chroma_loc_info_present_flag: 0
    timing_info_present_flag: 1
     num_units_in_tick: 1
     time_scale: 60
     fixed_frame_scale: 1
    nal_hrd_parameters_present_flag: 0
    vcl_hrd_parameters_present_flag: 0
    pic_struct_present_flag: 0

Compare to L4T 28.2.1:

Nal length 42 start code 4 bytes
 ref 3 type 7 Sequence parameter set
   profile: 100
   constaint_set0_flag: 0
   constaint_set1_flag: 0
   constaint_set2_flag: 0
   constaint_set3_flag: 0
field error - 4 bits should be 0 is c
   level_idc: 51
   seq parameter set id: 0
   chroma format idx: 1
   bit depth luma minus8: 0
   bit depth chroma minus8: 0
   Qpprime Y Zero Transform Bypass flag: 0
   Seq Scaling Matrix Present Flag: 0
   log2_max_frame_num_minus4: 4
   pic_order_cnt_type: 0
    log2_max_pic_order_cnt_lsb_minus4: 4
   num_ref_frames: 1
   gaps_in_frame_num_value_allowed_flag: 0
   pic_width_in_mbs_minus1: 119 (1920)
   pic_height_in_map_minus1: 67
   frame_mbs_only_flag: 1
     derived height: 1088
   direct_8x8_inference_flag: 1
   frame_cropping_flag: 1
     frame_crop_left_offset: 0
     frame_crop_right_offset: 0
     frame_crop_top_offset: 0
     frame_crop_bottom_offset: 4
   vui_parameters_present_flag: 1
    aspect_ratio_info_present_flag: 0
    overscan_info_present_flag: 0
    video_signal_info_present_flag: 0
    chroma_loc_info_present_flag: 0
    timing_info_present_flag: 1
     num_units_in_tick: 1000
     time_scale: 60000
     fixed_frame_scale: 0
    nal_hrd_parameters_present_flag: 1
     cpb_cnt_minus1: 0
     bit_rate_scale: 0
     cpb_size_scale: 0
      bit_rate_value_minus1[0]: 156249
      cpb_size_value_minus1[0]: 671087
      cbr_flag[0]: 1
     initial_cpb_removal_delay_length_minus1: 23
     cpb_removal_delay_length_minus1: 15
     dpb_output_delay_length_minus1: 5
     time_offset_length: 24
    vcl_hrd_parameters_present_flag: 0
    low_delay_hrd_flag: 1
    pic_struct_present_flag: 0
    motion_vectors_over_pic_boundaries_flag: 1
    max_bytes_per_pic_denom: 0
    max_bits_per_mb_denom: 0
    log2_max_mv_length_horizontal: 16                                                                                                              
    log2_max_mv_length_vertical: 16                                                                                                                
    num_reorder_frames: 0                                                                                                                          
     max_dec_frame_buffering: 1

The difference between these 2 VUI versions causes nvdec desktop decoder to buffer frames in case of L4T 32.1 bitstream. For L4T 28.2.1 nvdec returns frames without any delay (but only if VUI is enabled in encoder). I guess these flags are of interest here:

low_delay_hrd_flag: 1
    num_reorder_frames: 0                                                                                                                          
     max_dec_frame_buffering: 1

Is it possible to return the old behaviour? I don’t want to be stuck with older L4T release and this latency issue is crucial for me.

Hi,
We will check this and update.

Hi,
Please replace attached prebuilt library and try again.
r32_1_libnvmmlite_video.zip (99.1 KB)

Hi,

I see you’ve added back

motion_vectors_over_pic_boundaries_flag: 1
    max_bytes_per_pic_denom: 0
    max_bits_per_mb_denom: 0
    log2_max_mv_length_horizontal: 16                                                                                                              
    log2_max_mv_length_vertical: 16                                                                                                                
    num_reorder_frames: 0                                                                                                                          
     max_dec_frame_buffering: 1

It fixed this issue, thanx! So I guess low_delay_hrd_flag doesn’t matter, only last two fields are needed.

DaneLLL:

The update in r32_1_libnvmmlite_video.zip hasn’t made it to the 32.2.1 release. If the low latency setting can’t be made default, can it be exposed as something configurable?

Hi xlz,
The fix does not catch r32.2.1 and will be in next r32.3.