Decoding Issuse in TX2

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi, I have issue with decoding h.264 stream in Jetson TX2 in L4T 32.6.1, Jetpack

I have 2 cameras with diffrent manufacture, and I modified 00_video_decode code and applied to my application to decode stream.

When I decode h.265 stream I can receive frame normally.

But, when I decode h.264 stream, from one camera give frame in same interval but the other one gives strange interval. (for example, normal one is decode frame every 200ms, strange interval is like 800ms, 2ms, 2ms, 100ms, 100ms I decode 5 frame in every channel)

I saw this issue (TX2 H264 decode query - #2 by DaneLLL) and used disableDPB() to end of my code.

Then, I can receive frame normally from both cameras.

My question is,

  1. Is there any Issue with DPB (by codec or manufacture) ?
  2. do I need to set profile when I make NvVideoDecoder to prevent strange frame interval?
3

Hi,
disableDPB() can reduce latency in decoding but certain streams(certain streams which have B frames) cannot be correctly decoded. If your input stream can be correctly decoded, please call it to have less latency.

4

Hi, DaneDLL. Thanks for your reply.

I understand that disableDPB is reduce decoding latency.

But my Issue was
2 different manufacture cameras with same codec(h.264), same environmnet

time it takes to decode from one camera is constant. but the other camera is not constant (some frames is take too long like 800ms, some frames take very short time like 2ms).

disableDPB was solution to make decode time constantly.

But, I don’t want to use disableDPB.

Do you think the encoded stream for camera is problem?

Is there any way to solve this problem without using disableDPB()?

Thanks.

5

Hi
You can try to parse the h264 streams and do comparison through JM decoder:
https://avc.hhi.fraunhofer.de/

If decoder keeps more decoded YUVs for the specific camera, the value of reference frame number should be larger in the bitstream.

6

Hi, I parsed the h264 stream and found that It decode same YUVs in both camera.

Both Strange and normal stream is Main@3.0 (640 x 360 resolution in 30fps)

I receive only I frame and P frame in both streams. (P frame has one nActiveRefFrame)

Difference in those two camera is normal stream camera uses hisilicon chip and strange stream camera uses ambarella chip.

When I print v4l2_ctrl_videodec_outputbuf_metadata’s frameType after dqBuffer from capture_plane received from strange stream
[I, P Frames (* 29) (stop while)], [I, P Frames (* 29), (stop while)], [I, PPP…] → This situation continues. (I think decoding I frame takes long time)

I understand I can call disableDPB if stream has one reference frame.

But I think those two streams need to work same regardless of chip manufacturer.

Do you think this is problem of camera side?

What happens to decoding I frame If I call disableDPB()?

Thanks

7

Hi,
Please compare SPS/PPS of the two streams and see if there is deviation. Probably num_ref_frames is set to different value in the two streams.

@43    SPS: num_ref_frames                                         011 (  2)

You can enable TRACEFILE to get the information.

8

I dumped SPS of two streams

normal stream (hiilicon)
==================== SPS ====================
77 = profile_idc
0 = constraint_set0_flag
0 = constraint_set1_flag
0 = constraint_set2_flag
0 = constraint_set3_flag
0 = constraint_set4_flag
0 = constraint_set5_flag
0 = reserved_zero_2bits
30 = level_idc
0 = seq_parameter_set_id
0 = chroma_format_idc
0 = residual_colour_transform_flag
0 = bit_depth_luma_minus8
0 = bit_depth_chroma_minus8
0 = qpprime_y_zero_transform_bypass_flag
0 = seq_scaling_matrix_present_flag
6 = log2_max_frame_num_minus4
2 = pic_order_cnt_type
1 = max_num_ref_frames
1 = gaps_in_frame_num_value_allowed_flag
39 (640) = pic_width_in_mbs_minus1
22 (368) = pic_height_in_map_units_minus1
1 = frame_mbs_only_flag
0 = mb_adaptive_frame_field_flag
1 = direct_8x8_inference_flag
1 = frame_cropping_flag
0 = frame_crop_left_offset
0 = frame_crop_right_offset
0 = frame_crop_top_offset
4 = frame_crop_bottom_offset
1 = vui_parameters_present_flag
vui_parameters()
0 = aspect_ratio_info_present_flag
0 = overscan_info_present_flag
1 = video_signal_type_present_flag
5 = video_format
1 = video_full_range_flag
1 = colour_description_present_flag
1 = color_primaries
1 = transfer_characteristics
1 = matrix_coefficients
0 = chroma_loc_info_present_flag
0 = timing_info_present_flag
0 = nal_hrd_parameters_present_flag
0 = vcl_hrd_parameters_present_flag
0 = nal_hrd_parameters_present_flag||vcl_hrd_parameters_present_flag
0 = pic_struct_present_flag
0 = bitstream_restriction_flag

strange stream (ambarella)
==================== SPS ====================
77 = profile_idc
0 = constraint_set0_flag
1 = constraint_set1_flag
0 = constraint_set2_flag
0 = constraint_set3_flag
0 = constraint_set4_flag
0 = constraint_set5_flag
0 = reserved_zero_2bits
30 = level_idc
0 = seq_parameter_set_id
0 = chroma_format_idc
0 = residual_colour_transform_flag
0 = bit_depth_luma_minus8
0 = bit_depth_chroma_minus8
0 = qpprime_y_zero_transform_bypass_flag
0 = seq_scaling_matrix_present_flag
12 = log2_max_frame_num_minus4
0 = pic_order_cnt_type
12 = log2_max_pic_order_cnt_lsb_minus4
1 = max_num_ref_frames
0 = gaps_in_frame_num_value_allowed_flag
39 (640) = pic_width_in_mbs_minus1
22 (368) = pic_height_in_map_units_minus1
1 = frame_mbs_only_flag
0 = mb_adaptive_frame_field_flag
1 = direct_8x8_inference_flag
1 = frame_cropping_flag
0 = frame_crop_left_offset
0 = frame_crop_right_offset
0 = frame_crop_top_offset
4 = frame_crop_bottom_offset
1 = vui_parameters_present_flag
vui_parameters()
1 = aspect_ratio_info_present_flag
1 = aspect_ratio_idc
1:1 = sar
0 = overscan_info_present_flag
1 = video_signal_type_present_flag
5 = video_format
1 = video_full_range_flag
1 = colour_description_present_flag
1 = color_primaries
1 = transfer_characteristics
1 = matrix_coefficients
0 = chroma_loc_info_present_flag
1 = timing_info_present_flag
1000 = num_units_in_tick
60000 = time_scale
1 = fixed_frame_rate_flag
1 = nal_hrd_parameters_present_flag
hrd_parameters()
0 = cpb_cnt_minus1
4 = bit_rate_scale
3 = cpb_size_scale
2999 = bit_rate_value_minus1[0]
78124 = cpb_size_value_minus1[0]
0 = cbr_flag[0]
23 = initial_cpb_removal_delay_length_minus1
15 = cpb_removal_delay_length_minus1
5 = dpb_output_delay_length_minus1
24 = time_offset_length
1 = nal_hrd_parameters_present_flag||vcl_hrd_parameters_present_flag
0 = low_delay_hrd_flag
1 = pic_struct_present_flag
0 = bitstream_restriction_flag

num_ref_frames (max_num_ref_frames) value is same. but the value of constraint_set1_flag, log2_max_frame_num_mins4 value is different and strange stream has hrd_parameters.

Could those difference make such a result?

Do I need to see something else?

Thanks.

9

Hi,
It looks like hrd parameters make the difference. Is it possible to disable it and try ?

10

Hi,
I disabled hrd parameters but symptom was same.
I found another camera has same symptom which does not send vui_parameters (H.264 HIGH@4.1)

does value of log2_max_frame_num_minus4 and pic_order_cnt_type can also make difference?

H.264 HIGH@4.1 SPS dump
==================== SPS ====================
100 = profile_idc
0 = constraint_set0_flag
0 = constraint_set1_flag
0 = constraint_set2_flag
0 = constraint_set3_flag
0 = constraint_set4_flag
0 = constraint_set5_flag
0 = reserved_zero_2bits
41 = level_idc
0 = seq_parameter_set_id
1 = chroma_format_idc
0 = residual_colour_transform_flag
0 = bit_depth_luma_minus8
0 = bit_depth_chroma_minus8
0 = qpprime_y_zero_transform_bypass_flag
0 = seq_scaling_matrix_present_flag
9 = log2_max_frame_num_minus4
0 = pic_order_cnt_type
2 = log2_max_pic_order_cnt_lsb_minus4
1 = max_num_ref_frames
1 = gaps_in_frame_num_value_allowed_flag
39 (640) = pic_width_in_mbs_minus1
22 (368) = pic_height_in_map_units_minus1
1 = frame_mbs_only_flag
0 = mb_adaptive_frame_field_flag
1 = direct_8x8_inference_flag
1 = frame_cropping_flag
0 = frame_crop_left_offset
0 = frame_crop_right_offset
0 = frame_crop_top_offset
4 = frame_crop_bottom_offset
0 = vui_parameters_present_flag

11

Hi, any updates on this issue?

12

Hi,
Probably pic_order_cnt_type triggers the difference. Please check if it can be set to identical value for a try.

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.