PTS getting changed while DTS is in order, causing jitter/stutter/jumpy video playback

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Please provide complete information as applicable to your setup.

Jetson Nano
DeepStream 6.0.1
JetPack Version 4.6.3
TensorRT Version 8.2

gst-launch-1.0 filesrc location=videos/input.mov ! qtdemux ! h265parse ! nvv4l2decoder ! nvv4l2h265enc bitrate=100000000 control-rate=1 vbv-size=33000000 EnableTwopassCBR=true maxperf-enable=true ! \
h265parse ! qtmux ! filesink sync=false location=videos/output.mov -e

I have the above pipeline simplified - the goal is to use nvdewarper (in between the codecs). I am currently seeing the PTS getting changed ie. not in order while the DTS is still in order.

Below you can see for the PTS: 49.676009 is the odd one while its DTS -3.407741 is in order with the previous and following packet. When i playback the exported file, the player plays this frame sooner than it should.

I also realize that if i set the filesink: sync=True, it greatly reduces this occurence but it still exists

[PACKET]
codec_type=video
stream_index=0
pts=149980
pts_time=50.043377
dts=-10412
dts_time=-3.474141
duration=1
duration_time=0.000334
convergence_duration=N/A
convergence_duration_time=N/A
size=445868
pos=626677138
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=150079
pts_time=50.076410
dts=-10310
dts_time=-3.440107
duration=1
duration_time=0.000334
convergence_duration=N/A
convergence_duration_time=N/A
size=440336
pos=627123006
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=148879
pts_time=49.676009
dts=-10213
dts_time=-3.407741
duration=1
duration_time=0.000334
convergence_duration=N/A
convergence_duration_time=N/A
size=366259
pos=627563342
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=150189
pts_time=50.113113
dts=-10112
dts_time=-3.374041
duration=1
duration_time=0.000334
convergence_duration=N/A
convergence_duration_time=N/A
size=356662
pos=627929601
flags=__
[/PACKET]
[PACKET]
codec_type=video
stream_index=0
pts=150286
pts_time=50.145479
dts=-10013
dts_time=-3.341008
duration=1
duration_time=0.000334
convergence_duration=N/A
convergence_duration_time=N/A
size=447615
pos=628286263
flags=__
[/PACKET]
3

Encoder does not change the value of pts. Due to the B-frames, it is possible that dts and pts may not be in the same order.

4

Thanks for the reply, how do i address this problem? I am having to manually change the PTS of the buffer at the Encoder based on the frame rate.

Could you elaborate on why the P-frames could possibly have a different PTS and also what could be causing it to be out of order. Here is a log from ffprobe of 3 packets. the 2nd one with pkt_pts_time=4.690023 is the one that is getting ‘presented’ during playback earlier than it should:

[FRAME]
media_type=video
stream_index=0
key_frame=0
pkt_pts=14155
pkt_pts_time=4.723056
pkt_dts=13744
pkt_dts_time=4.585919
best_effort_timestamp=13744
best_effort_timestamp_time=4.585919
pkt_duration=1
pkt_duration_time=0.000334
pkt_pos=56487863
pkt_size=417262
width=2016
height=1520
pix_fmt=yuv420p
sample_aspect_ratio=1:1
pict_type=P
coded_picture_number=0
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=tv
color_space=unknown
color_primaries=unknown
color_transfer=unknown
chroma_location=unspecified
[/FRAME]
[FRAME]
media_type=video
stream_index=0
key_frame=0
pkt_pts=14056
pkt_pts_time=4.690023
pkt_dts=13843
pkt_dts_time=4.618952
best_effort_timestamp=13843
best_effort_timestamp_time=4.618952
pkt_duration=1
pkt_duration_time=0.000334
pkt_pos=56905125
pkt_size=418831
width=2016
height=1520
pix_fmt=yuv420p
sample_aspect_ratio=1:1
pict_type=P
coded_picture_number=0
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=tv
color_space=unknown
color_primaries=unknown
color_transfer=unknown
chroma_location=unspecified
[/FRAME]
[FRAME]
media_type=video
stream_index=0
key_frame=0
pkt_pts=14357
pkt_pts_time=4.790457
pkt_dts=13943
pkt_dts_time=4.652319
best_effort_timestamp=13943
best_effort_timestamp_time=4.652319
pkt_duration=1
pkt_duration_time=0.000334
pkt_pos=57323956
pkt_size=417626
width=2016
height=1520
pix_fmt=yuv420p
sample_aspect_ratio=1:1
pict_type=P
coded_picture_number=0
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=tv
color_space=unknown
color_primaries=unknown
color_transfer=unknown
chroma_location=unspecified
[/FRAME]
5

A B‑frame (‘Bi-predictive picture’) saves even more space by using differences between the current frame and both the preceding and following frames to specify its content.
Could you try to set the num-B-Frames=0 for encoder and check if there are same problem?

6

@yuweiw tried setting num-B-Frames = 0, no effect, also seems like it’s already 0 by default. Any other options i can look into?

7

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Could you attach the input.mov to us?

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.