And I found that even source is local file, without sync=0, there will be duplicated frames too.
The sample is @DaneLLL provided last week (Question about nvv4l2decoder element - #5 by DaneLLL):
gst-launch-1.0 filesrc location= /opt/nvidia/deepstream/deepstream-6.1/samples/streams/sample_1080p_h265.mp4 ! qtdemux ! queue ! h265parse ! nvv4l2decoder ! nv3dsink -e
with sync=0:
without sync=0: