Rtspsrc and nvv4l2decoder are causing jumps between frames in output video, Tesla T4

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Tesla T4
• DeepStream Version 5.1 and 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 470.57.02
• Issue Type( questions, new requirements, bugs) Bug
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) As below
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

We are experiencing an issue while reading from the rtsp source. The pipeline can be simplified to:

gst-launch-1.0 rtspsrc location=rtsp://  ! queue ! rtph264depay ! nvv4l2decoder ! nvv4l2h264enc ! h264parse ! splitmuxsink 

After a few minutes on the output video, we observe a loop of a few frames or a few frames are correct and then a few frames are from 4-5 minutes before.

Generally, we are using nvinfer element in this pipeline. However, the issue is visible in this simplified pipeline.

When we run it on GST_DEBUG=3 we get the following warnings in the output:

0:00:05.879711476    63 0x558a1d1388a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1066:gst_v4l2_buffer_pool_start:<nvv4l2h264enc0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:05.879829557    63 0x558a1d1388a0 WARN            v4l2videodec gstv4l2videodec.c:1673:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:06.252140749    63 0x7f7ec0042140 WARN          v4l2bufferpool gstv4l2bufferpool.c:1513:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY

We also tried using uridecode but the issue is the same.

Do you see the issue with local file?

No, I can’t reproduce it with local file.

Can you dump you rtsp output h264 file and playback the dumped h264 file?

Yes, if I use only pipeline like this

gst-launch-1.0 rtspsrc location=rtsp://  ! queue ! rtph264depay !  h264parse ! splitmuxsink

the output video is ok.

can you have a try to add h264parse before nvv4l2decoder? and set h264parse output caps to “alignment=au”?

Yes. It didn’t help, issue is still reproducing.

@kesong do you have any idea what may cause the problem?

Can you have a try to remove NV plugin to narrow down the issue?

I seem to be having the same problem. Made a thread about it here Nvv4l2decoder breaks on some H.264 streams