NVStreamMux Issue (Video source with variable FPS)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Jetson Xavier NX
• DeepStream Version 5.0
• JetPack Version4.4
• TensorRT Version 7.1.3.0
• CUDA version 10.2

Hello everyone I would like to share with you a thing that I noticed:
during the analysis of a video sources with variable FPS, nvvstreammux seems that it sets the wrong timestamps to the frames.
I am going to explain better my scenario:
I have a remote source that you can find at the following link:

the duration of that file is: 0:09:00.536000000
if I run the following pipeline I’ll get the right file duration
gst-launch-1.0 -v filesrc location=/tmp/test.mkv ! matroskademux ! h264parse ! nvv4l2decoder ! fakesink silent=false

If I run the following pipeline ( like the previous one but with nvvstreammux plugin added) I’ll get the following file duration 0:00:17.999999820

gst-launch-1.0 -v filesrc location=/tmp/test.mkv ! matroskademux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! fakesink silent=false

NB:All frames are processed but the timestamps stetted on them are wrong

With these results I think that there is a problem with NVStreamMux when it is processing a video source with variable FPS.

Do i made any mistake? is there any params that I can set to avoid this behaviour?

Is there any solution?

Regards

Ric

2 Likes
3

I test your command pipeline on deepstream verison 4 and have same result too. Hope to get information from nvidia team.

4

Basically it seems that nvstreammux assumes 30 fps for video with variable fps (framerate=(fraction)0/1) and it changes buffer timestamps accordingly and so it breaks any application that relies on buffer timestamps. nvstreammux should not change buffer timestamps at all

5

Hello, I tried to add nvstreamdemux to pipeline and get right file duration.

gst-launch-1.0 -v filesrc location=test.mkv ! matroskademux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvstreamdemux name=n n.src_0 ! fakesink silent=false
6

If you don’t want to use the demuxer you can get the original PTS from NvDsFrameMeta the buf_pts property will hold the original buffer timestamp