We are getting wrong frame order on the decoding streams on GPU. On CPU the order seems to be OK.
Nvidia Driver: 418.87
Video SDK: 9.0.20
gst-launch-1.0 nvstreammux name=mux width=544 height=544 batch-size=16 ! fakesink
uridecodebin uri=‘file:///storage/ext_media/sochi_wedding_1.h264’ ! nvvidconv ! ‘video/x-raw(memory:NVMM)’ ! videorate ! ‘video/x-raw(memory:NVMM), format=NV12, framerate=10/1’ ! queue name=q0 ! mux.sink_0
uridecodebin uri=‘file:///storage/ext_media/sochi_wedding_1.h264’ ! nvvidconv ! ‘video/x-raw(memory:NVMM)’ ! videorate ! ‘video/x-raw(memory:NVMM), format=NV12, framerate=10/1’ ! queue ! tee name=t
t. ! queue ! mux.sink_1
t. ! queue ! nvvidconv ! ‘video/x-raw’ ! videorate ! ‘video/x-raw, framerate=10/1’ ! videoscale ! ‘video/x-raw, width=640, height=360’ ! jpegenc quality=23 ! multifilesink location=’/storage/ext_media/out_dir/image_%d.jpg’
In my opinion the problem is with the “tee” plugin - one of its consumers is really fast (jpegenc and multifilesink) and the other (nvstreammux) waits for 16 frames before consuming the data (this is controlled by the “batch-size” parameter of nvstreammux). Reducing this parameter to 4 or 2 solves the problem, but it is not a viable option for us - later modules work optimally with batch-size of 32. Note that for 2 streams (i.e. using the command above) the pipeline gets stuck in PREROLLING if we set batch-size to 32.
I checked the timestamps of the buffers arriving to the jpegenc element and they have the right pts.
So indeed it looks like buffers being rewritten while travelling through the pipeline.
According to my tests, nvidia’s elements are not using the ALLOCATION query to compute the number of decoder output buffers.
The decoder seems to always allocates 4 buffers, regardless of the pipeline topology and nvstreammux is not even responding to the allocation query:
0:00:00.495817443 1831 0x16eaa80 DEBUG tee gsttee.c:617:gst_tee_query_allocation: Allocation query failed on pad src_0, ignoring allocation
I generated a graph of your full pipeline for easier visualization see attached