Not able to put delay on queue in deepstream with new streammux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• x86 Machine with dGPU

• DeepStream Version : deepstream-6.3

• TensorRT Version: 8.6.1.

• NVIDIA GPU Driver Version (valid for GPU only) : Driver Version: 535.86.10 CUDA Version: 12.2

• Issue Type: questions

I have a deepstream pipeline as followiing.

export USE_NEW_NVSTREAMMUX=yes

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-6.3/samples/streams/sample_1080p_h264.mp4 ! decodebin  ! nvvideoconvert !  videorate ! "video/x-raw,framerate=25/1" ! nvvideoconvert !  "video/x-raw(memory:NVMM), width=640, height=480" !  m.sink_0 nvstreammux name=m  ! nvstreamdemux name=d  d.src_0 !  nvvideoconvert ! tee name=t ! queue max-size-buffers=60 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=50 name=delay_queue !  fpsdisplaysink sync=false t. ! queue ! autovideosink

I want manual delaying of 50 frames for some experiment. But the pipeline is blocked if I am trying to play the video. If there is no delay queue it is working as normal. Please help me to put a delay queue of 50 frames.

PS: I need new streammux because of a shortcoming of the current streammux and old streammux works well with the delay.

2

Interestingly it works for smaller number of buffers(2/3) like the below one

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-6.3/samples/streams/sample_1080p_h264.mp4 ! decodebin  ! nvvideoconvert !  videorate ! "video/x-raw,framerate=25/1" ! nvvideoconvert !  "video/x-raw(memory:NVMM), width=640, height=480" !  m.sink_0 nvstreammux name=m config-file-path=streammux.txt ! nvstreamdemux name=d  d.src_0 !  nvvideoconvert ! tee name=t ! queue max-size-buffers=3 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=2 name=delay_queue !  autovideosink  sync=true async=true t. ! queue ! autovideosink sync=true async=true

Please advice on how to solve it.
@fanzh any idea?

3

new streammux
export USE_NEW_NVSTREAMMUX=yes
new streammux not working

gst-launch-1.0 --gst-debug-level=3 filesrc location=/opt/nvidia/deepstream/deepstream-6.3/samples/streams/sample_1080p_h264.mp4 ! decodebin  ! nvvideoconvert !  videorate ! "video/x-raw,framerate=25/1" ! nvvideoconvert !  "video/x-raw(memory:NVMM), width=640, height=480" !  m.sink_0 nvstreammux name=m  config-file-path=streammux.txt ! nvstreamdemux name=d  d.src_0 !  nvvideoconvert ! tee name=t ! queue max-size-buffers=30 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=20 name=delay_queue  !  fpsdisplaysink sync=false t. ! queue ! autovideosink sync=true async=false

new streammux working with smaller buffer size

gst-launch-1.0 --gst-debug-level=3 filesrc location=/opt/nvidia/deepstream/deepstream-6.3/samples/streams/sample_1080p_h264.mp4 ! decodebin  ! nvvideoconvert !  videorate ! "video/x-raw,framerate=25/1" ! nvvideoconvert !  "video/x-raw(memory:NVMM), width=640, height=480" !  m.sink_0 nvstreammux name=m  config-file-path=streammux.txt ! nvstreamdemux name=d  d.src_0 !  nvvideoconvert ! tee name=t ! queue max-size-buffers=4 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=3 name=delay_queue  !  fpsdisplaysink sync=false t. ! queue ! autovideosink sync=true async=false

old streammux

export USE_NEW_NVSTREAMMUX=no

gst-launch-1.0 --gst-debug-level=3 filesrc location=/opt/nvidia/deepstream/deepstream-6.3/samples/streams/sample_1080p_h264.mp4 ! decodebin  ! nvvideoconvert !  videorate ! "video/x-raw,framerate=25/1" ! nvvideoconvert !  "video/x-raw(memory:NVMM), width=640, height=480" !  m.sink_0 nvstreammux name=m  batch-size=1 width=1280 height=720 ! nvstreamdemux name=d  d.src_0 !  nvvideoconvert ! tee name=t !  queue max-size-buffers=30 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=20 name=delay_queue  !  fpsdisplaysink sync=false t. ! queue ! autovideosink sync=true async=false

Please find the pipelines with different outcomes. It will be help full for debugging the issue.

4

Hi @fanzh

any update on this?

I used DS 7 as well with the follwing pipeline

gst-launch-1.0 --gst-debug-level=3 filesrc location=/opt/nvidia/deepstream/deepstream-7.0/samples/streams/sample_1080p_h264.mp4 ! decodebin  ! nvvideoconvert !  videorate ! "video/x-raw,framerate=25/1" ! nvvideoconvert !  "video/x-raw(memory:NVMM), width=640, height=480" !  m.sink_0 nvstreammux name=m  config-file-path=streammux.txt batch-size=1 ! nvstreamdemux name=d  d.src_0 !  nvvideoconvert ! tee name=t ! queue max-size-buffers=30 max-size-time=2000000000 max-size-bytes= 440401920 min-threshold-time=2000000000 min-threshold-buffers=20 name=delay_queue  !  fpsdisplaysink sync=false t. ! queue ! autovideosink sync=true

and with following mux property

[property]

algorithm-type=1

batch-size=2

max-fps-control=1

overall-max-fps-n=30

overall-max-fps-d=1

overall-min-fps-n=25

overall-min-fps-d=1

max-same-source-frames=1

Not working as well, Please let me know if there is any update on this.

6

sorry for the late reply!

  1. what do you mean about “the pipeline is blocked if I am trying to play the video”? did the output video hung?
  2. some plugins have buffer pool, the plugins will wait if the pool is empty. I suggest simplify the pipeline to narrow down this issue. for example, please check if
    gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-6.3/samples/streams/sample_1080p_h264.mp4 ! decodebin + queue+… is fine, then add other plugins step by step.
7

Please help me.

PS: It doesnt work on the DS 7 as well

8

you can put the queue before nvvideoconvert. then it should not be affected by streammux.

9

But we need the queue after the streammux, because we are essentially trying to add overlays after a specific delay to the exact frame, using a queue and and pipe to send instantaneous overlays to the frame which is happening after delay.

Ajithkumar A K

10

I increased some bufferpool size. this command-line in the log works well on DS7.0. Here is the log log.txt (1.2 KB).

11

Thank you @fanzh ,
Let me check and get back the delay, with a thread and overlay.

12

Hi @fanzh

Does this solution works, for ds 6.3? Since we are heavily dependent on jetpack 5.1, which in my knowledge doesnt have DS7 supoort.

13

noticing you are using dGPU, why you are heavily dependent on jetpack 5.1. you can try the solution on DS63.

14

Hi @fanzh
we are parallel y developing an orin based solution too. I will get back on DS63 after checking the solution

15

Sorry for the late reply, Is this still an DeepStream issue to support? Thanks!

16

Hi @fanzh

it is working in DS63, new streammux

max_fps_dur 8.33333e+06 min_fps_dur 2e+08
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Missing element: MPEG-4 AAC decoder
max_fps_dur 8.33333e+06 min_fps_dur 2e+08
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:08.198833339
Setting pipeline to NULL ...
[ERROR push 353] push failed [-2]
[ERROR push 353] push failed [-2]
[ERROR push 353] push failed [-2]
[ERROR push 353] push failed [-2]
Freeing pipeline

Al though while pasuing the pipeline, We are getting [ERROR push 353] push failed [-2],

Also, it is working even without nvvideoconvert output-buffers=30 property. FYI

17

can this issue be reproduced on dGPU/Jetson with DS7.0? if yes, could you share the detailed reproducing steps and command-line? Thanks!

18

It is not replicable in DS7

19

Sorry for the late reply, Is this still an DeepStream issue to support? Thanks!

20

It is not happening in DS7, Yes we can close this

21

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.