New nvstreammux hangs the Pipeline

[lienes.cirules]

Working with DS.6.4, dGPU.
Driver Version: 535.183.06 CUDA Version: 12.2.
GeForce RTX 2080 Ti.

I have a pipeline with multiple rtspsrc input bins that goes into nvstreammux, than further to fakesink async=0.

The camera input configured to 30fps, 1920x1080.

I activated new nvstreammux element.
Configured by fallowing properties:

[property]
algorithm-type=1
adaptive-batching=0
max-fps-control=0
overall-max-fps-n=50
overall-max-fps-d=1
overall-min-fps-n=30
overall-min-fps-d=1

batch-size=1
batched-push-timeout=33000

Thanks in advance!

[junshengy]

How do you reproduce the issue? Which sample and configuration file do you use?
Or try this FAQ first.

DeepStream SDK FAQ

26.[Deepstream6.2 Gst-nvstreammux & Gst-nvstreammux New]How to set parameters reasonably to improve the efficiency of nvstreammux in live mode Gst-nvstreammux: Set the batch-size to the number of sources export NVSTREAMMUX_ADAPTIVE_BATCHING=yes If you want to get high fps, set the batched-push-timeout to the (1000000 us/maximum fps among the videos) Gst-nvstreammux New(export USE_NEW_NVSTREAMMUX=yes): Set the batch-size to the number of sources Should not turn off the adaptive-batching par…

[lienes.cirules]

I am using nvstreammux in my app development - meaning, no sample code.

I am using config file for new nvstreammux as written above.

Here is the log of GST_DEBUG=nvstreammux:5.

nvstreammux :0::<stream-muxer> [get_batch_size 331]get_batch_size inputs->size=1 num_sources_eos=0 num_surfaces_per_frame=1 config=1 adaptive_batching=0, batch_size=1

nvstreammux :0::<stream-muxer> [form_batch 479]sources_to_process=1

nvstreammux :0::<stream-muxer> [update_with_source 396]update_with_source sid=0

nvstreammux :0::<stream-muxer> [get_available 127]number of buffers available in queue of pad (0) = 2

nvstreammux :0::<stream-muxer> [get_allowed 304]allowed frames 0 fps 0.000000 diff 3157996330.000000 source id 0

I am wondering that the allowed frames 0 fps. Is it ok ?

Here is a log more that goes in the loop. Meanwhile no buffers running on nvstreammux src pad. I am monitoring buffer flow with BUFFER probe.

nvstreammux :0::<stream-muxer> [form_batch 479]sources_to_process=1

0:00:04.744440052 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [form_batch 496]update_with_source sid=0

0:00:04.744452155 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [get_batch_size 331]get_batch_size inputs->size=1 num_sources_eos=0 num_surfaces_per_frame=1 config=1 adaptive_batching=0, batch_size=1

0:00:04.744462504 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [update_with_source 396]update_with_source sid=0

0:00:04.744472553 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [get_available 127]number of buffers available in queue of pad (0) = 2

0:00:04.744486649 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [get_allowed 304]allowed frames 0 fps 0.000000 diff 3159974992.000000 source id 0

0:00:04.744496968 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [update_with_source 401]num_avail=0 sid=0

0:00:04.744508971 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [get_batch_size 331]get_batch_size inputs->size=1 num_sources_eos=0 num_surfaces_per_frame=1 config=1 adaptive_batching=0, batch_size=1

0:00:04.744520002 3739726 0x7fffdc001230 DEBUG            nvstreammux :0::<stream-muxer> [update_with_source 408]allowed_repeats=1 allowed_repeats_source=1 bs=1

Note, that nvstreammux and pipeline reports state change complete to PLAYING.

[junshengy]

@lienes.cirules

I am using nvstreammux in my app development - meaning, no sample code.

I need the complete pipeline and configuration files to analyze the problem. I cannot determine the problem with these logs.

@lienes.cirules

I am wondering that the allowed frames 0 fps. Is it ok ?

There seems to be nothing wrong here, this does not mean that the fps is 0, you can refer to the code of new nvstreammux, it is open source

/opt/nvidia/deepstream/deepstream/sources/libs/nvstreammux/nvstreammux_batch.cpp

[tomskih_pa]

@junshengy I also faced this issue. Pipeline works when config-file-path is not set, but hangs when I pass a config file to config-file-path.

Here’s config file (nvstreamux.txt):

[property]
algorithm-type=1
batch-size=1
overall-max-fps-n=120
overall-max-fps-d=1
overall-min-fps-n=5
overall-min-fps-d=1
max-same-source-frames=1
adaptive-batching=1
max-fps-control=0

All parameters are default according to Gst-nvstreammux New — DeepStream documentation 6.4 documentation

The pipeline:

docker run \
    --name test \
    --gpus all \
    --entrypoint gst-launch-1.0 \
    -v "$(pwd):/test" \
    -e USE_NEW_NVSTREAMMUX=yes \
    -e GST_DEBUG=GST_SCHEDULING:5,nvstreammux:5 \
    -e GST_DEBUG_NO_COLOR=1 \
    nvcr.io/nvidia/deepstream:6.4-samples-multiarch \
    --eos-on-shutdown \
    nvstreammux name=mux batch-size=1 config-file-path=/test/nvstreamux.txt ! \
    fakesink sync=false qos=false enable-last-sample=false async=false \
    videotestsrc ! \
    'video/x-raw,width=1280,height=720,framerate=30/1' ! \
    nvvideoconvert ! \
    'video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1' ! \
    mux.sink_0

Logs when config-file-path is specified:
nvstreammux-with-config.log (878.9 KB)

Only one buffer reached fakesink:

0:00:00.488622344     1 0x570cf09b9a40 DEBUG         GST_SCHEDULING gstpad.c:4443:gst_pad_chain_data_unchecked:<fakesink0:sink> calling chainfunction &gst_base_sink_chain with buffer buffer: 0x570cf09d2b40, pts 0:00:00.033333333, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:00.488665485     1 0x570cf09b9a40 DEBUG         GST_SCHEDULING gstpad.c:4449:gst_pad_chain_data_unchecked:<fakesink0:sink> called chainfunction &gst_base_sink_chain with buffer 0x570cf09d2b40, returned ok

Logs when config-file-path is not specified:
nvstreammux-no-config.log (2.4 MB)
All buffers reach fakesink:

0:00:01.322831795     1 0x629873576980 DEBUG         GST_SCHEDULING gstpad.c:4443:gst_pad_chain_data_unchecked:<fakesink0:sink> calling chainfunction &gst_base_sink_chain with buffer buffer: 0x629873596d80, pts 0:00:06.266666604, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:01.322882590     1 0x629873576980 DEBUG         GST_SCHEDULING gstpad.c:4449:gst_pad_chain_data_unchecked:<fakesink0:sink> called chainfunction &gst_base_sink_chain with buffer 0x629873596d80, returned ok
0:00:01.328078046     1 0x629873576980 DEBUG         GST_SCHEDULING gstpad.c:4443:gst_pad_chain_data_unchecked:<fakesink0:sink> calling chainfunction &gst_base_sink_chain with buffer buffer: 0x629873596c60, pts 0:00:06.299999937, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:00:01.328110948     1 0x629873576980 DEBUG         GST_SCHEDULING gstpad.c:4449:gst_pad_chain_data_unchecked:<fakesink0:sink> called chainfunction &gst_base_sink_chain with buffer 0x629873596c60, returned ok

[tomskih_pa]

@tomskih_pa

@junshengy I also faced this issue. Pipeline works when config-file-path is not set, but hangs when I pass a config file to config-file-path.

Looks like this was fixed in Deepstream 7.0.

[junshengy]

You can open a new topic for follow-up discussion.

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks