The number of supported RTSP streams decreases when the pipeline has many branches and adds many queues

noblehill · September 1, 2022, 8:15am

• GPU
• DeepStream6.1
• TensorRT 8.2.5.1
• 3080 NVIDIA Corporation Device 2216 (rev a1)
• bugs
• Now I have a pipeline project that NvStreamDemux after Nvinfer, but after adding queue to each branch, the number of pipeline supports decreases
**• **
1.Here’s my pipe diagram without a queue

2.Here’s my pipeline diagram for adding queue

I want to know why and fix it。

fanzh · September 1, 2022, 8:41am

which deepstream sample are you testing? how to reproduce this issue?
what will happen if add 25 RTSP source and queue? could you share the log?

noblehill · September 1, 2022, 8:47am

I wonder if there is a limit to the number of components in a pipe, or a limit to the number of threads?

fanzh · September 1, 2022, 8:57am

please refer to source30_1080p_dec_infer-resnet_tiled_display_int8.txt, deepstream can support 30 inputs, there is no limit to the number of components in a pipe if machine resource is enough.

noblehill · September 1, 2022, 8:57am

I modified the program based on test-3. When adding 25 RTSP sources and queues, the pipeline blocked after dynamically loading URI_DECODE_bin related components，
log:

0:00:05.543722653 5438 0x5594e132c4f0 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1900> [UID = 1]: deserialized trt engine from :/home/dushan/Git/Deeepstream_C_demo/configs/models/yolov5_Detector/mutilModelsA_25.engine
INFO: …/nvdsinfer/nvdsinfer_model_builder.cpp:610 [FullDims Engine Info]: layers num: 4
0 INPUT kFLOAT images 3x384x640 min: 1x3x384x640 opt: 8x3x384x640 Max: 25x3x384x640
1 OUTPUT kFLOAT output1 3x12x20x28 min: 0 opt: 0 Max: 0
2 OUTPUT kFLOAT output2 3x24x40x28 min: 0 opt: 0 Max: 0
3 OUTPUT kFLOAT output3 3x48x80x28 min: 0 opt: 0 Max: 0

0:00:05.572626104 5438 0x5594e132c4f0 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2003> [UID = 1]: Use deserialized engine model: /home/dushan/Git/Deeepstream_C_demo/configs/models/yolov5_Detector/mutilModelsA_25.engine
0:00:05.614027174 5438 0x5594e132c4f0 INFO nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus: [UID 1]: Load new model:/home/dushan/Git/Deeepstream_C_demo/configs/models/yolov5_Detector/config_infer_primary_yolov5s_copy.txt sucessfully
[2022-09-01 08:56:28.186][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.186][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.187][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.188][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.189][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.190][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.191][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.192][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.193][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.194][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.195][INFO ][AIStreamTask.cpp:1455]Decodebin child added:source

[2022-09-01 08:56:28.199][INFO ][AIStreamTask.cpp:787]Running…
[2022-09-01 08:56:28.199][INFO ][AIProcessFrame.cpp:32]AIProcessFrame::Init begin
[2022-09-01 08:56:28.200][INFO ][AIProcessFrame.cpp:50]AIProcessFrame::Init end
[2022-09-01 08:56:28.200][INFO ][AIProcessFrame.cpp:67]AIProcessFrame::TaskThread begin
[2022-09-01 08:56:30.228][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin0

[2022-09-01 08:56:30.229][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin1

[2022-09-01 08:56:30.230][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin2

[2022-09-01 08:56:30.231][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin3

[2022-09-01 08:56:30.231][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin4

[2022-09-01 08:56:30.233][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay0

[2022-09-01 08:56:30.233][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay1

[2022-09-01 08:56:30.234][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay2

[2022-09-01 08:56:30.234][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin5

[2022-09-01 08:56:30.235][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin6

[2022-09-01 08:56:30.235][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin7

[2022-09-01 08:56:30.235][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay3

[2022-09-01 08:56:30.236][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay4

[2022-09-01 08:56:30.236][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse0

[2022-09-01 08:56:30.236][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse1

[2022-09-01 08:56:30.236][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse2

[2022-09-01 08:56:30.237][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin8

[2022-09-01 08:56:30.237][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter0

[2022-09-01 08:56:30.237][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse3

[2022-09-01 08:56:30.237][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter1

[2022-09-01 08:56:30.237][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter2

[2022-09-01 08:56:30.237][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse4

[2022-09-01 08:56:30.238][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter3

[2022-09-01 08:56:30.238][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter4

[2022-09-01 08:56:30.239][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay5

[2022-09-01 08:56:30.239][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay6

[2022-09-01 08:56:30.239][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse5

[2022-09-01 08:56:30.240][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay7

[2022-09-01 08:56:30.240][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter5

[2022-09-01 08:56:30.240][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse6

[2022-09-01 08:56:30.241][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse7

[2022-09-01 08:56:30.241][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter6

[2022-09-01 08:56:30.241][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay8

[2022-09-01 08:56:30.242][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter7

[2022-09-01 08:56:30.243][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse8

[2022-09-01 08:56:30.243][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter8

[2022-09-01 08:56:30.247][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin9

[2022-09-01 08:56:30.250][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin10

[2022-09-01 08:56:30.251][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay9

[2022-09-01 08:56:30.253][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse9

[2022-09-01 08:56:30.253][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter9

[2022-09-01 08:56:30.255][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder0

[2022-09-01 08:56:30.255][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay10

[2022-09-01 08:56:30.255][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder1

[2022-09-01 08:56:30.256][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse10

[2022-09-01 08:56:30.256][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter10

[2022-09-01 08:56:30.257][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder2

[2022-09-01 08:56:30.258][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder3

[2022-09-01 08:56:30.260][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder4

[2022-09-01 08:56:30.260][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder5

[2022-09-01 08:56:30.261][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder6

[2022-09-01 08:56:30.262][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder7

[2022-09-01 08:56:30.264][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder8

[2022-09-01 08:56:30.275][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder9

[2022-09-01 08:56:30.275][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder10

[2022-09-01 08:56:30.281][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin11

[2022-09-01 08:56:30.282][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin12

[2022-09-01 08:56:30.288][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay11

[2022-09-01 08:56:30.289][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse11

[2022-09-01 08:56:30.289][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay12

[2022-09-01 08:56:30.289][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter11

[2022-09-01 08:56:30.291][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse12

[2022-09-01 08:56:30.292][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter12

[2022-09-01 08:56:30.307][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin13

[2022-09-01 08:56:30.307][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin14

[2022-09-01 08:56:30.307][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin15

[2022-09-01 08:56:30.311][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin16

[2022-09-01 08:56:30.315][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin17

[2022-09-01 08:56:30.318][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin18

[2022-09-01 08:56:30.318][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay13

[2022-09-01 08:56:30.318][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay14

[2022-09-01 08:56:30.318][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay15

[2022-09-01 08:56:30.318][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder11

[2022-09-01 08:56:30.321][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse13

[2022-09-01 08:56:30.321][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay16

[2022-09-01 08:56:30.321][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse14

[2022-09-01 08:56:30.321][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse15

[2022-09-01 08:56:30.321][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay17

[2022-09-01 08:56:30.321][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter13

[2022-09-01 08:56:30.321][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter14

[2022-09-01 08:56:30.322][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin19

[2022-09-01 08:56:30.325][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter15

[2022-09-01 08:56:30.325][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder12

[2022-09-01 08:56:30.325][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse16

[2022-09-01 08:56:30.325][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay18

[2022-09-01 08:56:30.325][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse17

[2022-09-01 08:56:30.326][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter16

[2022-09-01 08:56:30.326][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse18

[2022-09-01 08:56:30.326][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin20

[2022-09-01 08:56:30.329][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter17

[2022-09-01 08:56:30.329][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter18

[2022-09-01 08:56:30.329][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay19

[2022-09-01 08:56:30.329][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse19

[2022-09-01 08:56:30.332][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter19

[2022-09-01 08:56:30.333][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin21

[2022-09-01 08:56:30.338][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay20

[2022-09-01 08:56:30.341][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse20

[2022-09-01 08:56:30.341][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin22

[2022-09-01 08:56:30.341][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter20

[2022-09-01 08:56:30.341][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin23

[2022-09-01 08:56:30.342][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay21

[2022-09-01 08:56:30.347][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse21

[2022-09-01 08:56:30.347][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay22

[2022-09-01 08:56:30.348][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter21

[2022-09-01 08:56:30.348][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse22

[2022-09-01 08:56:30.349][INFO ][AIStreamTask.cpp:1455]Decodebin child added:decodebin24

[2022-09-01 08:56:30.349][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay23

[2022-09-01 08:56:30.350][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter22

[2022-09-01 08:56:30.352][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse23

[2022-09-01 08:56:30.355][INFO ][AIStreamTask.cpp:1455]Decodebin child added:rtph264depay24

[2022-09-01 08:56:30.356][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter23

[2022-09-01 08:56:30.356][INFO ][AIStreamTask.cpp:1455]Decodebin child added:h264parse24

[2022-09-01 08:56:30.356][INFO ][AIStreamTask.cpp:1455]Decodebin child added:capsfilter24

[2022-09-01 08:56:30.359][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder13

[2022-09-01 08:56:30.359][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder14

[2022-09-01 08:56:30.364][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder15

[2022-09-01 08:56:30.364][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder16

[2022-09-01 08:56:30.364][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder17

[2022-09-01 08:56:30.364][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder18

[2022-09-01 08:56:30.364][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder19

[2022-09-01 08:56:30.407][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder20

[2022-09-01 08:56:30.415][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder21

[2022-09-01 08:56:30.428][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.476][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.512][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.542][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.548][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.556][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.586][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.626][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.641][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.647][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.693][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.716][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder22

[2022-09-01 08:56:30.722][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.745][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.759][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.797][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.807][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.813][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder23

[2022-09-01 08:56:30.815][INFO ][AIStreamTask.cpp:1455]Decodebin child added:nvv4l2decoder24

[2022-09-01 08:56:30.835][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.836][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.927][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.933][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.944][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.945][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.949][INFO ][AIStreamTask.cpp:1423]In cb_newpad

[2022-09-01 08:56:30.966][INFO ][AIStreamTask.cpp:1423]In cb_newpad

noblehill · September 1, 2022, 9:00am

I can support 25 or even 36 paths when I don’t add a queue but I can’t support more than that, I want to support more paths

noblehill · September 1, 2022, 9:05am

Because I have to do something special with each stream, I have to use demux after the inference and process it separately, so I can’t follow the example. The C++ development used for the project is not using the command line or scripts.

fanzh · September 1, 2022, 9:32am

what is your GPU model? from the log “Decodebin child added:nvv4l2decoder”, Decodebin will use hareware decoding, but hardware has resource limit, please find your GPU 's ability in Video Encode and Decode GPU Support Matrix | NVIDIA Developer

noblehill · September 1, 2022, 9:49am

RTX 3080
NVIDIA Corporation Device 2216 (rev a1)
Three identical program examples on one card, I don’t think it is due to decoding hardware resources。

noblehill · September 1, 2022, 9:52am

You can see the difference in the two pipeline diagrams in the topic, except that the two components are connected to a queue, which I think has nothing to do with the codec capability

fanzh · September 1, 2022, 10:32am

about " I want to support more paths", it will be limited by GPU 's ability.
did you use the same RTSP source or 25 different RTSP sources? to narrow down this issue, please simplify the code, for example: remove nvinfer to test, add queue one by one to test, by this way, find which snippet will cause the issue.
please provide simplified code to reproduce this issue.

noblehill · September 1, 2022, 10:53am

1.I can run 25 routes without adding queue, and can only support 20 routes after adding queue。
2.Using the same RTSP source, distributed from streaming media, distributed streams can easily support thousands of routes. Don’t worry about that。
This is not related to nvinfer and I can tell you before we infer is decode_bin which we can’t debug.
3.I will try to provide a code to reproduce the problem。

noblehill · September 2, 2022, 3:45am

Hi, I’m providing a simple example of a repeatable C++ program, but you’ll need to provide your own RTSP streams that support 25 concurrent streams, or you’ll need to modify your own input code, which can be tested in deepstream5.1 or above. This is a pipe with a lot of components removed, and I added a bunch of queues is because my pipe is longer than that, so both components are connected to queues, and I’m just deleting these components and keeping the queue.
Now the pipeline is simple: Decodebin-Mux-Demux-Fakesink
Start parameters: 1. RTSP URL (25 pulls of the same) 2. Number of groups inserted into the queue (you can modify the number of groups inserted to replicate the pipeline blocking problem)

#include <gst/gst.h>
#include <glib.h>
#include <stdio.h>
#include <math.h>
#include <string.h>
#include <sys/time.h>
#include "gstnvdsmeta.h"
#include "gst-nvmessage.h"
#include <stdio.h>
#include <string>
#include <string.h>
using namespace std;
#define GST_CAPS_FEATURES_NVMM									"memory:NVMM"
#define NVDS_ELEM_SRC_URI "uridecodebin"
#define NVDS_ELEM_DECODEBIN "decodebin"
#define NVDS_ELEM_QUEUE "queue"
#define NVDS_ELEM_CAPS_FILTER "capsfilter"
#define NVDS_ELEM_PGIE "nvinfer"
#define NVDS_ELEM_VIDEO_CONV "nvvideoconvert"
#define NVDS_ELEM_STREAM_MUX "nvstreammux"
#define NVDS_ELEM_STREAM_DEMUX "nvstreamdemux"
#define NVDS_ELEM_OSD "nvdsosd"
#define NVDS_ELEM_SINK_FAKESINK "fakesink"
#define MUXER_OUTPUT_WIDTH 1920
#define MUXER_OUTPUT_HEIGHT 1080

/* Muxer batch formation timeout, for e.g. 40 millisec. Should ideally be set
 * based on the fastest source's framerate. */
#define MUXER_BATCH_TIMEOUT_USEC 40000

static GMainLoop *m_pLoop          		= NULL;
static GstElement *pipeline				= NULL;
static GstElement *m_pStreamMux			= NULL;
static GstElement *m_pStreamDemux			= NULL;
static GstElement *m_pStreamdemuxNext		= NULL;
static int m_nNumSources			        = 0;
static int m_nQueueid						= 0;
static int m_nGroupCount					= 0;
static bool m_bQueue						=false;

static GstPadProbeReturn fakesink_src_pad_buffer_probe(GstPad *pad, GstPadProbeInfo* info, gpointer u_data)
{
	printf("data come\n");
	return GST_PAD_PROBE_OK;
}

static void on_pad_added(GstElement *element, GstPad *pad, gpointer data)
{
	GstPad *sinkpad;
	GstElement *pGstElement = (GstElement *)data;

	/* We can now link this pad with the its corresponding sink pad */
	printf("Dynamic pad created, linking demuxer and other\n");
	sinkpad = gst_element_get_static_pad(pGstElement, "sink");
	gst_pad_link(pad, sinkpad);
	gst_object_unref(sinkpad);
}

static void cb_newpad(GstElement * decodebin, GstPad * decoder_src_pad, gpointer data)
{
	printf("In cb_newpad\n");
	GstCaps *caps = gst_pad_get_current_caps(decoder_src_pad);
	const GstStructure *str = gst_caps_get_structure(caps, 0);
	const gchar *name = gst_structure_get_name(str);
	GstElement *source_bin = (GstElement *)data;

	GstCapsFeatures *features = gst_caps_get_features(caps, 0);

	if (!strncmp(name, "video", 5)) {
	
		if (gst_caps_features_contains(features, GST_CAPS_FEATURES_NVMM)) {
			/* Get the source bin ghost pad */

			GstPad *bin_ghost_pad = gst_element_get_static_pad(source_bin, "src");
			if (!gst_ghost_pad_set_target(GST_GHOST_PAD(bin_ghost_pad),
				decoder_src_pad)) {
				printf("Failed to link decoder src pad to source bin ghost pad\n");
			}
			gst_object_unref(bin_ghost_pad);
		}
		else {
			printf("Error: Decodebin did not pick nvidia decoder plugin.\n");
		}
	}
}

static void decodebin_child_added(GstChildProxy * child_proxy, GObject * object, gchar * name, gpointer user_data)
{
	printf("Decodebin child added:%s\n", name);
	if (g_strrstr(name, "decodebin") == name)
	{
		g_signal_connect(G_OBJECT(object), "child-added", G_CALLBACK(decodebin_child_added), user_data);
	}
}
int DemuxQueue()
{
    m_nQueueid++;
	gchar elem_name[16] = { 0 };
	g_snprintf(elem_name, 15, "queue%d", m_nQueueid);
	GstElement *queue = gst_element_factory_make("queue", elem_name);
	gst_bin_add(GST_BIN(pipeline), queue);
	m_pStreamdemuxNext = queue;
	return TRUE;
}

int PipelineAddQueue()
{
    if(m_bQueue)
    {
        m_nQueueid++;
	    gchar elem_name[16] = { 0 };
	    g_snprintf(elem_name, 15, "queue%d", m_nQueueid);
	    GstElement *queue = gst_element_factory_make("queue", elem_name);
	    gst_bin_add(GST_BIN(pipeline), queue);
	    
    }
	return TRUE;
}

GstElement* StreamOpen(string uri,int nStreamID)
{
	printf("StreamOpen begin uri:%s\n", uri.c_str());
	GstElement *uri_decode_bin = NULL;
	gchar bin_name[32] = { };

	g_snprintf(bin_name, 31, "uri-decode-bin%d", nStreamID);
	printf("bin_name:%s\n", bin_name);
	uri_decode_bin = gst_element_factory_make("uridecodebin", bin_name);

	if (!uri_decode_bin) {
		printf("uri_decode_bin could not be created\n");
		return NULL;
	}
	
	g_object_set(G_OBJECT(uri_decode_bin), "uri", uri.c_str(), NULL);

	printf("StreamOpen end\n");
	return uri_decode_bin;
}

GstPad* StreamDecode(GstElement *uri_decode_bin, int nStreamID)
{
	printf("StreamDecode begin\n");
	GstPad *srcpad = NULL;
	GstElement *decode_bin = NULL;
	gchar bin_name[16] = { };

	g_snprintf(bin_name, 15, "decode-bin-%d", nStreamID);
	
	decode_bin = gst_bin_new(bin_name);

	if (!decode_bin) {
		printf("decode_bin could not be created.\n");
		return NULL;
	}
	
	g_signal_connect(G_OBJECT(uri_decode_bin), "pad-added", G_CALLBACK(cb_newpad), decode_bin);
	g_signal_connect(G_OBJECT(uri_decode_bin), "child-added", G_CALLBACK(decodebin_child_added), decode_bin);

	gst_bin_add(GST_BIN(decode_bin), uri_decode_bin);

	if (!gst_element_add_pad(decode_bin, gst_ghost_pad_new_no_target("src", GST_PAD_SRC)))
	{
		printf("Failed to add ghost pad in decode_bin\n");
		return NULL;
	}

	if (!decode_bin) {
		printf("Failed to create decode_bin. Exiting.\n");
		return NULL;
	}

	gst_bin_add(GST_BIN(pipeline), decode_bin);
	srcpad = gst_element_get_static_pad(decode_bin, "src");
	if (!srcpad) {
		printf("Failed to get src pad of source bin. Exiting.\n");
		return NULL;
	}
	printf("StreamDecode end\n");
	return srcpad;
}

bool Init(int nNumSize,string strPipelineName)
{
	
	m_nNumSources 	 = nNumSize;
	m_pLoop = g_main_loop_new(NULL, FALSE);
	pipeline = gst_pipeline_new(strPipelineName.c_str());
	if (!pipeline ) {
		printf("pipeline could not be created\n");
		return FALSE;
	}

	m_pStreamMux = gst_element_factory_make(NVDS_ELEM_STREAM_MUX, "stream-muxer");
	if (!m_pStreamMux) {
		printf("m_pStreamMux could not be created\n");
		return FALSE;
	}
	gst_bin_add(GST_BIN(pipeline), m_pStreamMux);
	g_object_set(G_OBJECT(m_pStreamMux), "batch-size", m_nNumSources, NULL);
	g_object_set(G_OBJECT(m_pStreamMux), "live-source", 1, NULL);
	g_object_set(G_OBJECT(m_pStreamMux), "width", MUXER_OUTPUT_WIDTH, "height",
		MUXER_OUTPUT_HEIGHT,
		"batched-push-timeout", MUXER_BATCH_TIMEOUT_USEC, NULL);

	return true;
}


int StreammuxSinkPadToSrcPad(string strUri,int nStreamID)
{
	gchar pad_name[16] = { };
	g_snprintf(pad_name, 15, "sink_%u", nStreamID);
	GstElement *uri_decode_bin = StreamOpen(strUri, nStreamID);
		
	GstPad *srcpad = StreamDecode(uri_decode_bin, nStreamID);
	if (NULL == srcpad)
	{
		printf("StreamDecode failed\n");
		return FALSE;
	}

	GstPad * sinkpad = gst_element_get_request_pad(m_pStreamMux, pad_name);
	if (!sinkpad) {
		printf("Streammux request sink pad failed. Exiting.\n");
		return FALSE;
	}

	if (gst_pad_link(srcpad, sinkpad) != GST_PAD_LINK_OK) {
		printf("Failed to link source bin to stream muxer. Exiting.\n");
		return FALSE;
	}
	gst_object_unref(srcpad);
	gst_object_unref(sinkpad);
	return TRUE;
}


int StreamdemmuxInit()
{
	PipelineAddQueue();
	m_pStreamDemux = gst_element_factory_make(NVDS_ELEM_STREAM_DEMUX, "stream-demuxer");
	gst_bin_add(GST_BIN(pipeline), m_pStreamDemux);
	return TRUE;
}

int Streamdemmux(int nStreamID)
{
	gchar demux_src_pad_name[16] = { };
	g_snprintf(demux_src_pad_name, 15, "src_%d", nStreamID);

	GstPad * demux_src_pad = gst_element_get_request_pad(m_pStreamDemux, demux_src_pad_name);
	if (!demux_src_pad) {
		printf("Streamdemux request src pad failed. Exiting.\n");
		return FALSE;
	}
	printf("get demux_src_pad_name:%s\n", demux_src_pad_name);
	GstPad *demux_src_pad1 = gst_element_get_static_pad(m_pStreamDemux, demux_src_pad_name);
	if (NULL == demux_src_pad1)
	{
		printf("get demux_src_pad failed\n");
	}
	GstPad *sink_pad = gst_element_get_static_pad(m_pStreamdemuxNext, "sink");
	
	if (NULL == sink_pad)
	{
		printf("get  sink_pad failed\n");
	}
		
	// link the pads
	if (gst_pad_link(demux_src_pad1, sink_pad) != GST_PAD_LINK_OK) {
		gchar *n1 = gst_pad_get_name(demux_src_pad1);
		gchar *n2 = gst_pad_get_name(sink_pad);
		printf("Failed to link '%s' and '%s'\n", n1, n2);
		g_free(n1);
		g_free(n2);
	
	}

	gst_object_unref(demux_src_pad1);
	gst_object_unref(sink_pad);
	gst_object_unref(demux_src_pad);
	g_signal_connect(m_pStreamDemux, "pad-added", G_CALLBACK(on_pad_added), m_pStreamdemuxNext);
	return TRUE;
}

int GetInferenceData()
{
	for (int i =0;i< m_nNumSources;i++)
	{
		gchar elem_name[32] = { 0 };
		g_snprintf(elem_name, 31, "fakesink%d", i);
		GstElement *fakesink = gst_bin_get_by_name(GST_BIN(pipeline), elem_name);//获取osd元组件对象
		GstPad *fakesink_sink_pad = gst_element_get_static_pad(fakesink, "sink");
		if (!fakesink_sink_pad)
			printf("Unable to get nvosd_src_pad\n");
		else
			gst_pad_add_probe(fakesink_sink_pad, GST_PAD_PROBE_TYPE_BUFFER,
				fakesink_src_pad_buffer_probe, NULL, NULL);
		gst_object_unref(fakesink_sink_pad);
		
	}

	return TRUE;
}

int ElementLink()
{
	GstIterator* pGstIterator = gst_bin_iterate_elements(GST_BIN(pipeline));
	GValue item = { 0, };

	GstElement *element[pGstIterator->size];
	int nsize = pGstIterator->size;
	printf("nsize:%d\n",nsize);
	int i = -1;
	while (nsize--)
	{
		i++;
		if (GST_ITERATOR_OK == gst_iterator_next(pGstIterator, &item))
		{
			element[i] = (GstElement *)g_value_get_object(&item);
			char *pEnd = strstr(element[i]->object.name, "decode-bin");

			if (NULL != pEnd)
			{
				printf("find the decode element names:%s\n", element[i]->object.name);
				i--;
				continue;
			}

			printf("i:%d,element names:%s\n",i, element[i]->object.name);

		}
		else
		{
			break;
		}
	}

	printf("PipelineRun begin\n");
	while (i > 2)
	{
		i--;
		char *pEnd = strstr(element[i]->object.name, "stream-demuxer");				
		if (NULL != pEnd)
		{
			printf("find the element[%d] names:%s\n", i, element[i]->object.name);//分流器管道元件
			continue;
		}
		pEnd = NULL;
		pEnd = strstr(element[i]->object.name, "udp-sink");
		if (NULL != pEnd)
		{
			printf("find the element[%d] names:%s\n", i, element[i]->object.name);
			continue;
		}
		pEnd = NULL;
		pEnd = strstr(element[i]->object.name, "fakesink");
		if (NULL != pEnd)
		{
			printf("find the element[%d] names:%s\n", i, element[i]->object.name);
			continue;
		}
		pEnd = NULL;
		pEnd = strstr(element[i]->object.name, "tee");
		if (NULL != pEnd)
		{
			printf("find the element[%d] names:%s\n", i, element[i]->object.name);
			continue;
		}
		printf("element[%d] names:%s,element[%d] names:%s\n", i, element[i]->object.name, i - 1, element[i - 1]->object.name);


		if (!gst_element_link_many(element[i],element[i - 1], NULL))
		{

			printf("elements link failure\n");
			return FALSE;
		}

	}
	return TRUE;
}
static gboolean bus_call(GstBus * bus, GstMessage * msg, gpointer data)
{
	return TRUE;
}
int MessageHandler()
{
	GstBus *bus = NULL;
	bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline));
	gst_bus_add_watch(bus, bus_call, NULL);
	gst_object_unref(bus);
	return TRUE;
}

int PipelineRun()
{
	printf("PipelineRun start\n");
	if (TRUE != ElementLink())
	{
		return FALSE;
	}
	printf("ElementLink end\n");
	gst_element_set_state(pipeline, GST_STATE_PLAYING);
	GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pipeline");
	printf("Running...\n");
	g_main_loop_run(m_pLoop);
	return TRUE;
}


int StreamFakesink(int nStreamID)
{
	int nGroupCount = m_nGroupCount;
	gchar elem_name[16] = { 0 };
	g_snprintf(elem_name, 15, "fakesink%d", nStreamID);
	GstElement *fakesink = gst_element_factory_make("fakesink",elem_name);
	if (!fakesink)
	{
	printf("One element( !fakesink ) could not be created. Exiting\n");
	return -1;
	}
	while(m_nGroupCount>0)
	{
		m_nGroupCount--;
		PipelineAddQueue();
	}
	m_nGroupCount = nGroupCount;
	if(!m_bQueue)
    {
		m_pStreamdemuxNext = fakesink;
	}
	g_object_set (G_OBJECT (fakesink), "sync", 0, "async", false,NULL);
	g_object_set (G_OBJECT (fakesink), "qos", 0, NULL);
	gst_bin_add(GST_BIN(pipeline), fakesink);
}

int
main (int argc, char *argv[])
{
	if(argc<2)
	{
		printf("Startup parameter error\n");
	}
	string strRtspUrl   = argv[1];
	m_nGroupCount	 = atoi(argv[2]);
	int nDeviceNum 	 = 25;
	m_bQueue	  	 = true;
	printf("strRtspUrl:%s",strRtspUrl.c_str());
	printf("nDeviceNum:%d,m_bQueue:%d,m_nGroupCount:%d\n",nDeviceNum,m_bQueue,m_nGroupCount);
	g_setenv("GST_DEBUG_DUMP_DOT_DIR", "./", TRUE);
	gst_init(&argc, &argv);
	Init(nDeviceNum, "Test");
	
    for (unsigned int nStreamID = 0; nStreamID < nDeviceNum; nStreamID++)
	{
		StreammuxSinkPadToSrcPad(strRtspUrl,nStreamID);
	}
	StreamdemmuxInit();
	for (unsigned int nStreamID = 0; nStreamID < nDeviceNum; nStreamID++)
	{
		if(m_bQueue)
		{	
			DemuxQueue();
			Streamdemmux(nStreamID);
			StreamFakesink(nStreamID);
		}else
		{
			StreamFakesink(nStreamID);
			Streamdemmux(nStreamID);
		}
	    
	}
	MessageHandler();
	GetInferenceData();
	PipelineRun();
	
	return 0;
}

noblehill · September 5, 2022, 1:51am

Hello, are you still paying attention to this issue or have you made any progress?

fanzh · September 5, 2022, 2:03am

I can run your code, how do you know the pipeline blocked?

noblehill · September 5, 2022, 2:07am

I looked at the log printing at GST_DEBUG=4 and no error was thrown, nor was there any log printing after decodebin2

fanzh · September 5, 2022, 2:42am

about “nor was there any log printing after decodebin2”, do you mean there was no any log after a while? I can’t reproduce this issue on my PC.
to narrow down this issue, please simplify the code, for example: add queue one by one to test, by this way, find which snippet will cause the issue.

noblehill · September 5, 2022, 2:47am

Is there no recurrence using the code I provided?You need to change the second parameter in the input, increasing it from 0 in order to see the phenomenon.

fanzh · September 5, 2022, 2:54am

I did not modify your code, and start by export GST_DEBUG=4 && ./deepstream-preprocess-test rtsp://127.0.0.1:8554/test 25, the logs did not stop.

noblehill · September 5, 2022, 2:56am

Could you please send the pipe drawing you generated?