Nvstreamdemux linked with multiple tee pads fail

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU - Tesla T4
• DeepStream Version 5.0.1
• JetPack Version (valid for Jetson only)
• TensorRT Version The latest version with Deeepstream sdk 5.0.1
• NVIDIA GPU Driver Version (valid for GPU only) 440.33.01

The following pipeline fails for a custom application.

uridecodebin -> nvstreammux -> pgie -> sgie -> tracker -> nvideoconverter(NV12) -> tee -> (queue1, queue2, queue3)
queue1 -> fakesink 
queue2 -> appsink
queue3-> demux -> vidconv(RGBA)-> nvosd -> nvvidconv(I420) -> encoder(H264) -> h264parser -> flvmux -> rtmp sink.

Error
[I 210208 13:32:07 GstPipeline:291] Finished building pipeline

[I 210208 13:32:07 GstPipeline:305] Starting pipeline
0:00:00.083489768    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.083513540    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe minimum capture size for pixelformat YM12
0:00:00.083529940    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.083541394    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe maximum capture size for pixelformat YM12
0:00:00.083553893    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x2fba370 Failed to determine interlace mode
0:00:00.083576086    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.083586055    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe minimum capture size for pixelformat NM12
0:00:00.083593011    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.083600933    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe maximum capture size for pixelformat NM12
0:00:00.083613152    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x2fba370 Failed to determine interlace mode
0:00:00.083661849    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<encoder:src> Unable to try format: Unknown error -1
0:00:00.083672707    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<encoder:src> Could not probe minimum capture size for pixelformat H264
0:00:00.083689390    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<encoder:src> Unable to try format: Unknown error -1
0:00:00.083701506    88      0x2fcfb30 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<encoder:src> Could not probe maximum capture size for pixelformat H264
0:00:00.602266824    88      0x1bc8cf0 WARN              aggregator gstaggregator.c:1717:gst_aggregator_query_latency_unlocked:<muxerflv> Latency query failed
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
0:00:02.302923983    88      0x2fcfb30 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<nvinfer1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1701> [UID = 2]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-5.0/sources/deepstream-python-pipeline-structuring/fd_lpd_model/fd_lpd.caffemodel_b1_gpu0_fp32.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [FullDims Engine Info]: layers num: 3
0   INPUT  kFLOAT data            3x270x480       min: 1x3x270x480     opt: 1x3x270x480     Max: 1x3x270x480     
1   OUTPUT kFLOAT output_cov      4x17x30         min: 0               opt: 0               Max: 0               
2   OUTPUT kFLOAT output_bbox     16x17x30        min: 0               opt: 0               Max: 0               

0:00:02.303018837    88      0x2fcfb30 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<nvinfer1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1805> [UID = 2]: Use deserialized engine model: /opt/nvidia/deepstream/deepstream-5.0/sources/deepstream-python-pipeline-structuring/fd_lpd_model/fd_lpd.caffemodel_b1_gpu0_fp32.engine
0:00:02.303819706    88      0x2fcfb30 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<nvinfer1> [UID 2]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/sources/deepstream-python-pipeline-structuring/pgie_config.ini sucessfully
0:00:02.303892993    88      0x2fcfb30 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<nvinfer0> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1715> [UID = 1]: Trying to create engine from model files
Warning, setting batch size to 1. Update the dimension after parsing due to using explicit batch size.
INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:39 [TRT]: Reading Calibration Cache for calibrator: EntropyCalibration2
INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:39 [TRT]: Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales.
INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:39 [TRT]: To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache.
INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:39 [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[W 210208 13:32:11 GstPipeline:907] Timed out for camera: 0
[W 210208 13:32:13 GstPipeline:907] Timed out for camera: 0
[W 210208 13:32:15 GstPipeline:907] Timed out for camera: 0
INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:39 [TRT]: Detected 1 inputs and 2 output network tensors.
0:00:09.718788308    88      0x2fcfb30 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<nvinfer0> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1748> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-5.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [FullDims Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x368x640       min: 1x3x368x640     opt: 1x3x368x640     Max: 1x3x368x640     
1   OUTPUT kFLOAT conv2d_bbox     16x23x40        min: 0               opt: 0               Max: 0               
2   OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40         min: 0               opt: 0               Max: 0               

0:00:09.721892261    88      0x2fcfb30 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<nvinfer0> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/sources/deepstream-python-pipeline-structuring/dstest_imagedata_config.ini sucessfully
0:00:09.722281864    88      0x2fcfb30 WARN                 basesrc gstbasesrc.c:3583:gst_base_src_start_complete:<source> pad not activated yet
[D 210208 13:32:16 GstPipeline:649] Decodebin child added:source
[D 210208 13:32:16 GstPipeline:649] Decodebin child added:decodebin0
0:00:09.723238917    88      0x2fcfb30 WARN                 basesrc gstbasesrc.c:3583:gst_base_src_start_complete:<source> pad not activated yet
[D 210208 13:32:16 GstPipeline:649] Decodebin child added:qtdemux0
0:00:09.742045274    88 0x7fec18071630 WARN                 qtdemux qtdemux_types.c:233:qtdemux_type_get: unknown QuickTime node type Xtra
0:00:09.742094757    88 0x7fec18071630 WARN                 qtdemux qtdemux.c:3031:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:09.742154337    88 0x7fec18071630 WARN                 qtdemux qtdemux.c:3031:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 2
0:00:09.758696370    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758716456    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MJPG
0:00:09.758728796    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758739615    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MJPG
0:00:09.758762389    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758776361    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MPG4
0:00:09.758788026    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758798942    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MPG4
0:00:09.758820072    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758834083    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat H265
0:00:09.758848258    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758860495    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat H265
0:00:09.758882978    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758906903    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat H264
0:00:09.758919280    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
0:00:09.758930886    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat H264
0:00:09.758968611    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:09.758978469    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2921:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat NM12
0:00:09.758994831    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:3035:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:09.759004358    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2927:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat NM12
0:00:09.759023173    88 0x7fec0c00ced0 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7fec0001f5a0 Failed to determine interlace mode
0:00:09.865657131    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865684033    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865713106    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865721797    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865744085    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865752179    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865786637    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865800349    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865831736    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865843001    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865875296    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.865887207    88 0x7fec0c00ced0 ERROR                   v4l2 gstv4l2object.c:2074:gst_v4l2_object_get_interlace_mode: Driver bug detected - check driver with v4l2-compliance from http://git.linuxtv.org/v4l-utils.git

0:00:09.867921021    88 0x7fec0c00ced0 WARN            v4l2videodec gstv4l2videodec.c:1614:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:09.867958674    88 0x7fec0c00ced0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1057:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:09.868593630    88 0x7fec0032ba30 WARN          v4l2bufferpool gstv4l2bufferpool.c:1503:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
KLT Tracker Init
[W 210208 13:32:17 GstPipeline:907] Timed out for camera: 0

The application works if, there’s only a single source pad from tee to demux.

uridecodebin -> nvstreammux -> pgie -> sgie -> tracker -> nvideoconverter(NV12) -> tee -> (queue1)
queue1-> demux -> vidconv(RGBA)-> nvosd -> nvvidconv(I420) -> encoder(H264) -> h264parser -> flvmux -> rtmp sink.

It also works if the demux and sinkbin are replaced with a fakesink instead

uridecodebin -> nvstreammux -> pgie -> sgie -> tracker -> nvideoconverter(NV12) -> tee -> (queue1, queue2, queue3)
queue1 -> fakesink 
queue2 -> appsink
queue3-> fakesink

Where am i going wrong ?

Hello @Fiona.Chen apologies for tagging you out of the blue, was wondering if there’s any insight you could provide

Can you give us the gst-launch command of your pipeline? Or can you send us your code to reproduce the problem?