Nvinfer/PGIE not working on second stream. Deepstream Python

Please provide complete information as applicable to your setup.

• Hardware Platform GPU
• DeepStream Version 6.1
• TensorRT Version 8.4.3-1+cuda11.6*
• NVIDIA GPU Driver Version (valid for GPU only) 510.47.03
• Issue Type Question
• How to reproduce the issue ? Using following code to create multistream deepstream python app

for i in range(len(self.sources)):
           self.source_bin.append(self._create_source_bin(index=i))
self.streammux = self._create_streammux()

After this I’m linking them as provided in deepstream-test3 of deepstream_python_apps, just refactoring to my use case as:

for i in range(len(self.sources)):
            padname = "sink_%u" % i
            sinkpad = self.streammux.get_request_pad(padname)
            if not sinkpad:
                self.logger.error("Unable to get the sink pad of streammux")
            srcpad = self.source_bin[i].get_static_pad("src")
            if not srcpad:
                self.logger.error("Unable to get source pad of decoder")
            srcpad.link(sinkpad)

The rest of the element are linked sequentially. Following is a snippet from associated logs from app run.

INFO:src.pipeline_test.Pipeline:Linking elements in the Pipeline: source-bin-00 -> source-bin-01 -> stream-muxer -> primary-inference -> tracker -> analytics -> convertor1 -> capsfilter1 -> nvtiler -> convertor2
 -> onscreendisplay -> queue3 -> nveglglessink

INFO:src.pipeline_test.Pipeline:Decodebin child added: source
INFO:src.pipeline_test.Pipeline:Decodebin child added: decodebin0
INFO:src.pipeline_test.Pipeline:Decodebin child added: source
INFO:src.pipeline_test.Pipeline:Decodebin child added: decodebin1
INFO:src.pipeline_test.Pipeline:Decodebin child added: qtdemux0
INFO:src.pipeline_test.Pipeline:Decodebin child added: qtdemux1
INFO:src.pipeline_test.Pipeline:Decodebin child added: multiqueue0
INFO:src.pipeline_test.Pipeline:Decodebin child added: multiqueue1
INFO:src.pipeline_test.Pipeline:Decodebin pad added
INFO:src.pipeline_test.Pipeline:Decodebin pad added
* Fps of stream 0, is  15.2
* Fps of stream 1, is  15.2
...

The issue that I’m facing is that PGIE is not working on the second stream, or rather working very sparsely. I’ve tried tinkering with interval and batch-size and comparing my code with examples yet I cannot locate the issue. This is probably because the issue is in my logic. I just can’t see it and would appreciate any help or nudge in the right direction. Also attaching the pgie config that I’m using.
Thank you

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
custom-network-config=/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/ds-yati/yolor_csp.cfg
model-file=/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/ds-yati/yolor_csp_99.wts
model-engine-file=/home/lalit/pyds-yati/model_b2_gpu0_fp16.engine
#onnx-file=best.onnx
#int8-calib-file=calib.table
labelfile-path=/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/ds-yati/labels.txt
batch-size=2
network-mode=2
num-detected-classes=4
interval=2
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseYolo
custom-lib-path=/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/ds-yati/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
pre-cluster-threshold=0.4

You code may have some problem. You can add multiple source stream into one source_bin. But you create multiple source_bin which is wrong. You can refer the deepstream-test3:

    for i in range(number_sources):
        print("Creating source_bin ",i," \n ")
        uri_name=args[i]
        if uri_name.find("rtsp://") == 0 :
            is_live = True
        source_bin=create_source_bin(i, uri_name)
        if not source_bin:
            sys.stderr.write("Unable to create source bin \n")
        pipeline.add(source_bin)
        padname="sink_%u" %i
        sinkpad= streammux.get_request_pad(padname) 
        if not sinkpad:
            sys.stderr.write("Unable to create sink pad bin \n")
        srcpad=source_bin.get_static_pad("src")
        if not srcpad:
            sys.stderr.write("Unable to create src pad bin \n")
        srcpad.link(sinkpad)

I’ve tried creating the source_bin as described in the deepstream-test3 yet the issue persists. Attaching my graph for reference. FYI the issue is that nvinfer is not working on one of the streams

Also, here’s the GST_DEBUG=3 logs from pipeline. I’m not an expert but if these can point to a fix in logic, kindly help me with that. Thanks.

0:00:05.305952899 39885      0x80f9760 WARN                 basesrc gstbasesrc.c:3600:gst_base_src_start_complete:<source> pad not activated yet                                                                   
INFO:src.pipeline_test.Pipeline:Decodebin child added: source                                                                                                                                                      
INFO:src.pipeline_test.Pipeline:Decodebin child added: decodebin0                                                                                                                                                  
0:00:05.307576043 39885      0x80f9760 WARN                 basesrc gstbasesrc.c:3600:gst_base_src_start_complete:<source> pad not activated yet                                                                   
0:00:05.308002409 39885      0x80f9760 WARN                 basesrc gstbasesrc.c:3600:gst_base_src_start_complete:<source> pad not activated yet                                                                   
INFO:src.pipeline_test.Pipeline:Decodebin child added: source                                                                                                                                                      
INFO:src.pipeline_test.Pipeline:Decodebin child added: decodebin1                                                                                                                                                  
0:00:05.308475339 39885      0x80f9760 WARN                 basesrc gstbasesrc.c:3600:gst_base_src_start_complete:<source> pad not activated yet                                                                   
INFO:src.pipeline_test.Pipeline:Decodebin child added: qtdemux0                                                                                                                                                    
INFO:src.pipeline_test.Pipeline:Decodebin child added: qtdemux1                                                                                                                                                    
0:00:05.318421000 39885 0x7f7b48071cc0 WARN                 qtdemux qtdemux_types.c:239:qtdemux_type_get: unknown QuickTime node type pasp                                                                         
0:00:05.318426133 39885     0x8da78f60 WARN                 qtdemux qtdemux_types.c:239:qtdemux_type_get: unknown QuickTime node type pasp                                                                         
0:00:05.318480912 39885 0x7f7b48071cc0 WARN                 qtdemux qtdemux.c:3250:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1                                                     
0:00:05.318510822 39885     0x8da78f60 WARN                 qtdemux qtdemux.c:3250:qtdemux_parse_trex:<qtdemux1> failed to find fragment defaults for stream 1                                                     
INFO:src.pipeline_test.Pipeline:Decodebin child added: multiqueue0                                                                                                                                                 
INFO:src.pipeline_test.Pipeline:Decodebin child added: multiqueue1                                                                                                                                                 
INFO:src.pipeline_test.Pipeline:Decodebin child added: h264parse1                                                                                                                                                  
INFO:src.pipeline_test.Pipeline:Decodebin child added: h264parse0                                                                                                                                                  
INFO:src.pipeline_test.Pipeline:Decodebin child added: capsfilter0                                                                                                                                                 
INFO:src.pipeline_test.Pipeline:Decodebin child added: capsfilter1                                                                                                                                                 
INFO:src.pipeline_test.Pipeline:Decodebin child added: nvv4l2decoder0                                                                                                                                              
INFO:src.pipeline_test.Pipeline:Decodebin child added: nvv4l2decoder1                                                                                                                                              
0:00:05.325806391 39885 0x7f7b40015e40 WARN                    v4l2 gstv4l2object.c:3056:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1                             
0:00:05.325835445 39885 0x7f7b40015e40 WARN                    v4l2 gstv4l2object.c:2941:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MJPG     
0:00:05.325855682 39885 0x7f7b40015e40 WARN                    v4l2 gstv4l2object.c:3056:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1                             
0:00:05.325852577 39885 0x7f7b3c028b60 WARN                    v4l2 gstv4l2object.c:3056:gst_v4l2_object_get_nearest_size:<nvv4l2decoder1:sink> Unable to try format: Unknown error -1                             
0:00:05.325869512 39885 0x7f7b40015e40 WARN                    v4l2 gstv4l2object.c:2947:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MJPG     
0:00:05.325875738 39885 0x7f7b3c028b60 WARN                    v4l2 gstv4l2object.c:2941:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder1:sink> Could not probe minimum capture size for pixelformat MJPG     
0:00:05.325886141 39885 0x7f7b3c028b60 WARN                    v4l2 gstv4l2object.c:3056:gst_v4l2_object_get_nearest_size:<nvv4l2decoder1:sink> Unable to try format: Unknown error -1                             
0:00:05.325898270 39885 0x7f7b3c028b60 WARN                    v4l2 gstv4l2object.c:2947:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder1:sink> Could not probe maximum capture size for pixelformat MJPG     
0:00:05.325898077 39885 0x7f7b40015e40 WARN                    v4l2 gstv4l2object.c:3056:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1                             
0:00:05.325917393 39885 0x7f7b3c028b60 WARN                    v4l2 gstv4l2object.c:3056:gst_v4l2_object_get_nearest_size:<nvv4l2decoder1:sink> Unable to try format: Unknown error -1

Update: pipeline is working fine with three sources but not with two sources.

Did you set the batch-size paras to the number of sources in the config file?

Yes, also checking while setting up pipeline to have batch-size same as number of sources. Other than this I’ve tried upgrading pyds to 1.14 as well. The only difference between test3 and my pipeline is that I use object oriented creation, linking and addition of probes. I’ve modified the code at the following github to support multiple streams as mentioned in the first post.

A workaround instead was setting batch-size=1 in pgie config. Setting batch-size=2 in config is causing erroneous behavior from model (yolor) in my case

Your model may not support dynamic batch input. You can debug it by yourself or you can refer the yolo models from the link below:https://forums.developer.nvidia.com/t/deepstream-sdk-faq/80236/24

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.