nvstreammux can't batch more than 4 frames from local file

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

Hi,

When I try to run the pipeline with buffer-size > 4, looks like the nvstreammux collects 4 frames and can’t get the next frames:
gst-launch-1.0 filesrc location=sample_720p.h264 ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m buffer-pool-size=10 batch-size=10 width=1280 height=720 ! nvstreamdemux name=d d.src_0 ! nvmultistreamtiler ! nveglglessink

but the pipelines below work fine:
gst-launch-1.0 souphttpsrc location=… ! decodebin ! m.sink_0 nvstreammux name=m buffer-pool-size=10 batch-size=10 width=1280 height=720 ! nvstreamdemux name=d d.src_0 ! nvmultistreamtiler ! nveglglessink

gst-launch-1.0 filesrc location=sample_720p.h264 ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m buffer-pool-size=4 batch-size=4 width=1280 height=720 ! nvstreamdemux name=d d.src_0 ! nvmultistreamtiler ! nveglglessink

Ubuntu 18.04
GeForce RTX 2080

#2

nvv4l2decoder by default allocates 4 buffers. So muxer is waiting indefinitely for 6 more buffers. This will work after setting num-extra-surfaces property of nvv4l2decoder to a high enough number, you can refer https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream_Plugin_Manual%2Fdeepstream_plugin_details.02.12.html

#3

Hello bcao, I face the same issue when link a uridecodebin to nvstreammux. But I checked that uridecodebin has no properties such as num-extra-surfaces. Do you have any ideas how to solve this problem? Thank you.

#4

Hi jeffzhu1990, actually num-extra-surfaces is a property of the nvv4l2decoder plugin which is the child of uridecodebin. You may do smth like this:

    def decoder_added(uridecodebin, sub_bin, element, data):
        try:
            nvv4l2decoder_type = Gst.ElementFactory.find("nvv4l2decoder").get_element_type()
            element_type = element.get_factory().get_element_type()
            if element_type == nvv4l2decoder_type:
                element.set_property("num-extra-surfaces", 24)
        except:
            pass

    uridecodebin.connect("deep-element-added", decoder_added, None)