DeepStream Python bindings with new Streammux outputs nothing

Zygislu · September 30, 2023, 8:59am

• Hardware Platform (Jetson / GPU)
GPU
• DeepStream Version
6.2
• NVIDIA GPU Driver Version (valid for GPU only)
535
• Issue Type( questions, new requirements, bugs)
Bugs
• How to reproduce the issue?

I have setup DeepStream 6.2 and installed python bindings for it. I’m using USE_NEW_NVSTREAMMUX=yes and have modified deepstream-imagedata-multistream python script to use New Streammux as such:

    if not sink:
        sys.stderr.write(" Unable to create egl sink \n")

    if os.environ.get('USE_NEW_NVSTREAMMUX') != 'yes':  # Only set these properties if not using new gst-nvstreammux
        if is_live:
            print("Atleast one of the sources is live")
            streammux.set_property('live-source', 1)

        streammux.set_property('width', STREAMMUX_WIDTH)
        streammux.set_property('height', STREAMMUX_HEIGHT)
        streammux.set_property('batched-push-timeout', 40000)
    streammux.set_property('batch-size', number_sources)
    pgie.set_property('config-file-path', "inference_config.txt")

and modified this code part aswell:

    if not is_aarch64():
        # Use CUDA unified memory in the pipeline so frames
        # can be easily accessed on CPU in Python.
        mem_type = int(pyds.NVBUF_MEM_CUDA_UNIFIED)
        if os.environ.get('USE_NEW_NVSTREAMMUX') != 'yes':
            streammux.set_property("nvbuf-memory-type", mem_type)
        nvvidconv.set_property("nvbuf-memory-type", mem_type)
        nvvidconv1.set_property("nvbuf-memory-type", mem_type)
        tiler.set_property("nvbuf-memory-type", mem_type)

So now when i don’t use New Streammux everything works as intended, however if I set USE_NEW_NVSTREAMMUX to yes then the pipeline doesn’t work and sources don’t get processed. I suspect it’s because of not using nvbuf-memory-type with New Streammux, or maybe it’s something else?

yingliu · October 1, 2023, 9:17am

Should be other reason as nvbuf-memory-type doesn’t exist in the new streammux. (refer the comparation here: Gst-nvstreammux New — DeepStream 6.3 Release documentation (nvidia.com))
You can set GST_DEBUG=3 to get more information before the issue happens.

Zygislu · October 1, 2023, 9:36am

I have set GST_DEBUG=3 and this is what i get when i run the pipeline:

deepstream-python | 0:00:06.235296306     1 0x7fd5a00674c0 WARN                 qtdemux qtdemux.c:3250:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
deepstream-python | 0:00:06.250728709     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250763256     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MJPG
deepstream-python | 0:00:06.250771312     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250778342     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MJPG
deepstream-python | 0:00:06.250815328     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250822672     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat AV10
deepstream-python | 0:00:06.250828807     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250836709     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat AV10
deepstream-python | 0:00:06.250852199     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250885063     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat DVX5
deepstream-python | 0:00:06.250891687     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250898792     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat DVX5
deepstream-python | 0:00:06.250918528     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250935246     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat DVX4
deepstream-python | 0:00:06.250961846     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.250979191     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat DVX4
deepstream-python | 0:00:06.251011225     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251043500     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MPG4
deepstream-python | 0:00:06.251050352     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251057649     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MPG4
deepstream-python | 0:00:06.251070342     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251078831     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat MPG2
deepstream-python | 0:00:06.251085505     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251092834     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat MPG2
deepstream-python | 0:00:06.251106423     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251114912     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat H265
deepstream-python | 0:00:06.251121884     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251129550     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat H265
deepstream-python | 0:00:06.251141244     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251148874     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat VP90
deepstream-python | 0:00:06.251155639     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251165011     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat VP90
deepstream-python | 0:00:06.251194881     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251213401     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat VP80
deepstream-python | 0:00:06.251221056     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251228902     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat VP80
deepstream-python | 0:00:06.251242939     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251261926     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe minimum capture size for pixelformat H264
deepstream-python | 0:00:06.251289747     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:sink> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.251324932     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:sink> Could not probe maximum capture size for pixelformat H264
deepstream-python | 0:00:06.252196724     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.252228989     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat NM12
deepstream-python | 0:00:06.252262626     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
deepstream-python | 0:00:06.252277025     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat NM12
deepstream-python | 0:00:06.252316156     1 0x7fd5a0067de0 WARN                    v4l2 gstv4l2object.c:2395:gst_v4l2_object_add_interlace_mode:0x7fd598019ff0 Failed to determine interlace mode
deepstream-python | 0:00:06.256055046     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.368502102     1 0x7fd5a0067de0 WARN             nvstreammux gstnvstreammux.cpp:1051:configure_module:<Stream-muxer> No config-file provided; falling back to default streammux config 1
deepstream-python | 
deepstream-python | 0:00:06.372027769     1 0x7fd5a0067de0 WARN            v4l2videodec gstv4l2videodec.c:1836:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
deepstream-python | 0:00:06.372114377     1 0x7fd5a0067de0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1082:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
deepstream-python | 0:00:06.372505305     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.373082126     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.373202620     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.373293192     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.488436676     1 0x7fd598027c00 WARN          v4l2bufferpool gstv4l2bufferpool.c:1533:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
deepstream-python | 0:00:06.488553900     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.489604991     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.490558220     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.490755246     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.491985916     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.492204384     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.492520567     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message
deepstream-python | 0:00:06.492856508     1 0x7fd5a0067de0 FIXME              h265parse gsth265parse.c:1850:gst_h265_parse_parse_frame:<h265parse0> Implement timestamp/duration interpolation based on SEI message

fanzh · October 8, 2023, 6:33am

sorry for the late reply.

what do you mean about “sources don’t get processed”?
what is your start commandline?

fanzh · October 8, 2023, 7:20am

using the same code, I can’t reproduce the issue on T4+DS6.2. here are the code and log.
deepstream_imagedata-multistream.py (17.5 KB)
log.txt (6.5 KB)

Zygislu · October 8, 2023, 11:00am

Thanks for the reply, your code works, thanks for it as well.
In the main question I forgot to mention that I’m using additionally these lines to crop ROI from input source:

        if uri_name.find("rtsp://") == 0:
            is_live = True
        source_bin = create_source_bin(i, uri_name)
        if not source_bin:
            sys.stderr.write("Unable to create source bin \n")

        nvvidconv_crop = Gst.ElementFactory.make("nvvideoconvert", f'nvvideoconvert{i}')
        nvvidconv_crop.set_property('src-crop', '14:26:819:288')

        pipeline.add(source_bin)
        pipeline.add(nvvidconv_crop)

        padname = "sink_%u" % i
        sinkpad = streammux.get_request_pad(padname)
        if not sinkpad:
            sys.stderr.write("Unable to create sink pad bin \n")
        srcpad = source_bin.get_static_pad("src")
        if not srcpad:
            sys.stderr.write("Unable to create src pad bin \n")

        nvvidconv_srcpad = nvvidconv_crop.get_static_pad("src")
        nvvidconv_sinkpad = nvvidconv_crop.get_static_pad("sink")

        srcpad.link(nvvidconv_sinkpad)
        nvvidconv_srcpad.link(sinkpad)

    print("Creating Pgie \n ")
    pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")

When I comment them out, everything works, however when I enable them, the pipeline doesn’t work.
Maybe you have an example or suggestion how I could fix this, so that ROI cropping would work with new streammux, because with old it works.

fanzh · October 10, 2023, 9:13am

you can work around by adding a NV12 capsfilter after nvvideoconvert. please refer to this code
1.diff (7.1 KB)

Zygislu · October 14, 2023, 8:21am

Thank you for all the help and the code above, the pipeline runs now. However when i save frames in tiler_sink_pad_buffer_probe function, more specifically when I save n_frame it saves it at the same resolution as my video, e.g. 1920x1080, but not cropped, e.g. 416x256.
What other changes should I make so that I could save n_frame as cropped?

fanzh · October 15, 2023, 2:47pm

please refer to this sample test, the resolution of 7.jpeg is still 720p.
gst-launch-1.0 filesrc location= /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.jpg ! jpegdec ! ‘video/x-raw,format=I420’ ! nvvideoconvert src-crop=“0:0:300:300” ! jpegenc ! filesink location=7.jpeg

Zygislu · October 17, 2023, 4:37am

Yeah you are right. Is there anything I can add to maybe the end of the pipeline so that the output image would be the same resolution as the crop?
Or should I use something else instead of nvvideoconvert src-crop?

fanzh · October 17, 2023, 6:32am

please refer to this command, the output resolution is the same with crop’s.
gst-launch-1.0 filesrc location= /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.jpg ! jpegdec ! ‘video/x-raw,format=I420’ ! nvvideoconvert ! ‘video/x-raw,format=I420,width=300,height=300’ ! jpegenc ! filesink location=7.jpeg

system · October 31, 2023, 6:33am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Put delay after streammux DeepStream SDK deepstream	14	49	March 21, 2025
Deepstream pauses on processing onto video frames inside triton container after few frames DeepStream SDK gstreamer , computer-vision-cv , segmentation	9	1009	November 22, 2022
TensorRT Engine Error while building DeepStream SDK deepstream	10	49	October 23, 2024
Is it correct that DeepStream Python bindings cannot be used to analyze RTP video streams? DeepStream SDK cudnn , jetson , deepstream	21	99	March 4, 2025
How to use DeepStream nvv4l2h264enc DeepStream SDK	6	2746	October 12, 2021
How to use splitmuxsink in deepstream pipeline DeepStream SDK	2	995	February 3, 2023
NvStreammux plugin seems to cause problem in a decoded h264 video stream DeepStream SDK	11	705	December 25, 2023
Deepstream RTSP Usability in Python DeepStream SDK	17	834	November 7, 2023
Deepstream_parallel_inference_app add rtsp reconnects DeepStream SDK	61	204	September 3, 2024
The hardware encoding using a jetson NX (nvv4l2h265enc ) GPU-Accelerated Libraries gstreamer	6	1381	August 14, 2020

DeepStream Python bindings with new Streammux outputs nothing

Related topics