How to use deepstream_imagedata-multistream in custom app

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi all,
I want to use deepstream_imagedata-multistream for decoding of multi-stream, and then use the decoded frames in my custom python app, I want to integrate decoder(deepstream_imagedata-multistream) with my processing part and both work next to each other .

3

Do you mean to process the decoded video data(in YUV or RGB format)? The sample app deepstream_imagedata-multistream is not just a decoder. There is no interface for app to access the data buffer directly now.
In c level, you can develop your own gstreamer plugin to process frame data. https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream_Development_Guide/deepstream_custom_plugin.html#

4

Yes I want to process the decoded streams in python.
I cant instread of write to disk the decoded frames, push them into queue?

5

Python interface is application level interface, it can not access the frame data directly.

6

So there is no way to use decoded frames in custom python app?

I used gstreamer + opencv in python, but this use both GPU buufer + CPU buffer,

        gstream_elemets = (
            'rtspsrc location=rtsp latency=300 !'
            'queue !'
            'rtph264depay ! h264parse !'
            'omxh264dec  disable-dvfs=1 enable-low-outbuffer=1 !'
            'queue !'
            'video/x-raw(memory:NVMM), framerate=25/1, format=(string)NV12 !'
            'queue !'
            'nvvidconv ! video/x-raw, width={}, height={}, framerate=25/1, format=(string)BGRx !'
            'videoconvert ! video/x-raw, framerate=25/1, format=(string)BGR !'
            'appsink max-buffer=100 sync=0 ')

cv2.VideoCapture(gstream_elemets, cv2.CAP_GSTREAMER)

I want to skip some frames, becuasethe input frame rate = 25 and output frame rate = 5, then I don’t want to decode all evey 25 input frames, I want to decode every 5th frames. This option is exist in nvv4l2decoder , but there is not such a way in omxh264dec to skip some frames. and I tested the nvv4l2decder, but I don’t know why this decoder don’t work in rtsp source but work for file source.
I also used videorate elemnt for framerate=5/1, but this cause cpu usage dramatically increased. I have to decoder every 25 frames then take only 5 frame with counter state in python program. because of I don’t want to decode every 25 frames.

please see this link.

7

Do you mean you want to use “drop-frame-interval” property of nvv4l2decoder to change output framerate?

8

I don’t want to use “drop-frame-interval” property of nvv4l2decoder for setting framerate of output, But because the output frame rate is lower than input frame rate, why I do doing decoder for all frames, I want to drops some of frames without decoding.

9

Dropping frame function needs to consider many conditions in pipeline. It is better to implement in c level than in python.