Multi-stream rtsp using Gstreamer in python code

[LoveNvidia]

Hello guys,
I’m working on multi-stream decoder using gstreamer + opencv python code with jetson nano.
Q1- I want to know, what’s best gstreamer pipeline with opencv in python code?
I used this :

gstream_elemets = (
    'rtspsrc location=rtsp latency=300 !'
    'queue leaky=2 !'
    'rtph264depay !'
    'h264parse !'
    'omxh264dec enable-max-performance=1 enable-low-outbuffer=1 !'
    'video/x-raw(memory:NVMM),format=(string)NV12 !'
    'nvvidconv ! video/x-raw, width=(int)300, height=(int)300, format=(string)BGRx !'
    'videoconvert ! video/x-raw, format=(string)BGR !'
    'appsink').
cv2.VideoCapture(gstream_elemets, cv2.CAP_GSTREAMER)

My input frame-rate is above 20 FPS and I want the output frame rate to be under 10 FPS.

Q1.1- in your opinion, Is it better to use enable-low-outbuffer=1 ?
Q1.2 - what’s difference using of 'videoconvert ! ’ instead of ‘videoconvert ! video/x-raw, format=(string)BGR !’ ? In two case, It’s on and correctly work. I want to know, which of case have extra processing step and is time-consuming and resource-consuming?
Q1.3- why we use rtph264depay? Is it needed in rtsp?

As you know, the opencv only supported CPU buffer, because of this, we use videoconvert ! video/x-raw, format=(string)BGR !.
Q2- Is it possile to grap the batch of decoded streamed without opencv? I don’t know to CPU buffer to be bottleneck for me.

Q3- I used multi-thread python for handeling multi-stream decoding and the NVDEC of jetson nano is activated. Because my output is lower than input, I defined a dynamic programming that give me a given FPS of output, like this snnipet code: Is this way is efficient?

while cap.isOpened():
    frame = cap.read()[1]
    sleep(1./frame_rate_output)

and I used 6 streams and I runned as multi-thread and is corrctly work. my problem is in the capturing of 6 decode frames at the same time, not with for loop like this:

frames = []
for cam in cams
      frames.append(cam.read())

It’s work and ok, One thing to mention, time-consuming of this for loop is 1 ms in my opinion this value is very samll, perhapse It’s not neccesory to use multi-thread for capturing the batch of frames, right?

[DaneLLL]

Hi,
The information are provided and shared by other users:

Multi-stream rtsp GStreamer + Opencv

Opencv can work with different framerates, but you would have to set these accordingly. Decreasing framerate is not difficult. You can do both ways: Use gstreamer plugin videorate: #include <iostream> #include <opencv2/core.hpp> #include <opencv2/highgui.hpp> #include <opencv2/videoio.hpp> int main(void) { const char *gst_cap = "videotestsrc ! video/x-raw, format=BGR, width=1280, height=720, framerate=30/1" " ! videorate ! video/x-raw, format=BGR, framerate=…

Pipeline multi-stream hardware decoder RTSP with jetson nano

Hi guys, I want to do pipeline multi-stream decoder on jetson nano using H264 hardware decoder. when I run directly this function: def stream_rtsp(uri, width, height, latency): gst_str = ('rtspsrc location={} latency={} ! ’ 'rtph264depay ! h264parse ! omxh264dec ! ’ 'nvvidconv ! ’ 'video/x-raw, width=(int){}, height=(int){}, ’ 'format=(string)BGRx ! videoconvert ! ’ ‘appsink’).format(uri, latency, width, height) then I check to use NVDEC ro not with sudo tegrastats, I see NVDEC is ac…

Gradualy increased memory usage when use gstreamer + opencv

Hi, Please check the source code in OpenCV: // we support 11 types of data: // video/x-raw, format=BGR -> 8bit, 3 channels // video/x-raw, format=GRAY8 -> 8bit, 1 channel // video/x-raw, format=UYVY -> 8bit, 2 channel // video/x-raw, format=YUY2 -> 8bit, 2 channel // video/x-raw, format=YVYU -> 8bit, 2 channel // video/x-raw, format=NV12 -> 8bit, 1 channel (height is 1.5x larger than true height) // video/x-raw, format=NV21 …

The solution is optimal for using gstreamer + OpenCV. If you don’t need OpenCV, you may try DeepStream SDK. There are python samples:

NVIDIA-AI-IOT/deepstream_python_apps