Issues decoding RTSP stream using nvv4l2decoder with Jetpack 4.4

[Honey_Patouceul]

First, apologies to @benky58un for squatting this topic.

@LoveNvidia

IN cv2.videoCapture + gstreamer, first the decoded frames pushed into NVVM buffer and then copied into CPU buffer, right?

Only in case of a MIPI/CSI camera being accessed through ISP with argus. In most other cases such as with v4l2src not going through ISP, frames would be received in CPU allocated memory.

@LoveNvidia

because of opencv only accept CPU buffer,

Yes, an application linked to opencv would only receive frames into CPU allocated cv::Mat using opencv videoio.
However, you can access NVMM buffers with gstreamer plugin nvivafilter. It is intended to perform CUDA operations on NVMM hosted frames, so you can use it with opencv CUDA. You would have to output RGBA frames from this plugin. You may have a look to this example.
Also note that you can directly access from gstreamer buffer.

@LoveNvidia

So, In this mode, for one frame decoding we use double memory from 4GB of jetson nano? (NVVM buffer + CPU buffer), I want to know because the deep stream don’t use opencv for decoding, and only use gstreamer for decoding the frames, when I used opencv + gstreamer and decoding the frames with HW, in this case in the same situation, opencv + gstreamer decoder used twice memory usage than deep stream decoder?

and what’s main difference between opencv+gstreamer and deep-stream decoders? CPU/RAM usage.

There may be (from tail to head) a BGR buffer, a BGRx buffer, a NVMM NV12 buffer, and some H264 encoded/depayed/RTP buffers. For a 4K resolution, worst case being BGRx would be 33MB only, so it should be affordable to have a few copies (I have no Nano, but I think that even with Linux and Ubuntu you would have more than 3.5GB available for your application).