How to create a custom GStreamer Deepstream plugin which uses OpenCV for out of place transforms?

[agentharke]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson Orin Nano 8GB
• DeepStream Version: 6.3
• JetPack Version (valid for Jetson only): 5.1.2
• Issue Type( questions, new requirements, bugs): Question

I am trying to write a custom Deepstream element for GStreamer which utilizes CUDA resources and NVDS object metadata to generate a depth map from a batched frame input. This requires me to do an out of place transform rather than an in place transform, since CUDA-accelerated OpenCV operations don’t necessarily overwrite all data in their buffers which can lead to erroneous disparity calculations. All of the provided custom deepstream plugin examples perform in place transformations except for the dewarper, which does not seem to utilize opencv in any meaningful way. While I have figured out how to get my data out of the input buffer in my transfrom function, I cannot figure out how to allocate and use the output buffer. When I try to map the output buffer using NvBufSurfaceMap it throws an error that does not provide any helpful information as to why the buffer mapping failed. Are there any examples of doing OpenCV processing with Deepstream in a custom plugin which utilize the output buffer in their transform function? Furthermore, are there any examples of doing this with CUDA accelerated OpenCV functions? I have done a ton of research and cannot seem to find any examples or documentation detailing how this should be accomplished.

[junshengy]

1. Did you check the sample code of gst-dsexample ? This element shows how to work with opencv, you can rewrite it to an out of place transform element.

2.This code snippet show how to map data from NvBufSurfaceMap and use for opencv.

Deepstream sample code snippet

Simple description for your code a. Creating nv12 opencv mat from NvMM memory, b. Work on opencv nv12 mat, do conversion to other opencv RGBA mat, rgba mat. c. Create another scratch RGBA NvBufSurface and do opencv conversion to rotate RGBA in rotate mat. d. Transform rotated RGBA mat to NV12 memory in original input surface e. So now NvMM NV12 memory from original buffer is also rotated. Which version are you based on a. DS 5.0 DP Which platform have you verified for your code a. T4 --…

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks