Using cv::cuda in DeepStream gst-dsexample

[xnodrk]

Hello, I’m trying to develop a gstreamer plugin from the deepstream gst-plugin example on Jetson AGX Orin, where I rotate the input buffer by a constant.

I want to use the cv::cuda::WarpAffine function , with the available “NvBufSurface* surface” inside transform_ip function, map cv::cuda::GpuMat to it, and also use the “NvBufSurface* inter_buf” from the example

GStreamer crash with cv::Exception "…cuda/gpu_mat.cu:386: error: (-217:Gpu API call) invalid argument in function ‘setTo’ "

I don’t know how to progress from here, if it is related to image format like CV_8UC3, or some CUDA initialization. I don’t know how to debug the openCV library.

I posted my whole code on GitHub - ZeroVital/DsExample at DsExample_0.1 if someone have clue on the problem , DsExample has been simplified to maximum

Here, the most important lines :

    dsexample->gpumat = new cv::cuda::GpuMat(dsexample->processing_height, dsexample->processing_width,
                        CV_8UC3, surface->surfaceList[0].dataPtr, dsexample->processing_width * RGB_BYTES_PER_PIXEL);

    //Memset the memory
    NvBufSurfaceMemSet (dsexample->inter_buf, 0, 0, 0);

    dsexample->rot_gpumat = new cv::cuda::GpuMat(dsexample->processing_height, dsexample->processing_width,
                        CV_8UC3, dsexample->inter_buf->surfaceList[0].dataPtr, dsexample->processing_width * RGB_BYTES_PER_PIXEL);

  dsexample->pt_rot = new cv::Point2f(dsexample->processing_height/2., dsexample->processing_width/2.);

  dsexample->rot_mat = new cv::Mat(2, 3, CV_64F);
  *dsexample->rot_mat = cv::getRotationMatrix2D(*dsexample->pt_rot, 10, 1.0); // 10° rotation for example

  cv::cuda::warpAffine(*dsexample->gpumat, *dsexample->rot_gpumat, *dsexample->rot_mat, dsexample->gpumat->size());

Also, after success on WarpAffine, how should i replace the input_buffer of gstreamer for the next element ? is transform_ip the right tool for the job ?

Thank you for your support.
Regards,
Anton

[fanzh]

please refer to Deepstream sample code snippet, it is a sample about How to access and modify the NvBufSurface

[xnodrk]

this example manipulates cv::Mat
i want to use cv::cuda::GpuMat, it is same logic ?

[fanzh]

here is a sample to use GpuMat Implementing a Custom GStreamer Plugin with OpenCV Integration Example — DeepStream 6.2 Release documentation
GpuMat and Mat’s interfaces are similar , the big difference is that GpuMat has upload/download functions, which are used to transfer date between CPU and GPU.

[xnodrk]

Its what I’m using already, well i will try to make separate program doing the rotation with gpumat first

[fanzh]

Sorry for the late reply, Is this still an DeepStream issue to support? Thanks

[xnodrk]

Exception seems to be related to the build of Dsexample

In some threads(OpenCV build script) , you mention launching the script to build opencv for cuda, what i did

It allows me to make run successfully algorithms with cv::cuda::GpuMat

I find it is a shame that Deepstream SDK Gstreamer Plugin Example just works with usual openCV / Cpu Mat, since Jetson revolves around GPU and Nvidia CUDA

You mention using cv::cuda::GpuMat in your Developer Guide but all the rest is left to us to look for and information on internet is quasi inexistant

Maybe because you want us to use VPI ? and then there is no example of VPI integration into Deepstream

I don’t have any “issue” with Deepstream SDK since what i want to do is not officially supported, I’m just looking for help, this is what a forum is about

Nobody can point me to a modified Makefile of Dsexample that would successfully build with OpenCV on CUDA ?

Sorry for the rant but its kinda frustrating

[fanzh]

@xnodrk

Hello, I’m trying to develop a gstreamer plugin from the deepstream gst-plugin example on Jetson AGX Orin, where I rotate the input buffer by a constant.

do you mean arbitrary degree? currently, NvBufSurfTransform, which has GPU acceleration, only supports some fixed degree, degree

[xnodrk]

Hello thanks for your answer,

I’m not using NvBufSurfaceTransform, I’m mapping NvBufSurface onto GpuMat

I’m just doing :

cv::cuda::GpuMat gmat2;

dsexample->gpumat =  new cv::cuda::GpuMat(720,1080, CV_8UC4, (void*)surface->surfaceList[0].dataPtr, surface->surfaceList[0].pitch);

cv::cuda::warpAffine(*dsexample->gpumat, gmat2, *dsexample->rot_mat, dsexample->gpumat->size());

inside gst_dsexample_transform_ip function

then i have the following cv::exception :

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.6.0) /tmp/build_opencv/opencv/modules/core/include/opencv2/core/cuda/common.hpp:102: error: (-217:Gpu API call) named symbol not found in function 'bindTexture'

my environment is :
• Hardware Platform (Jetson / GPU)
Jetson AGX Orin
• DeepStream Version
6.1.1
• JetPack Version (valid for Jetson only)
5.0.2
• Cuda Version
11.4
• Opencv Version
4.6.0 ( compiled WITH_CUDA=ON)
• Issue Type( questions, new requirements, bugs)

[fanzh]

@xnodrk

terminate called after throwing an instance of 'cv::Exception'

please refer to opencv code, there is no more sample about cv::cuda::GpuMat in deepstream SDK.

[xnodrk]

Its working by passing through CUDA EGL Interop

as seen in How to create opencv gpumat from nvstream? - #18 by DaneLLL