DS-8.0: gst-dsexample-cuda get_converted_mat() does not populate cvgpumat:-(

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Ubuntu 24.04 x86_64 laptop with ADA 2000
• DeepStream 8.0
• TensorRT - does not matter here, but 10.9.0.34-1+cuda12.8
• NVIDIA GPU Driver - 570.195.03-0ubuntu1

• OpenCV 4.12 with CUDA support
• Issue Type - Example has a problem:-(

I know, the OpenCV support is kind of deprecated. But it’s still in the code and it should work when enabled.

The gst-dsexample-cuda creates a GPU buffer/mat for for OpenCV to work on. That mat is then handed to the sample library code. But the input frame is never copied there:-( From what I’ve seen the get_converted_mat() should prepare the input for the library, and it does most of it.

I changed the code around line 603 a bit to actually populate the cvgpumat;-) The original just leaves the converted mat in the local variable and forgets about it.

#ifdef WITH_OPENCV
/* In place GPU operation on the transformed buffer.
  Visual output will be different to original input.
*/
  in_mat =
      cv::cuda::GpuMat (dsexample->processing_height, dsexample->processing_width,
      CV_8UC4, (void *) dsexample->inter_buf->surfaceList[0].dataPtr,
      dsexample->inter_buf->surfaceList[0].pitch);

  // XXX MARCO
  cv::cuda::cvtColor(in_mat, *dsexample->cvgpumat, cv::COLOR_RGBA2BGR);

  /* 
  //Swapped R and B channels
  cv::cuda::swapChannels(in_mat, aDstOrder);

  // XXX MARCO
  cudaMemcpy(dsexample->cvgpumat->data, in_mat.data, dsexample->processing_height *
             dsexample->processing_width * RGBA_BYTES_PER_PIXEL, cudaMemcpyDeviceToDevice);
  */

/* In place GPU operation can also be verified by directly using the input buffer.
  Visual output will be different to original input.
*/
/*
  in_mat =
      cv::cuda::GpuMat (ip_surf.surfaceList[0].height, ip_surf.surfaceList[0].width,
      CV_8UC4, (void *) ip_surf.surfaceList[0].dataPtr,
      ip_surf.surfaceList[0].pitch);

  cv::cuda::swapChannels(in_mat, aDstOrder);
*/

#endif

I’m only interested inthe RGB/BGR channels, so I do the conversion there.

– Marco

3

So this is a bugfix, right?

As a supplement, The following modifications will allow you to modify nvbufsurface in-place using cv::cuda::GpuMat.You can verify this using the following command line.

#ifdef WITH_OPENCV
/* In place GPU operation on the transformed buffer.
  Visual output will be different to original input.
*/
  // in_mat =
  //     cv::cuda::GpuMat (dsexample->processing_height, dsexample->processing_width,
  //     CV_8UC4, (void *) dsexample->inter_buf->surfaceList[0].dataPtr,
  //     dsexample->inter_buf->surfaceList[0].pitch);
  
  // //Swapped R and B channels
  // cv::cuda::swapChannels(in_mat, aDstOrder);

/* In place GPU operation can also be verified by directly using the input buffer.
  Visual output will be different to original input.
*/

  in_mat =
      cv::cuda::GpuMat (ip_surf.surfaceList[0].height, ip_surf.surfaceList[0].width,
      CV_8UC4, (void *) ip_surf.surfaceList[0].dataPtr,
      ip_surf.surfaceList[0].pitch);

  cv::cuda::swapChannels(in_mat, aDstOrder);


#endif

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.jpg ! nvjpegdec ! nvvideoconvert ! m.sink_0  nvstreammux name=m width=1280 height=720 batch_size=1 ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=RGBA" ! dsexample ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=I420" !  nvjpegenc ! filesink location=out.jpg
4

The problem is not the ‘in-place’ modification via the local in_mat variable. The problem is that the rest of the code expects the frame data in dsexample→cvgpumat and that is also handed to the DsExampleProcess() for further processing.

5

I understand what you mean, but I think as a demo plugin, dsexample works fine. We can add this topic to the FAQ.
Thanks your feedback.

6

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.