pass pointer from CU file to opencv cv::cuda::GpuMat.data

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

I have a CUDA kernel file
I want to pass unsigned char* directly from CU files to opencv cv::cuda::gpumat

In one of the sample(v4l2cuda) , in multimedia api,
in capture.cpp,
 there is a function (to call cuda kernel) for passing the V4L2 userptr

static unsigned char * cuda_out_buffer = NULL;

static void process_image (void *p){
gpuConvertYUYVtoRGB ((unsigned char *) p, cuda_out_buffer, width, height);
}

process_image ((void *) buf.m.userptr);

and I suppose it works allright, because I got an Image by using this cuda_out_buffer pointer
FILE *fp = fopen (“cu.ppm”, “wb”);
fprintf (fp, “P6\n%u %u\n255\n”, width, height);
fwrite (cuda_out_buffer, 1, width * height * 3, fp); //got an image
fclose (fp);

myapp.cpp:
 //function to get cuda_out_buffer from capure.cpp
unsigned char* getcudaptr;
fromCU(&getcudaptr, bytes_used, width, height);//function to get cuda_out_buffer from capure.cpp

capture.cpp:
 //now I passed cuda_out_buffer to my opencv file myapp.cpp, by using getcudaptr
void fromCU(unsigned char **setcudaptr, int bytes_used,int width, int height){
*setcudaptr = cuda_out_buffer;
}

myapp.cpp:
 FILE *fp = fopen (“fromcudaptr.ppm”, “wb”); //this works, I got the same image
fprintf (fp, “P6\n%u %u\n255\n”, width, height);
fwrite (getcudaptr, 1, width * height * 3, fp);
fclose (fp);

now I intent to pass cuda_out_buffer, directly to cv::cuda::GpuMat,
so, I did

cv::cuda::GpuMat <b>gpu_src</b>;
    gpu_src.create(height, width, CV_8UC3);
    gpu_src.data = getcudaptr;                  //passing the pointer to opencv gpumat
        FILE *fp = fopen ("gpu_src_data.ppm", "wb");
        fprintf (fp, "P6\n%u %u\n255\n", width, height);
        fwrite (gpu_src.data, 1, width * height * 3, fp);   //gpu_src.data gives the same image output as, getcudaptr & cuda_out_buffer
        fclose (fp);

then,
imshow(“OpenCV V4L2”, gpu_src);
 //gives error:
///home/is/src/opencv-3.4.6/modules/core/src/opengl.cpp:230: error: (-217:Gpu API call) invalid argument in function ‘copyFrom’

why I am to able to save the image by fopen/fwrite, but not able to display it using imshow( I have tested, imshow+opengl, works for mat->upload->gpumat->imshow )
or is my poor handling during creating the gpusrc header?

2

Hi,

You can find more information in OpenCV’s document:
https://docs.opencv.org/4.0.0/d0/d60/classcv_1_1cuda_1_1GpuMat.html

1. You will need to pass the pointer when the GpuMat is created.

GpuMat (int rows, int cols, int type, void *data, size_t step=Mat::AUTO_STEP)

2. GPU memory is only accessible by the CUDA kernel.
Please copy the memory back to CPU before serializing it with CPU.

fwrite (gpu_src.data, 1, width * height * 3, fp);   //gpu_src.data gives the same image output as, getcudaptr & cuda_out_buffer

Thanks.

3

Hi,
my apologies for the very late reply, I tried what you recommended , something like this:

cv::cuda::GpuMat gpu_frame2(height, width, CV_8UC3, (void*)getcudaptr, cv::Mat::AUTO_STEP);

this compiled,but when I try to display it with imshow

cv::imshow(winName, gpu_frame2);

it throws this error->
terminate called after throwing an instance of ‘cv::Exception’
what(): OpenCV(3.4.6) /home/is/src/opencv-3.4.6/modules/core/src/opengl.cpp:230: error: (-217:Gpu API call) invalid argument in function ‘copyFrom’

curiously, I tried to download to mat and then imshow:

gpu_frame2.download(preview);
cv::imshow("temp", preview);

then also it is throwing run time error:
terminate called after throwing an instance of ‘cv::Exception’
what(): OpenCV(3.4.6) /home/is/src/opencv-3.4.6/modules/core/src/cuda/gpu_mat.cu:249: error: (-217:Gpu API call) invalid argument in function ‘download’

4

Hi,

OpenCV display function doesn’t support GPU memory.
You will need to copy the memory back to CPU first.

For your second message, it looks like there are some issues when copying the memory back to CPU.
May I know how do you compile the OpenCV package from source.
For Jetson Nano, you will need to build it with sm=5.3 GPU capacity support.

Thanks.

5

@paul23, is your ‘preview’ a cv::Mat (not cv::cuda::GpuMat)? The error says invalid argument which seems to mean it is not a CPU side cv::Mat.

6

You may try to use a window created with WINDOW_OPENGL flag, for example:

cv::namedWindow("MyOpenGlWindow", cv::WINDOW_AUTOSIZE | cv::WINDOW_OPENGL);

(Of course you would need an opencv build with opengl support)

Then in your loop, you would specify this window to imshow:

cv::imshow("MyOpenGlWindow",gpu_frame2);
7

sorry for the delay,
Hi AastaLLL,
on the contrary I think opencv supports it, as I have tested with a small snippet,

cv::Mat src_bgr = cv::imread(n1080.jpg);
cv::cuda::GpuMat gp;
gp.upload(src_bgr,s[0]);
cv::namedWindow("gpu_show", CV_WINDOW_OPENGL);
cv::imshow("gpu_show", gp);
cv::waitKey(0);

and it works!
and yes while compiling opencv I gave the flags -D CUDA_ARCH_BIN=“5.3” -D WITH_OPENGL=ON …

8

HI Maheriya,
As I checked its cpu side cv::Mat only

9

HI Honey_Patouceul,
Indeed,in this

cv::imshow(winName, gpu_frame2);

I declared with opengl flags only before as,

cv::namedWindow("winName", CV_WINDOW_OPENGL);