Doubt about getting the frame of a pipeline

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
, ,
1

Hi,
I’m working with opencv and gstreamer, I’m using the following code to extract the frames of the pipeline.
static GstPadProbeReturn frame_processing(GstPad *pad, GstPadProbeInfo *info, gpointer u_data)
{
buffer = (GstBuffer *)info->data;
map_info = {0};
dmabuf_fd = 0;
gst_buffer_map(buffer, &map_info, GST_MAP_READ);

  ExtractFdFromNvBuffer((void *)map_info.data, &dmabuf_fd);
  {
    egl_image = NvEGLImageFromFd(egl_display, dmabuf_fd);
    cudaFree(0);
    status = cuGraphicsEGLRegisterImage(&p_resource, egl_image, CU_GRAPHICS_MAP_RESOURCE_FLAGS_WRITE_DISCARD);

    status = cuGraphicsResourceGetMappedEglFrame(&egl_frame, p_resource, 0, 0);
    status = cuCtxSynchronize();

    cv::cuda::GpuMat d_mat_BGRA(H, W, CV_8UC4, egl_frame.frame.pPitch[0]);

   ...
   doing things...
   ...

  }
  return GST_PAD_PROBE_OK;
}

Ok, my doubt is with the following line:

    cv::cuda::GpuMat d_mat_BGRA(H, W, CV_8UC4, egl_frame.frame.pPitch[0]);

I want try to understand why here is CV_8UC4. I have two cameras, one of them is a thermal camera and with this configuration my program doesn’t work. But with my other visible camera, my program it’s works.
When I put the thermal camera, when I try to convert the image to gray with “cvtcolor” for example my program crashes.

I try too putting:

cv::cuda::GpuMat d_mat_BGRA(H, W, CV_8UC1, egl_frame.frame.pPitch[0]);
d_mat_BGRA.setTo(cv::Scalar(255));

And the image is correct but when I try to modify it I see the image wrong, as if the image had noise or something similar.

Sin título
Sin título1211×597 27.8 KB

Any help about this topic will be well receive.
Thanks

3

Hi,
Please refer to this sample:

The captured frames from nvarguscamerasrc is in NV12. We need to convert it to RGBA in nvvidconv, and map to OpenCV RGBA buffer. This is the working flow.

4

Ok, I have another question, I wont to download the gpumat to an another Mat to see it with imshow:

cv::cuda::GpuMat d_mat_BGRA(H, W, CV_8UC4, egl_frame.frame.pPitch[0]);
cv::Mat mat;
d_mat_BGRA.download(mat);
cv::imshow("image", mat);
cv::waitKey(30);

I get an error that gives me very little information.

OpenCV(4.4.0) /home/nvidia/opencv-4.4.0/opencv-4.4.0/modules/core/src/cuda/gpu_mat.cu:249: error: (-217:Gpu API call) invalid argument in function 'download'

Any ideas? I start to think that my opencv installation could be the problem.

5

Per more information,
when I change:
cv::cuda::GpuMat d_mat_BGRA(H, W, CV_8UC4, egl_frame.frame.pPitch[0]); → cv::cuda::GpuMat d_mat_BGRA(H, W, CV_8UC3, egl_frame.frame.pPitch[0]);
My program works, but when I see the stream of the result, the image is not modified. But I know that the image would have to be modified because with the other camera is modified.
It’s like when I switch to CV_8UC3 I can’t ever modify the image but I can visualize it, how can I do it?
Thanks

6

Hi,
We don’t have much experience in cuda filters of OpenCV. Would need other users to share experience. Or go to OpenCV forum.

In the demo code, we utilize sobel filter and it looks working fine.