cuGraphicsResourceGetMappedEglFrame() is not working as expected (when with Argus)

TL;DR

  • Argus provides cuEGLStream
  • To access cuEGLFrame, cuGraphicsResourceGetMappedEglFrame() is necessary
  • However, it maps to the host and device memory (not always to the device memory)
  • (RAW16 maps to the device, YCrCb maps to the CPU)

Hello!

I use Argus to retrieve camera frame via EGLStream, and it is then processed with OpenCV.

To take full advantage of GPU, I’m currently working on cuEGLStream so that I can directly connect to cv::cuda::GpuMat.

Meanwhile, I’ve found something unexpected.


argus/samples/syncSensor demonstrates how to use cuEGLStream with Argus

For EGLStream, it uses PIXEL_FMT_YCbCr_420_888.

        : m_connection(connection), m_stream(NULL), m_resource(0)
    {
        CUresult r = cuEGLStreamConsumerAcquireFrame(&m_connection, &m_resource, &m_stream, -1);
        if (r == CUDA_SUCCESS)
        {
            r = cuGraphicsResourceGetMappedEglFrame(&m_frame, m_resource, 0, 0);

            if (r == CUDA_SUCCESS)
            {
                uchar* ptr = (uchar*) m_frame.frame.pPitch[0];

                // In theory, it should throw Segfault since ptr is pointing the device memory 
                for (int i = 0; i < 10; i++)
                    printf("%u ",ptr[i]);
                
                printf("\n");
                
            } 
        } 
    }

The above code actually prints values and terminates without any error.

However, if I change the pixel forma to PIXEL_FMT_RAW16, the code stuck at cuEGLStreamConsumerAcquireFrame()

Also, I tried NvCvCam which uses PIXEL_FMT_RAW16 for EGLStream.

In frame.cpp you can find a snippet where acquiring and mapping are done to convert cuEGLFrame into cv::cuda::GpuMat

Frame::Frame(CUgraphicsResource resource,
             CUeglStreamConnection conn,
             cudaStream_t stream)
    : _resource(resource), _conn(conn), _stream(stream), _raw_frame(), _mat() {
  CUresult cu_err;

  DEBUG << "frame:Mapping from resource.";
  cu_err = cuGraphicsResourceGetMappedEglFrame(&_raw_frame, resource, 0, 0);
  if (cu_err) {
    ERROR << "frame:Could not map CUgraphicsResource to CUeglFrame becuase: "
          << error_string(cu_err) << ".";
    return;
  }

  if (_raw_frame.width > std::numeric_limits<int>::max()) {
    ERROR << "_raw_frame.width out of range: " << _raw_frame.width;
    std::terminate();
  }

  if (_raw_frame.height > std::numeric_limits<int>::max()) {
    ERROR << "_raw_frame.height out of range: " << _raw_frame.height;
    std::terminate();
  }

  if (!utils::printCUDAEGLFrame(_raw_frame)) {
    return;
  }

  uchar* ptr = (uchar*) _raw_frame.frame.pPitch[0];

  // When PIXEL_FMT_YCbCr_420_888, it prints 
  // When RAW16, throws Segfault (illegal memory access)
  for (int i = 0; i < 10; i++)
    printf("%u ",ptr[i]);
  printf("\n");
 
  // map the data to a GpuMat
  _mat = cv::cuda::GpuMat(_raw_frame.height, _raw_frame.width, CV_16UC1,
                          _raw_frame.frame.pPitch[0], _raw_frame.pitch);
}

Conclusion

My conclusion is, when using YCrCb, it maps to CPU.
Unfortunately, I want it to be GPU, a device memory.

Any help will be appreciated!

syncSensor.zip (11.0 KB)