I convert OpenCV's mat to dwImageCuda, but the dwLandmarkDetector_detectLandmarks return error....

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hi everyone,

In my project, I try to integrate the driveworks with ROS. So I use cv_bridge to convert Image message to OpenCV’s Mat format. Then in the driveworks app, I convert Mat to dwImageCPU and then convert it to dwImageCUDA. But when I call dwLandmarkDetector_detectLandmarks or dwPathDetector_processDeviceAsync to do some dections, the program will terminate.

I have check the converted dwImageCPU by convert it back to Mat and compare it with the origin one, that works fine. So I am sure the problem is happening in convertion from dwImageCPU to dwImageCUDA.

Would somebody to check the bugs of my program…

Here is the convertion code snippet:

void getNextFrame(dwImageCUDA **nextFrameCUDA, const cv::Mat &opencv_image) {
  dwImageCPU *nextFrameCPU;
  dwImage_getCPU(&nextFrameCPU, m_imageCPU);
  mempcpy(nextFrameCPU->data[0], opencv_image.data,
          opencv_image.total() * opencv_image.elemSize());
  CHECK_DW_ERROR(dwImageStreamer_producerSend(m_imageCPU, m_streamerCPU2CUDA));
  dwImageHandle_t imageCUDA;
  CHECK_DW_ERROR(
      dwImageStreamer_consumerReceive(&imageCUDA, 1000, m_streamerCPU2CUDA));
  CHECK_DW_ERROR(dwImage_getCUDA(nextFrameCUDA, imageCUDA));
  std::cout << "nextFrameCUDA info:"
            << std::to_string((*nextFrameCUDA)->prop.width) + ":"
            << std::to_string((*nextFrameCUDA)->prop.height);

  CHECK_DW_ERROR(
      dwImageStreamer_consumerReturn(&imageCUDA, m_streamerCPU2CUDA));
  CHECK_DW_ERROR(
      dwImageStreamer_producerReturn(nullptr, 1000, m_streamerCPU2CUDA));
};

Here is the process code snippet:

void OnProcess(){
    dwImageCUDA *rcbImage;
    cv::Mat mat = cv::imread("lane_sample.png");
    getNextFrame(&rcbImage, mat);;
    CHECK_DW_ERROR(dwLandmarkDetector_detectLandmarks(
        &m_landmarkDetectionResult, rcbImage, m_landmarkDetector));
    CHECK_DW_ERROR(dwPathDetector_processDeviceAsync(rcbImage, m_pathDetector));
    CHECK_DW_ERROR(dwPathDetector_interpretHost(m_pathDetector));
    CHECK_DW_ERROR(dwPathDetector_getPathDetections(&m_paths, m_pathDetector));
}
2

Dear dengqi,
Could you double check the image format values for both CPU and CUDA Image. Is this related to your other thread [url]https://devtalk.nvidia.com/default/topic/1047693/driveworks/convert-cv-mat-to-dwimagecpu/[/url]. If so, we will mark it as duplicate issue

3

I found what the question is, the dwLandmarkDetector or dwPathDetector or another CNN detector need 3-plane RGB image. So I need to convert 1-plane mat to 3-plane RGB mat and then convert it to dwImageCPU and dwImageCUDA.

The more detail for convertion can be found in another thread https://devtalk.nvidia.com/default/topic/1047693/driveworks/convert-cv-mat-to-dwimagecpu/.

Thank SivaRamaKrishna to solve my question.