Extracting data from dwImageCPU to be published in ROS

I am trying to send image data across rosnodes and I want to get the dwImageCPU and send a pointer the data in ROS stream

by using CameraGMSL as base sample and building other things on top of it like this:

In OnInitialize()

//------------------------------------------------------------------------------
    // initializes streamer
    // -----------------------------------------
    {
      dwImageProperties cpuImageProperties{};
      cpuImageProperties.width = m_cameraProperties.resolution.x;
      cpuImageProperties.height = m_cameraProperties.resolution.y;
      cpuImageProperties.format = DW_IMAGE_FORMAT_RGBA_UINT8;
      cpuImageProperties.type = DW_IMAGE_CUDA;
      cpuImageProperties.memoryLayout = DW_IMAGE_MEMORY_TYPE_DEFAULT;
      CHECK_DW_ERROR(
          dwImageStreamer_initialize(&m_streamerCUDAtoCPU, &cpuImageProperties, DW_IMAGE_CPU, m_sdk_context));
    }

In Render():

dwImageHandle_t frameCUDA;
      CHECK_DW_ERROR(dwSensorCamera_getImage(&frameCUDA, DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8, frame));

// xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

//--------------- Get a processed CUDA image from camera ---------------

// stream that image to the CPU domain
      CHECK_DW_ERROR(dwImageStreamer_producerSend(frameCUDA, m_streamerCUDAtoCPU));

// receive the streamed image as a handle
      dwImageHandle_t frameROS;
      CHECK_DW_ERROR(dwImageStreamer_consumerReceive(&frameROS, timeout, m_streamerCUDAtoCPU));

      dwImageCPU *img_raw;
      CHECK_DW_ERROR(dwImage_getCPU(&img_raw, frameROS));  // Getting the image from the CPUImageHandle
      auto image_CPU = img_raw->data;

// Header for image
      m_img.header.stamp = ros::Time::now();
      m_img.header.frame_id = "gmsl_cam";
      m_img.header.seq = getFrameIndex();
// using FillImage
      sensor_msgs::fillImage(m_img, sensor_msgs::image_encodings::RGBA8, img_raw->prop.height, img_raw->prop.width,
                             img_raw->prop.width * 4, img_raw->data);

      m_img_pub.publish(m_img);

Reference to fillImage: sensor_msgs/Image Documentation

And when I visualize image feed on image_view, it is something like this : https://navistarinc-my.sharepoint.com/:i:/r/personal/atul_ahirrao_navistar_com/Documents/dwCPU_screenshot_17.12.2019.png?csf=1&e=pdcb50

I am unable to pinpoint out the exact problem with this method but I read somewhere that I should be using cudaMemcpy().

Can you please guide me what would be the arguments for this function? I have no clue about CUDA programming for that matter.

Dear atul.ahirrao,
Could you check printing image_CPU and see if it is matching with cuda buffer data.
cudaMemcpy is cuda to transfer data between CPU and GPU. Please check https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY.html#group__CUDART__MEMORY_1gc263dbe6574220cc776b45438fc351e8 for cudaMemcpy arguments.

Hey SivaRamaKrishna,

my bad. I should have just pointed to “data[0]” and not “data” directly. The problem is solved, thanks!

Hello SivaRamaKrishna,

Although the problem is solved, I have question about the dwImageCPU img_raw->data.

According to documentation of dwImageCPU:

/// Specifies the raw image data.
    uint8_t* data[DW_MAX_IMAGE_PLANES]; 

// where DW_MAX_IMAGE_PLANES = 3

And like I said that I referenced to “data[0]” which worked but now I am curious what is stored at data[1] and data[2]?

Dear atul.ahirrao,
if the image is multiplanar, data[1], and data[2] contains data for 2 and 3 planes respectively Otherwise data[0] contains the image data.