How to convert dwImageNvMedia or NvMediaImage to OpenCV's cv::Mat?

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hi all!

I have read the “sample_camera_gmsl”, and it shows how to use ImageStreamer to convert a dwImageNvMedia to dwImageGL. But is there a way to convert dwImageNvMedia/NvMediaImage to OpenCV’s cv::Mat?
Thanks!

2

Hello taoshaoyuan,

Please see /usr/local/vibrante/docs/nvvib_docs/index.html in DrivePX2 PDK. Thanks.

OpenCV, which is required for some VisionWorks samples (and other applications), is no longer provided by NVIDIA. The NVIDIA Tegra-specific optimizations have been upstreamed to the open-source OpenCV package; instructions to download and build this package are at:

http://docs.opencv.org/master/d6/d15/tutorial_building_tegra_cuda.html

3

Thanks Steve,

I have already compiled OpenCV for PX2, and found a way to convert dwImageNvMedia to cv::Mat:

// create a cv::Mat from a dwImageNvMedia rgba_image
cv::Mat mat_img(cv::Size(rgba_image->prop.width, rgba_image->prop.height), CV_8UC4, surface_map.surface[0].mapping);
// convert from RGBA to BGRA
cv::cvtColor(mat_img, mat_img, CV_RGBA2BGRA);
// encode image to a jpg
cv::imencode(".jpg", mat_img, *encode_buf, encode_params);

But cv::cvtColor and cv::imencode are really inefficient, is there a way to encode an image to .jpg directly from dwImageNvMedia?
Thanks again!

4

hi,have u solved this issue?

5

Do you know how to get a cv::cuda::GpuMat from a dwImageNvMedia rgba_image?

Thank you!
Regards

6

hello taoshaoyuan and Steve,
According to your ideas,I still have not solve this problem,the following is my code:

void DWPerception::convertToCVMat(dwImageNvMedia *imageNvMedia, cv::Mat &cvimg)
{
    if (imageNvMedia) {
        // Read buffer to cvMat
        NvMediaImageSurfaceMap surfaceMap;
        memset(&surfaceMap, 0, sizeof(surfaceMap));
        if(NvMediaImageLock(imageNvMedia->img, NVMEDIA_IMAGE_ACCESS_READ_WRITE, &surfaceMap)==NVMEDIA_STATUS_OK){           
            if(surfaceMap.surface[0].mapping == nullptr){
                std::cout << "can not get NvMediaImage Surface \n";
                return;
            }
            cvimg = OpenCVConnector::WriteToOpenCV(surfaceMap.surface[0].mapping, surfaceMap.width, surfaceMap.height);
            //std::cout << "original img dims are: " << cvimg.rows << "," << cvimg.cols << std::endl;

            NvMediaImageUnlock(imageNvMedia->img);
        }
        else{
            std::cout << "img read fail \n" ;
        }
   }
}

and then i meet the problem is:
1.NvMediaImageLock function return NVMEDIA_STATUS_OK,but surfaceMap.surface[0].mapping == nullptr. Why?

Thank you!
Regards

7

Dear zhuweisheng,

You may use Imagestreamer to read data from one format to another. Please check the disccusion on the thread dwImageNvMedia dwImageStreamer dwImageCUDA - DriveWorks - NVIDIA Developer Forums to see if it helps.

8

Hi Steve, thank you for your reply,
I really want to know why in my function convertToCVMat, NvMediaImageLock return NVMEDIA_STATUS_OK but surfaceMap.surface[0].mapping in null ?

I have visited many disccusions about convert dwImageNvMedia to cv::Mat, and only this way is simple and clear.
so I want to fix the problem I meet and use this way.
the disccusion link you send I’ve already seen it,I’ve not found a good way to convert dwImageNvMedia to cv::Mat.
Thank you.

void DWPerception::convertToCVMat(dwImageNvMedia *imageNvMedia, cv::Mat &cvimg)
{
    if (imageNvMedia) {
        // Read buffer to cvMat
        NvMediaImageSurfaceMap surfaceMap;
        memset(&surfaceMap, 0, sizeof(surfaceMap));
        if(NvMediaImageLock(imageNvMedia->img, NVMEDIA_IMAGE_ACCESS_READ_WRITE, &surfaceMap)==NVMEDIA_STATUS_OK){           
            if(surfaceMap.surface[0].mapping == nullptr){
                std::cout << "can not get NvMediaImage Surface \n";
                return;
            }
            cvimg = OpenCVConnector::WriteToOpenCV(surfaceMap.surface[0].mapping, surfaceMap.width, surfaceMap.height);
            //std::cout << "original img dims are: " << cvimg.rows << "," << cvimg.cols << std::endl;

            NvMediaImageUnlock(imageNvMedia->img);
        }
        else{
            std::cout << "img read fail \n" ;
        }
   }
}
9

Dear zhuweisheng,

Could you please check the following link to see if it helps?