NvMedia convert dwImageNvMediat to cv::Mat

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hi,

I am trying to convert a dwImageNvMedia to cv::Mat. I first streamed a dwImageGL to dwImageNvMedia in RGBA format. Then I use NvMediaImageGetBits to get a pointer and put it into a cv::Mat.

Seems like I just get partial of the image and it keeps flashing. Could you please help me check how to accomplish this conversion?

Thanks in advaned!

{
    dwImageHandle_t nextFrame = m_camera->readFrame();
    if (nextFrame == nullptr) {
        m_camera->resetCamera();
    } else {
        CHECK_DW_ERROR(dwImage_getCUDA(nextFrameCUDA, nextFrame));
        CHECK_DW_ERROR(dwImage_copyConvert(m_imageRGBA, nextFrame, m_sdk));
        dwImageHandle_t frameGL = m_streamerCUDA2GL->post(m_imageRGBA);
        CHECK_DW_ERROR(dwImage_getGL(nextFrameGL, frameGL));

        dwTime_t timeout = 1000;
		// stream that image to the GL domain
		CHECK_DW_ERROR(dwImageStreamer_producerSend(frameGL, m_streamer2NVM));
		// receive the streamed image as a handle
		dwImageHandle_t frameNVM;
		CHECK_DW_ERROR(dwImageStreamer_consumerReceive(&frameNVM, timeout, m_streamer2NVM));
		
		// get the specific image struct to be able to access texture ID and target
		dwImageNvMedia* nextFrameNvMedia = nullptr;
		CHECK_DW_ERROR(dwImage_getNvMedia(&nextFrameNvMedia, frameNVM));

	    NvMediaImageSurfaceMap surfaceMap;
	    void *pSrcBuff = NULL;
        unsigned int srcPitch = 0;
		unsigned int RGBABytesPerPixel = 2;
    	NvMediaStatus status;
		NVM_SURF_FMT_DEFINE_ATTR(NVMSurfAttr);

        if (NvMediaImageLock(nextFrameNvMedia->img, NVMEDIA_IMAGE_ACCESS_READ_WRITE, &surfaceMap) !=
            NVMEDIA_STATUS_OK) {
            std::cout << "NvMediaImageLock failed\n";
        }

		NvMediaSurfaceFormatGetAttrs(surfaceMap.type, NVMSurfAttr, surfaceMap.surfaces);
		std::cout << "attribute properties : " << NVMSurfAttr[NVM_SURF_ATTR_SURF_TYPE].value << std::endl;
		
		unsigned int NVMsrcImageSize = surfaceMap.height * surfaceMap.width * srcPitch * RGBABytesPerPixel;
		//NVMsrcImageSize += nextFrameNvMedia->img->embeddedDataTopSize;
		//NVMsrcImageSize += nextFrameNvMedia->img->embeddedDataBottomSize;

        if (!(pSrcBuff = malloc(NVMsrcImageSize))) {
            std::cout << "Out of memory\n";
        }

        status = NvMediaImageGetBits(nextFrameNvMedia->img, NULL, &pSrcBuff, &srcPitch);
        if (status != NVMEDIA_STATUS_OK) {
            std::cout << "NvMediaImageGetBits() failed\n";

        }

        m_cvimg = OpenCVConnector::WriteToOpenCV(pSrcBuff, surfaceMap.width, surfaceMap.height);
        NvMediaImageUnlock(nextFrameNvMedia->img);

		// returned the consumed image
		CHECK_DW_ERROR(dwImageStreamer_consumerReturn(&frameNVM, m_streamer2NVM));
		// notify the producer that the work is done
		CHECK_DW_ERROR(dwImageStreamer_producerReturn(nullptr, timeout, m_streamer2NVM));
}
2

BTW, as it gets a portion of the image (not stable), it continues output error log as follows:

NvMediaRGBASurfaceGetBits: TVMROutputSurfaceGetBits() Failed
NvMediaImageGetBits: NvMediaRGBASurfaceGetBits() Failed
NvMediaImageGetBits() failed

3

Already solved!

4

Hi weixi, Could I ask how you can solve this problem ? I’m also trying to convert dwImageNvMedia to opencv Mat. Thanks you in advance