How to do YUV to RGB conversion?

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hi,

I’m working on a PX2 project, and we want to use the streaming data capturing from camera (ar0231) to do some DNN applications.
Now, we are using ipp_raw as our capturing side, and we write our own egl consumers to receive the streaming data from cameras.
In this architecture, we try to send out images with RGB format since that’s the color format our applications could handle.
However, when I changed the surface format from YUV to RGBA in the ipp pipeline, I got really weird color images.
I would like to confirm if this is a right way to convert the color format in the ipp pipeline?
Or I can’t do this and I need to convert this color format by other ways, such as 2D engine?
Please advise me the right way and refer me any doc I could study.

Thanks,
Krammer

1 Like
2

Dear chenghul,
Please check the thread https://devtalk.nvidia.com/default/topic/1029674/driveworks/drive5s-nvmedia-documentation-vs-reality/

3

Hi Siva,

Thank you for your prompt response.

After reading your link, I still have some questions.

  1. Looks like this means that the Nvidia IPP Framework doesn't support the YUV to RGBA conversion?
  2. If I want to use NvMedia2DBlitEx to convert the format from YUV to RGBA, what kinds of setting should I set? Or is there any sample code? Now, I only create a buffer pool and set the surfType to be RGBA. Are there any other things I need to take care?
  3. Is this NvMedia2DBlitEx a hardware-based conversion?
  4. Is this NvMedia2DBlitEx able to convert YUV to BGR, not BGRA?

Thanks for your help in advance.

Krammer

4

Dear Krammer,
Please create input NvMediaImage as YUV and output NVMediaImage as RGBA and use NvMedia2DBlit API for doing YUV to RGBA conversion. Please check https://docs.nvidia.com/drive/nvvib_docs/NVIDIA%20DRIVE%20Linux%20SDK%20Development%20Guide/baggage/group__blit.html#ga2248b8312152b4f88f7ef2462032833a.

Is this NvMedia2DBlitEx a hardware-based conversion?

Yes.

5

Hello

Could you please share an example on how to convert dwImageNvMedia YUV to dwImageNvMedia RGB?
I would thank you if you could continue the example below:

dwSensorCamera_getImageNvMedia(&frameNVMyuv, DW_CAMERA_PROCESSED_IMAGE, frameHandle);

dwImageNvMedia *frameNVMrgb;
NvMedia2DBlitEx(i2d, frameNVMrgb, NULL, frameNVMyuv, NULL, NULL, NULL);// How to set this up?
6

Yes, this is also my question. Can we do the YUV to RGB conversion by using 2D engine?

7

Dear Krammer,
It Looks like you are using DW APIs. If so, you can directly use dwImageFormatConverter_copyConvertNvMedia() API to convert YUV to RGBA conversion using Tegra 2D engine. Please refer camera_multiple_gmsl sample.

8

@SivaRamaKrishna

Yes, I am using DW API and I get a pointer to an RGBA with NvMediaImageLock and surfaceMap.surface[0].mapping
The point is, I need only a RGB buffer.
How should I filter that mapping to get only RGB channels?

In he other hand: is there any efficient GPU based JPEG compression tool that provides a CPU accessible jpeg encoded image?

Regards

9

Dear Krammer,
There is no inbuilt function to Extract RGB from RGBA. you need to access them byte by byte to extra individual channels.
The RGBA stored format is given at file:///usr/local/driveworks/doc/nvdwx_html/group__image__group.html#gga40e445bbc92b6c60197dd57fc9ca602da2fad36b6d1b6c257083f5ff6afcb7f07

For compressing, you can take a look at NvMedia Encoder sample(https://docs.nvidia.com/drive/nvvib_docs/index.html#page/NVIDIA%2520DRIVE%2520Linux%2520SDK%2520Development%2520Guide%2FMultimedia%2Fnvmedia_nvmimg_jpgenc.html%23)

10

An Nvidia example on this is highly appreciated! Thanks!

11

Is it workable for setting pixlformat with “DW_IMAGE_RGB” then resize by cudaMemcpy2D()?
…
// format converter
dwImageProperties cameraImageProperties;
dwSensorCamera_getImageProperties(&cameraImageProperties, DW_CAMERA_PROCESSED_IMAGE, cameraSensor);
dwImageProperties displayImageProperties = cameraImageProperties;
displayImageProperties.pxlFormat = DW_IMAGE_RGB;
displayImageProperties.planeCount = 1;
…