Extracting Y,UV images from a YUV420 image

[alon.eshel]

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.2.10884
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

extracting Y,UV from a YUV420 image

Using DriveWorks SDK 5.10.
I’m trying to convert a YUV input image to 2 separate input images that are required for the DNN I’m using (code is based on sample_object_detector_tracker).
I’m using the dwImage_copyConvert function for this purpose:

dwImageHandle_t nextFrame = m_camera->readFrame();
dwImageHandle_t yImageHandle = nullptr;
dwImageHandle_t uvImageHandle = nullptr;

// Set the properties of the Y and UV images
dwImageProperties yImageProperties = m_rcbProperties;
yImageProperties.format = DW_IMAGE_FORMAT_R_UINT8;
dwImageProperties uvImageProperties = m_rcbProperties;
uvImageProperties.format = DW_IMAGE_FORMAT_RG_UINT8; // NOT SUPPORTED

// Allocate memory for the Y and UV images
CHECK_DW_ERROR(dwImage_create(&yImageHandle, yImageProperties, m_sdk));
CHECK_DW_ERROR(dwImage_create(&uvImageHandle, uvImageProperties, m_sdk));

// Convert the YUV420 image to Y and UV
CHECK_DW_ERROR(dwImage_copyConvert(yImageHandle, nextFrame, m_sdk));
CHECK_DW_ERROR(dwImage_copyConvert(uvImageHandle, nextFrame, m_sdk));

According to the supported conversion formats (DriveWorks SDK Reference: Image Interface)- only DW_IMAGE_FORMAT_R_UINT8 is supported and would supposedly extract the Y channel given a YUV input. Would this work?

My questions are:

1. how do I extract the UV channels since the DW_IMAGE_FORMAT_RG_UINT8 is not supported?
2. another gap I have regarding the UV channels extraction is that according to the documentation of dwImage_copyConvert it may not support it since it requires input and output to be of the same size while UV height will be half the size of the original YUV image.
3. how else would you recommend to perform this task using the SDK library functions?

[SivaRamaKrishnaNV]

Dear @alon.eshel,
Is the intention is to extract Y,UV buffers into a seperate cuda buffers? If so, could you check if DW_IMAGE_FORMAT_YUV420_UINT8_SEMIPLANAR helps?
In that case, After getting dwImageCUDA, The dwImageCUDA.dptr[0] will have Y data buffer and dwImageCUDA.dptr[1] will have UV data buffer.

[alon.eshel]

Hi @SivaRamaKrishnaNV , thanks for replying.
Thanks for pointing out the information regarding the format.
My intention is to create instances of dwImageCUDA for Y & UV and then call dwDataConditioner_prepareDataRaw to prepare the GPU input blobs for the DNN I’m using.
So only extracting the data is not enough for my need.
I tried just inserting a nullptr like this:
CHECK_DW_ERROR(dwImage_getCUDA(&yImage, yImageHandle));
CHECK_DW_ERROR(dwImage_getCUDA(&uvImage, uvImageHandle));
yImage->dptr[1]=0;
uvImage->dptr[0]=0;
but this results in an exception thrown at dwDataConditioner_prepareDataRaw function.

I noticed there used to be a property that declares the number of planes (prop.planeCount) but didn’t notice anything alike in DW 5.10, I imagine I get the error due to the fact it’s expecting 2 planes but only receives one.
How would you suggest to create a Y dwImageCUDA and UV dwImageCUDA instances?
Thanks,
Alon

[SivaRamaKrishnaNV]

@alon.eshel

dwDataConditioner_prepareDataRaw

Dear @alon.eshel,
Yes. you are right. dwDataConditioner_prepareDataRaw does not accepts YUV420 image type.