NvMediaImageGetBits on YUV420: Decoding UV plane.


I’m using NvMediaImageGetBits on a YUV420 plane that was created with NVM_SURF_FMT_SET_ATTR_YUV(surfFormatAttrs,YUV,420,SEMI_PLANAR,UINT,16,BL);
with the color type NVM_SURF_ATTR_COLOR_STD_REC709_ER on a Pegasus using, streamed from a GMSL camera.

Locking the image returns the expected number of planes (2) with the expected pitch.
Trying to manually decode the image it doesn’t seem to be UV pairs:
When assuming UV tuples, the color information is doubled in one line. Using width/2 makes the color channel correspond to items in the scene. But then the second plane is missing (verified with a red target; the blue channel is missing). Tried different offsets etc.

The image looks fine on the monitor when using the ipp_raw examples; saving it yields the same wrong image plane result.

Any hints?

Dear philip8g86p,
Did you make changes in ipp_raw samples? Could you please provide code snippet with changes?

I can replicate it with a unmodified ipp_raw executable saving the ISP output. Happy to provide an example stream.

We do use a different camera/driver combination than the Sekonix. Our output images are 3840x1920; but since we are looking at the output at ISP level I guess this should not matter.

I tried opencv, libyuv as well as wrote my own implementation of RGB’fying the output.

To prevent doubling the color layer in each line, the color conversion inner loop has:

int yVal = *(yplane + y * width + x);  // expected
int uVal = *(uplane + (y>>1) * width + (x >> 1));  // For a semi-planar version I would expect a 2 * (x >> 1) or (x & ~1)
int vVal = ??  // tried many combinations, like *(uplane + (y>>1) * width + (x >> 1)) + width/2 but that didn't work.

with uplane = yplane + width * height

Dear philip8g86p,

When the application calls NvMediaImageGetBits on a YUV420 SP Blocklinear NvMediaImage buffer that was created with NVM_SURF_FMT_SET_ATTR_YUV(surfFormatAttrs,YUV,420,SEMI_PLANAR,UINT,16,BL), NvMediaImageGetBits API internally does Semi-planar to Planar conversion and also does Block linear to pitch linear conversion before returning the data to the application.

The application will always get the YUV data in Planar format (3 planes for Y, U and V respectively).

So in this case, the application is expected to allocate memory for holding YUV planar data and is expected to pass 3 buffer pointers (Y, U, and V buffer pointers) to NvMediaImageGetBits.

The data received by the application from NvMediaImageGetBits() API will be in the planar format even though NvMediaImage is holding data in Semi-Planar format.

Note: NvMediaImageLock returns the number of planes as 2 is because underlying NvMediaImage is holding data in Semi-Planar format.

In release, API description is not reflecting the proper usage of the NvMediaImageGetBits API which has been updated later in release.

In, how to covert Block linear to pitch linear? Can you give me some sample code?
picture width x height is 1920x1208
block size is 16x16