Hello,
I have a camera that can output UYVY and NV16 to dma. I use it in a pipeline “camera->color conversion->encode”.
And while the UYVY input is handled properly, setting to NV16 produces unintelligible images.
I convert color with NvBufSurf::NvTransform. So the relevant settings are:
struct v4l2_format fmt = { 0 };
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.width = 1920;
fmt.fmt.pix.height = 1080;
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_NV16; // !!Works with V4L2_PIX_FMT_UYVY
fmt.fmt.pix.field = V4L2_FIELD_NONE;
ioctl(CAMERA_FD, VIDIOC_S_FMT, &fmt);
inParams_.width = 1920;
inParams_.height = 1080;
inParams_.layout = NVBUF_LAYOUT_PITCH;
inParams_.memType = NVBUF_MEM_SURFACE_ARRAY;
inParams_.colorFormat = NVBUF_COLOR_FORMAT_NV16; // !!Works with NVBUF_COLOR_FORMAT_UYVY
inParams_.memtag = NvBufSurfaceTag_CAMERA;
encInFrameParams.width = 1920;
encInFrameParams.height = 1080;
encInFrameParams.layout = NVBUF_LAYOUT_BLOCK_LINEAR;
encInFrameParams.memType = NVBUF_MEM_SURFACE_ARRAY;
encInFrameParams.colorFormat = NVBUF_COLOR_FORMAT_NV12;
encInFrameParams.memtag = NvBufSurfaceTag_VIDEO_CONVERT;
memset(&transform_params, 0, sizeof(transform_params));
transform_params.src_width = 1920;
transform_params.src_height = 1080;
transform_params.dst_width = 1920;
transform_params.dst_height = 1080;
transform_params.flip = NvBufSurfTransform_None;
transform_params.filter = NvBufSurfTransformInter_Nearest;
transform_params.flag = NVBUFSURF_TRANSFORM_FILTER;
...
NvBufSurf::NvTransform(&transform_params, camFrameFd, encBufFd);
So plugging in UYVY format works just fine, but with NV16 it does not. I tried playing around with the width and height to accomodate multiplanar NV16 and you can actually make out something from the image, but it does not make sense because NV12 implicitly knows that it is a 1920x1080 image while having multiple planes.
After allocating buffers with NVBUF_COLOR_FORMAT_NV16, I can get an NvBufsurface with NvBufSurfaceFromFd, and it indeed shows that the allocated buffer has 2 planes and the size seems to be correct.
So my question is: How do I handle NV16? Why does it look like a bunch of artifacts?
p.s. I have verified that NV16 does indeed work on my camera with gstreamer