Hi,
In the Tegra K1 TRM (TegraK1_TRM_DP06905001_public_v03p.pdf) section 28.2.4, it is mentioned that the VI unit has support for converting YUV422 to YUV planar/semiplanar formats.
But we are unable to achieve this. We have added support in VI_CSI_0_CSI_IMAGE_DT_0 as well as in VI_CSI_0_IMAGE_DEF_0 registers for processing YUV422 (YUV422_8) to YUV semi planar (T_Y8__U8V8_N422). But the output we get is not satisfactory. Are we missing something? Can somebody point us in the right direction?
W.R.T Tegra4 TRM, there are additional registers when processing YUV planar buffers. ie.
VI_VB0_START_ADDRESS_U_0, VI_VB0_BASE_ADDRESS_U_0,
VI_VB0_START_ADDRESS_V_0, VI_VB0_BASE_ADDRESS_V_0,
VI_VB0_START_ADDRESS_FIRST_0, VI_VB0_BASE_ADDRESS_FIRST_0.
These registers are used in the Tegra 4 vi driver also.
But no such registers are present in the TK1 TRM. Has the procedure for processing YUV420 in the VI2 unit in TK1 significantly different?
Hi dilipkumar
Could you have detail information and how you are verify the result.
Hi Shane,
When we configure
format = TEGRA_IMAGE_FORMAT_T_U8_Y8__V8_Y8;
data_type = TEGRA_IMAGE_DT_YUV422_8;
in vi2.c we get the proper UYVY image from the camera to the memory. We tried to configure for YUV420 as follows
format = TEGRA_IMAGE_FORMAT_T_Y8__U8V8_N422;
data_type = TEGRA_IMAGE_DT_YUV422_8;
The output image that we receive contains only Y data. We have tried changing the bits_per_sample field in tegra_camera_yuv_formats structure in common.c as follows
{
.fourcc = V4L2_PIX_FMT_YUV420,
.name = "YUV420 (YU12) planar",
.bits_per_sample = 12,
.packing = SOC_MBUS_PACKING_NONE,
.order = SOC_MBUS_ORDER_LE,
},
We tried with different bits per sample (8,12, 16). The output image always contains only Y data but the number of zeros padded to a line varies.
I have attached a few raw images (test pattern in 640x480) to show the output we get.
when bits_per_sample = 8 : Dropbox - yuv420-semi-planar-8bpp.raw - Simplify your life
bits_per_sample = 12 : Dropbox - yuv420_semi-planar-12bpp.raw - Simplify your life
bits_per_sample = 16 : Dropbox - yuv420-semi-planar-16bpp.raw - Simplify your life
Normal UYVY buffer in 16bpp (for reference) : Dropbox - yuv422-uyvy-original.raw - Simplify your life
We noticed this comment in vi2_capture_buffer_setup in vi2.c
case V4L2_PIX_FMT_YUV420:
case V4L2_PIX_FMT_YVU420:
/* FIXME: Setup YUV buffer */
What kind of processing has to be done here?
Can you answer this question also please?
These are the programming that we need to do for planar/semi-planar output from VI.
The CSI data-type should be programmed appropriately in VI_CSI_0_CSI_IMAGE_DT_0
The memory format should go in VI_CSI_0_IMAGE_DEF_0_FORMAT
The base addresses for each surface needs to be programmed here.
VI_CSI_0_SURFACE0_OFFSET
VI_CSI_0_SURFACE1_OFFSET
VI_CSI_0_SURFACE2_OFFSET
Thanks for the info Shane. We’ll try the configuration you have mentioned and update the results we get.
Hi Shane,
We are able to capture NV16 (16bits per pixel semi planar) format successfully now. Your inputs were valuable in understanding more about how the VI unit works. However I have a few more doubts. Our camera can support both YUV420 8 bit (legacy) as well as YUV422 8 bit over MIPI. Which one would you recommend to be used to obtain the following formats to memory.
The following configuration does not seem to work:
format = TEGRA_IMAGE_FORMAT_T_Y8__U8V8_N420;
data_type = TEGRA_IMAGE_DT_YUV422_8;
According to the TRM, the VI2 can output the following formats to memory.
Only T_A2Y10U10V10, T_V10U10Y10A2, T_Y8__U8__V8_N444, T_Y8__U8V8_N444 and T_Y8__V8U8_N444 are not supported on the TK1. So it is possible to get NV12/I420 to memory from the VI2 unit. Am I correct?
I have marked the formats we are interested in.
VI unit doesn’t do any chroma-decimation in horizontal or vertical direction.
So we should use the same output YUV format as we are getting from the CSI.
We can get YUV422 on CSI and write out 420 memory format.
Please follow the table below (from IAS) to select the CSI format/pixel format combination
that is supported.
There are two formats captured YUV422 and YUV420. Since the YUV420 formats are not being used in current products or are they anticipated to be used in the future, a de-featuring of the planarization support for YUV420 has been proposed. In this proposal, YUV420 data would be captured and stored to memory without planarization.
Just so we are clear about the formats we are discussing about. This is the convention we are using:
UYVY - YUV422 16bpp format (packed)
NV16 - YUV422 16bpp format (semi planar)
NV12 - YUV420 12bpp format (semi planar)
I420 - YUV420 12bpp format (full planar)
Q1.
The above statements are confusing. Do you mean to say that we can get YUV422 from CSI and write it to memory in planar format. Which means that it will be NV16 format (422) and not NV12 format (420).
Q2.
If these are the ONLY formats supported, then it is NOT possible to write NV12 directly to memory. Am I correct? Please give us a clear cut answer.
Q3.
Is T_Y8_U8V8_N420 etc not supported by hardware or not exposed to users? We would like to test and implement NV12 support if the hardware supports it.
My opinion:
I beg to differ. From what I have observed, the video encoder plugins (or any gstreamer plugin from nvidia) uses NV12 or I420 format. List of some of the plugins that use NV12:
It would be immensely helpful to be able to capture NV12 directly to memory as this would reduce a lot of additional overhead required to convert YUV422 to YUV420 in userspace. Not just for me, but for any TK1 user interfacing a CSI camera which can output YUV422.
Of course, if the VI hardware does not have the functionality to write NV12 directly to memory, then there would be no other choice but to convert YUV422 to YUV420 in userspace using nvvidconv. The TRM is a bit confusing and gave us the opinion that the hardware was capable. Kindly clarify this confusion.
Hi
That is the document mistake TRM “31.3.4 YUV Pixel Data Formatting” are correct. For requirement of NV12/YUV420 that could convert by nvvidconv (VIC).
Thank you for your time.
