How best to handle RAW8 double wide data type through VI5?

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Our camera is sending RAW8 double wide data type and we’re able to receive the frames through the NVCSI/VI block with the following video format settings:

TEGRA_VIDEO_FORMAT(RAW8, 8, SGRBG8_1X8, 1, 1, T_R8, RAW8, SGRBG8, "GRGR.. BGBG.."),

By doubling the width size to 3840 with RAW8 data type, we can receive the image through the NVCSI/VI block correctly. The image looks good.

We would like the VI block to merge the 2x RAW8 and swap the bytes for us, however. The TRM describes a register that I think should work for us (VI_CH_n_PIXFMT_WIDE_).

I made these changes here but it has NOT worked yet:

vi5_fops.c::vi5_setup_surface()
…
desc->ch_cfg.pixfmt_wide_enable = 1;
desc->ch_cfg.pixfmt_wide_endian = 1;
…

With the bits set above, and 2x RAW8 merged, I’d expect a T_R16 from the VI5 block. Is this correct?

We want to then tell the V4L2 driver to expect a SGRBG16_1X16 from the table below. The assumption is that the VI merged the 2x RAW8 into a RAW16 and now the VI is going to DMA it in T_R16 format. Is this correct?

TEGRA_VIDEO_FORMAT(RAW16, 16, SGRBG16_1X16, 2, 1, T_R16, RAW16, SGRBG16, "GRGR.. BGBG.."),

Do I understand the logic correct?

Setting the bits above didn’t work for us.

2

hello kurt_ottaway,

currently, our camera software driver has not implemented to support merging two RAW8 data into RAW16 input.
you may also check similar discussion thread, Topic 1042926 for reference,
thanks

3

Thanks for the quick response! I have already read the other thread related to the subject.

I already know the nvidia driver doesn’t support the merging of 2x RAW8 which is why we made the code changes.

My question is:

If we set the bits as described in the TRM, should the 2x RAW8 get merged as defined in register VI_CH_n_PIXFMT_WIDE_? i.e. Can you confirm if hardware supports these bits? even if software does not support it. If hardware supports it then we’ll try and add the support. If the VI5 doesn’t actually support the feature then it can take a long time figuring out it isn’t even supported.

Can you propose another solution for how best to byte swap or merge the 2x RAW8 into a RAW16 in hardware? or are we forced to handle the merge/swap after the frame has been received on the Target?

4

Hello JerryChang,

I have another issue come with the VI block settings.

Now I can merge and swap the image from RAW8 to RAW16 using:

desc->ch_cfg.pixfmt_wide_enable = 1;
desc->ch_cfg.pixfmt_wide_endian = 1;

The problem is

I have the resolution of the input RAW8 is 3840x1080, I expect the output from VI block has resolution=1920x1080.

But v4l2-ctl cannot capture 1920x1080, RAW16 format from VI block:
–set-fmt-video=width=1920,height=1080,pixelformat=GR16

I have to use 3840x1080 and RAW16 to capture a double sized image, which has half black(value=0)at right side in its frame:
–set-fmt-video=width=3840,height=1080,pixelformat=GR16

I think VI block send out a buffer of size=3840x1080.
Then I want to crop the image after merge and swap, so I tried:

desc->ch_cfg.frame.crop.x = 1920;
desc->ch_cfg.frame.crop.y = 0;

but it cropped my input RAW8 image and get merge and swap on the cropped image.

Can anyone provide an idea how to crop the image buffer in VI block, or let me know if it is possible to crop.

Thank you very much.

5

hello kurt_ottaway and debbie.guo,

FYI,
VI driver did not implemented to support merging two RAW8 data into RAW16 input.
also, multi-brick streams (i.e. gang mode) also did not implemented for Xavier’s VI drivers. please refer to Topic 1064874.

could you please dump the raw files to confirm the buffers.

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=3840,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw

if it outputs as expect capture results, you may enable gstreamer pipeline for adding controls.
here’s an example to crop and scale video captures, Video capture crop and scaling.
thanks