Camera test pattern image distorted

[atharvanandanwar]

Hello,

I’m currently working with Jetson AGX Xavier module with a camera which connects to an FPGA and goes through a GMSL serializer and deserializer. Here, we are attempting to test FPGA → Serializer → Deserializer → Xavier connection. FPGA is generating the test pattern at 1928x1208.

I am able to capture a YUV422 8-bit image, which is what the FPGA is sending over to the serializer. However, we have noticed that the image that is being captured using:
yavta -n 3 -c1 -f UYVY -s 1928x1208 -Ftest.yuv /dev/video0
does not contain the correct data sequence.

In this test image, we are sending UYVY packed with
U - 0, Y - 0, V - 0, Y_next - 0. Containing same data for 4 bytes which represent two pixels. Here is the screenshot with pixel data from the start of the image.

This was viewed using GitHub - IENT/YUView: The Free and Open Source Cross Platform YUV Viewer with an advanced analytics toolset with resolution set to 1928x1208 - which is the resolution FPGA is sending the test pattern. Interesting thing is that at the end of the image we are getting the following:

This is unexpected, as in our test pattern we are NOT resetting the count from 191 → 0, but rather it should go to 191->195 until the end of the line. 0 shall start from the next time. To summarize,

16 bytes of data from the line is missing, and is replaced by 16 bytes of data from the next line.

Another interesting thing that we observe is:

On line 4 we see that we count till 195 (which is what it should be), however we see that we are observing zeros on the data that is seemingly replaced in the previous row.

To summarize,

1. We are missing 32 bytes of data from Line 1, 2 and 3.
2. This appears at the end of line 4 where remaining pixels are set to zero.
3. We are somehow seeing the full data in Line 4.

Question/Query:

How can one go about debugging this from Xavier perspective (yes we are actively trying to debug this from FPGA, serializer, deserializer)? What device tree settings shall we look? What kernel parameters shall be look at?

Thanks in advance!

[ShaneCCC]

Please make the output size to 32 alignment like 1920x1080 due to VI have 32 alignment limited.

Thanks

[atharvanandanwar]

Hey ShaneCCC,

Thanks for quick reply. What does it mean by

out size to 32 alignment like 1920x1080

Does 1920 and 1080 needs to be individually 32 bit aligned? I’ll proceed with placing

active_w = “1920”;
active_h = “1080”;
line_length = “1920”;

Do we need to change any other settings? I’d also be changing this on the driver end.

Thanks again!

[ShaneCCC]

I mean output width to 32 alignment.

[atharvanandanwar]

So, I’d change the 1928 → 1920. My question to you would be similar as above,

1. Where does vi expects to find this?
   • So far, I understand that there is kernel code that reads from the device tree to set few things, such as line_length for timing calculations for the vi pipeline.

Also, can you let me know where can I find information on vi being 32 aligned?

Thanks!

[ShaneCCC]

You can ignore the line_length due to it only for Bayer sensor.
For VI alignment not sure if TRM have it or not. If not the we don’t have public document for it now.

[atharvanandanwar]

Hey ShaneCCC,

Thanks a lot for your help here.

I’ll try changing the resolution to 1920x1208 and see what I can find. I’ll report back with my findings.

Thanks!

[atharvanandanwar]

Changing the resolution internally is leading to

[  448.201045] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 256

It’s resulting into CHANSEL_FAULT when looked at the rtcpu logs:

CHANSEL_FAULT channel:0x23 frame:0 vi_tstamp:14221560055 data:0x04b70182

PIXEL_LONG_LINE, PIXEL_SPURIOUS and PIXEL_EOF errors as per Jetson/l4t/Camera BringUp - eLinux.org

As per Xavier TRM Page 1386, it means:

{PIXEL,EMBED}_LONG_LINE: More pixels (or embedded data bytes) were received than expected in a line.

{PIXEL,EMBED}_SPURIOUS: A pixel data packet was received without a line-start packet first.

PXL_SOF / PXL_EOF / EMBED_SOF / EMBED_EOF (normal) are emitted under their eponymous conditions.

Looks like we would also have to change the resolution of the MIPI packets coming in from the image sensor.

[atharvanandanwar]

@ShaneCCC I’m curious about the case in Image #3 that I posted in the original post.

Why does it realigns on every 4th line?
Why do we wrap data from the next line into the previous line?

It would help me out a lot to understand the cause.

Thanks!

[ShaneCCC]

What’s the “Bytes per Line”?

nvidia@nvidia-desktop:~$ v4l2-ctl --all
Driver Info (not using libv4l2):
        Driver name   : tegra-video
        Card type     : vi-output, ov5693 30-0036
        Bus info      : platform:15700000.vi:0
        Driver version: 4.9.253
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: no power)
Format Video Capture:
        Width/Height      : 2592/1944
        Pixel Format      : 'BG10'
        Field             : None
        Bytes per Line    : 5376
        Size Image        : 10450944
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Flags             :
        Max Framerate     : 0

Camera Controls

[atharvanandanwar]

Hey ShaneCCC,

After changing the resolution to 1920x1208, Bytes per Line value is 3840. I guess, under resolution of 1928x1208, Bytes per Line would be 3856 - considering 32 bytes of alignment (?), this would cuts off 16 bytes of data = 8 pixels of data effectively. This explains why we see that offset.

That still doesn’t explain why we would get full line data in 4th line.

Thanks again for your input here!

[ShaneCCC]

Sorry, I can’t tell why you get full line data in 4th line.

Does the image looks un-normally?
Could you attached the output file here.

Also try set preferred_stride by v4l2-ctl before to capture.

v4l2-ctl -c preferred_stride=1952

[atharvanandanwar]

That doesn’t help. However, we changed the resolution to 1920 and that worked fine.