Raw data format through v4l2

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

I am comparing the output of the raw data captured through v4l2 on the Xavier NX and Nano platforms, and found an unexpected difference.

I configure the camera to create a very bright image (to better see the data), and using the command

v4l2-ctl --stream-mmap --stream-count=1 --stream-to=raw.dat

I get the following output on the Nano (seen in a hex editor):

FF 0F FF 0F …

This is consistent with the RG12 pixel format, that v4l2 declares to deliver when asking for formats. The information is in the 12 least significant bits.

However, when doing exactly the same test on a Xavier NX (same camera, same driver software, same device tree modifications), I get the following output:

FF FF FF FF …

When interpreting that image data in further processing, it becomes clear that it is in fact not RG12, but the most significant 12 bits contain the information from the camera sensor (which would be the RG16 format, if I am not mistaken).

Is this a known difference in behavior between the Nano and the NX, or is there some kind of configuration that I might have changed unknowingly that can cause this change in the data?

1 Like
3

Did you check the pixel format in the memory from TRM?

raw_memory_layout
raw_memory_layout2158×1121 203 KB

4

If that was the output format, shouldn’t I capture data that always has the top 2 bits cleared? Instead, I get data with all bits set.
Is there a way to find out which of the T_R16* formats is actually configured in the system? How would I configure that?

Also, are you referring to this document: Xavier_TRM_DP09253002_v1.4p.pdf ?
I find a different graphic in there, that suggests that everything is shifted one more bit to the left. However, this would still lead to a leading zero that I don’t see in my data.

Additionally, is there any explanation for why the Nano behaves differently? In the TX1 TRM, I find a similar diagram than the one you posted, and it does not in any way match the data I see either.

5

I don’t remember this is from TX1 or TX2 but I think they should be similar.
For NX you need to check the Xavier TRM.

6

Yes, I did check the Xavier TRM, and found the diagram that is at least similar to the one you posted above. Is yours from a previous version of the Xavier TRM?

In any case, both diagrams suggest that there are zeros in the top bits, which I do not see in the data I get out of v4l2. So I wonder why that might be …

7

Could it be MSB LSB reverse?

8

I cannot imaging a way where a bit reversal or byte swap would turn zeros into ones?

One thing about that documentation is true though, it does indeed fill up the low bits with bits from the top. So what I see on the Xavier NX is like

D11 D10 D9 D8 D7 D6 D5 D4 D3 D2 D1 D0 D11 D10 D9 D8

while on the Nano, I see

0 0 0 0 D11 D10 D9 D8 D7 D6 D5 D4 D3 D2 D1 D0

Bit/byte order is fine, the image looks perfectly normal when interpreting the data in this way.

We are developing products that customers can attach to both Xavier and Nano platforms, so I really would like to understand what is going on, and not blindly accept what I see as given.

And ideally find out whether there is a way to control this behavior.

9

Just check the looks like vi5(NX) set the video_formats as T_R16 different with Nano T_R16_I,
Have modify it to confirm it.

10

Interesting. You are right, vi5_formats.h and vi4_formats.h seem to differ where they assign T_R16 or T_R16_I to the RAW12 pixel formats.

Is it possible to control the VI format from the outside, without modifying kernel code?

Is there any explanation of what the definition of T_R16 is? I don’t see it in the TRM.

11

T_R16_I has 0 on MSB, so it’s 1-bit right shifted from T_R16