Alignment issue when debayering images optained with v4l2-ctl

[user27558]

Board: Nvidia Jetson Xavier NX
L4T version: 32.5.1

Hey, I am getting images from my AR0231 RCCB sensor with the following command:

v4l2-ctl --device=/dev/video0 --stream-mmap --stream-to=./frame.raw --stream-count=1 --set-ctrl bypass_mode=0

The camera resolution is 1928x1208 and the file is 4658048 bytes.

AFAIK, the image format is RAW12 (2 bytes with the 4lsb bits to be discarded):

So the file has the right size : 1928⋅1208⋅2 = 4 658 048

But when I debayer the image with opencv it seems that there is an alignment issue and the color is off:

The camera works fine via argus:

What could be the issue ?

Python code I used to debayer the raw file

import cv2

# Load the raw data using OpenCV
raw_data = np.fromfile(file_path, dtype=np.uint16).reshape((height, width))

# Convert the raw data to an 8-bit image for OpenCV processing
raw_data_8bit = (raw_data >> 8).astype(np.uint8)

# Debayer the image using OpenCV
rgb_image_cv = cv2.cvtColor(raw_data_8bit, cv2.COLOR_BAYER_RG2RGB)

# Display the image using OpenCV
plt.figure(figsize=(10, 6))
plt.imshow(cv2.cvtColor(rgb_image_cv, cv2.COLOR_BGR2RGB))
plt.axis('off')
plt.show()

Thanks for your help!

EDIT: Raw file : cam3.raw - Google Drive

[proventusnova]

Hello,

That seems very interesting.

Would it be possible for you to share a raw image file with us so we can play around with it?

Regards,
Andrew
support@proventusnova.com

[user27558]

Hey @proventusnova! you can see the file attached in the post.

[proventusnova]

@user27558 Thanks!

And just out of curiosity.
Why is it that you need to perform the debayering on python, instead of using the IPS with nvargus?

[JerryChang]

hello user27558,

it looks like alignment issue, please also note that we should have follow VI’s 64 byte aligned to set the correct stride.
here’s an example, your active width 1928/64=30.125 which should be configure to 31*64=1984 to follow width alignment.

besides,
you may download MMAPI with… $ sudo apt install nvidia-l4t-jetson-multimedia-api
here’s also another example to demosaic Raw with CUDA. /usr/src/jetson_multimedia_api/argus/samples/cudaBayerDemosaic

[user27558]

Hey @JerryChang and thanks fore your answer.

The sensor maximum resolution is 1928x1208, so you would advise to reduce the active window to 1920x1208 or 1920x1080 ?
I would need to configure the sensor’s active window and update the device tree.

Do you believe it would solve the issue?

Hey @proventusnova

We are experimenting bypassing the ISP to get more control on the image processing

[JerryChang]

hello user27558,

it’s width alignment, so, you may give it a quick try reduce active width as 1920x1208 for testing.