For Coherent Image, Need to Shift Pixels in Odd Rows for IMX219 CSI Camera

Platform: Orin Nano Development Kit with numerous IMX219 based CSI cameras, R35 (release), REVISION: 4.1, GCID: 33958178

Comparison Platform: Xavier NX with same cameras, R32 (release), REVISION: 7.2, GCID: 30192233

Executive summary: On Xavier NX, all cameras work with all utilities (v4l2-ctl, gstreamer, etc.) but on Orin Nano, I need to shift the odd rows right by 16 pixels to get them to line up with the even rows.

The first (slight) difference is for the following command:

v4l2-ctl --list-formats-ext

Orin Nano:

	[0]: 'RG10' (10-bit Bayer RGRG/GBGB)
		Size: Discrete 3280x2464
			Interval: Discrete 0.048s (21.000 fps)
		Size: Discrete 3280x1848
			Interval: Discrete 0.036s (28.000 fps)

Xavier NX:

	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 3264x2464
			Interval: Discrete 0.048s (21.000 fps)
		Size: Discrete 3264x1848
			Interval: Discrete 0.036s (28.000 fps)

I’m guessing the problem I’m having stems from the 3280x2464 versus the 3264x2464. It’s the exact same camera so why would the width and height of sensor mode zero be different?

Next command run on both systems:

v4l2-ctl -d /dev/video0 -c override_enable=1 -c bypass_mode=0 \
   --set-ctrl sensor_mode=0 --set-fmt-video=width=3280,height=2464,pixelformat=RG10 \
   -c exposure=33333 -c frame_rate=10000000 \
   --stream-mmap --stream-count=1 --stream-to=frame.raw

Note that it doesn’t change the results whether or not I use 3280 or 3264 for this command. The --set-ctrl sensor_mode=0 seems to define the width no matter what? This command creates frame.raw. I use the following simple python program to convert it to something viewable (pgm and ppm files):

import sys
import cv2
import numpy as np

# get parameters
width = int( sys.argv[1] )
height = int( sys.argv[2] )
shift = int( sys.argv[3] )

#read the file
with open( "frame.raw", "rb") as f:
    raw_data =

# Scale to 8-bit
data = np.frombuffer(raw_data, dtype=np.uint16).reshape(height, width)
data = ( data / 257 ).astype(np.uint8)

# save pgm
cv2.imwrite( "frame.pgm", data )

# shift and save shifted pgm 
shifted_data = np.copy( data )
for iRow in range(1, data.shape[ 0 ], 2 ):
    shifted_data[iRow] = np.roll(shifted_data[iRow], shift )
cv2.imwrite( "frame_shift.pgm", shifted_data )

# convert bayer to color and save
rgb_data = cv2.cvtColor( shifted_data, cv2.COLOR_BayerRG2BGR )
cv2.imwrite( "frame_shift.ppm", rgb_data )

Running the python code on the Orin Nano:

./ 3280 2464 0

Running the python code on the Xavier NX:

./ 3264 2464 0

Note that here the number of columns does matter here because the previous v4l2-ctl command creates different file size for frame.raw.

The results at this point show images with weird shadow effects for the Orin Nano and perfectly fine images for the Xavier NX.

It took a while to figure out what was causing the weird shadow effects but eventually I discovered that the odd numbered rows need to be shifted 16 pixels right relative to the even numbered rows. That is why added the shifting code to the python code and using the following alternate command yields good images (except at the left and right edges):

./ 3280 2464 16

Why do I need to do that shift of 16 pixels to correct the image and what can I do to avoid needing to do that? Why does the exact same configuration that works on a Xavier NX not work on the Orin Nano Development Kit?

Further notes:
Gstreamer work fine on both.

Could you confirm XNX on r35 release.
Also does 1920x1080 mode have the same problem?

I don’t know what you mean by “XNX on r35 release.” The Orin Nano Dev Kit was installed using sdkmanager. The Xavier NX was also installed using sdkmanager.

1920x1080 mode (sensor mode 2) does NOT have the same problem (i.e. no shift is required for a good image).

3280x1848 mode (sensor mode 1) DOES have the same problem (a right shift of 16 is required on odd rows for a good image).

1640x1232 mode (sensor mode 3) is NOT fixable with any shift - the image remains garbled no matter what.

Other notes: I have two other Orins (1 NX, 1 Nano) and they both exhibit the same problem.

Xavier NX on r35.4.1 to confirm the problem relative with the output width.


Ah. I was afraid of that. I borrowed the Xavier NX from another project and I don’t have leave to install R35 on it.

I’ll see if I can find a different one.

Yes, same problem on Xavier NX on r35.4.1.

From this I have confidence say the VI have 32 alignment limitation.


OK. So how do I fix it?

You have to handle it if the output size doesn’t 32 alignment like you shift the pixels.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.