Platform: Orin Nano Development Kit with numerous IMX219 based CSI cameras, R35 (release), REVISION: 4.1, GCID: 33958178
Comparison Platform: Xavier NX with same cameras, R32 (release), REVISION: 7.2, GCID: 30192233
Executive summary: On Xavier NX, all cameras work with all utilities (v4l2-ctl, gstreamer, etc.) but on Orin Nano, I need to shift the odd rows right by 16 pixels to get them to line up with the even rows.
The first (slight) difference is for the following command:
v4l2-ctl --list-formats-ext
Orin Nano:
[0]: 'RG10' (10-bit Bayer RGRG/GBGB)
Size: Discrete 3280x2464
Interval: Discrete 0.048s (21.000 fps)
Size: Discrete 3280x1848
Interval: Discrete 0.036s (28.000 fps)
...
Xavier NX:
Name : 10-bit Bayer RGRG/GBGB
Size: Discrete 3264x2464
Interval: Discrete 0.048s (21.000 fps)
Size: Discrete 3264x1848
Interval: Discrete 0.036s (28.000 fps)
...
I’m guessing the problem I’m having stems from the 3280x2464 versus the 3264x2464. It’s the exact same camera so why would the width and height of sensor mode zero be different?
Next command run on both systems:
v4l2-ctl -d /dev/video0 -c override_enable=1 -c bypass_mode=0 \ --set-ctrl sensor_mode=0 --set-fmt-video=width=3280,height=2464,pixelformat=RG10 \ -c exposure=33333 -c frame_rate=10000000 \ --stream-mmap --stream-count=1 --stream-to=frame.raw
Note that it doesn’t change the results whether or not I use 3280 or 3264 for this command. The --set-ctrl sensor_mode=0 seems to define the width no matter what? This command creates frame.raw. I use the following simple python program to convert it to something viewable (pgm and ppm files):
#!/usr/bin/python3
import sys
import cv2
import numpy as np
# get parameters
width = int( sys.argv[1] )
height = int( sys.argv[2] )
shift = int( sys.argv[3] )
#read the file
with open( "frame.raw", "rb") as f:
raw_data = f.read()
# Scale to 8-bit
data = np.frombuffer(raw_data, dtype=np.uint16).reshape(height, width)
data = ( data / 257 ).astype(np.uint8)
# save pgm
cv2.imwrite( "frame.pgm", data )
# shift and save shifted pgm
shifted_data = np.copy( data )
for iRow in range(1, data.shape[ 0 ], 2 ):
shifted_data[iRow] = np.roll(shifted_data[iRow], shift )
cv2.imwrite( "frame_shift.pgm", shifted_data )
# convert bayer to color and save
rgb_data = cv2.cvtColor( shifted_data, cv2.COLOR_BayerRG2BGR )
cv2.imwrite( "frame_shift.ppm", rgb_data )
Running the python code on the Orin Nano:
./imgfromraw.py 3280 2464 0
Running the python code on the Xavier NX:
./imgfromraw.py 3264 2464 0
Note that here the number of columns does matter here because the previous v4l2-ctl command creates different file size for frame.raw.
The results at this point show images with weird shadow effects for the Orin Nano and perfectly fine images for the Xavier NX.
It took a while to figure out what was causing the weird shadow effects but eventually I discovered that the odd numbered rows need to be shifted 16 pixels right relative to the even numbered rows. That is why added the shifting code to the python code and using the following alternate command yields good images (except at the left and right edges):
./imgfromraw.py 3280 2464 16
Why do I need to do that shift of 16 pixels to correct the image and what can I do to avoid needing to do that? Why does the exact same configuration that works on a Xavier NX not work on the Orin Nano Development Kit?
Further notes:
Gstreamer work fine on both.