Dear Community,
I am integrating the driver of the Raspberry Pi v3 based on IMX708 developed by RidgeRun for the Jetson Xavier NX based system (the driver originally supports Jetson Nano / Orin Nano).
The RPi camera is connected to the CSI G. I followed the instruction in the following discussions for the port-index of the VI, NVCSI, and the sensor scopes :
and
The device-tree nodes :
- VI :
tegra-capture-vi {
num-channels = <3>;
ports {
#address-cells = <1>;
#size-cells = <0>;
vi_port0: port@0 {
reg = <0>;
vs_vi_in0: endpoint {
port-index = <0>;
bus-width = <4>;
remote-endpoint = <&vs_csi_out0>;
};
};
vi_port1: port@1 {
reg = <1>;
vs_vi_in1: endpoint {
port-index = <4>;
bus-width = <1>;
remote-endpoint = <&vs_csi_out1>;
};
};
vi_port2: port@2 {
reg = <2>;
vs_vi_in2: endpoint {
port-index = <5>;
bus-width = <2>;
remote-endpoint = <&vs_csi_out2>;
};
};
};
};
- NVCSI
nvcsi@15a00000 {
num-channels = <3>;
#address-cells = <1>;
#size-cells = <0>;
csi_chan0: channel@0 {
reg = <0x0>;
ports {
#address-cells = <1>;
#size-cells = <0>;
csi_chan0_port0: port@0 {
reg = <0>;
vs_csi_in0: endpoint@0 {
port-index = <0>;
bus-width = <4>;
remote-endpoint = <&vs_cam_out0>;
};
};
csi_chan0_port1: port@1 {
reg = <1>;
vs_csi_out0: endpoint@1 {
remote-endpoint = <&vs_vi_in0>;
};
};
};
};
csi_chan1: channel@1 {
reg = <0x1>;
discontinuous_clk = "no";
ports {
#address-cells = <1>;
#size-cells = <0>;
csi_chan1_port0: port@0 {
reg = <0>;
vs_csi_in1: endpoint@2 {
port-index = <4>;
bus-width = <1>;
remote-endpoint = <&vs_cam_out1>;
};
};
csi_chan1_port1: port@1 {
reg = <1>;
vs_csi_out1: endpoint@3 {
remote-endpoint = <&vs_vi_in1>;
};
};
};
};
csi_chan2: channel@2 {
reg = <0x2>;
ports {
#address-cells = <1>;
#size-cells = <0>;
csi_chan2_port0: port@0 {
reg = <0>;
vs_csi_in2: endpoint@4 {
port-index = <6>;
bus-width = <2>;
remote-endpoint = <&vs_cam_out2>;
};
};
csi_chan2_port1: port@1 {
reg = <1>;
vs_csi_out2: endpoint@5 {
remote-endpoint = <&vs_vi_in2>;
};
};
};
};
};
};
- IMX708 :
i2c@3180000 {
status = "okay";
#address-cells = <1>;
#size-cells = <0>;
rpiv3_imx708_g@1a {
status = "okay";
compatible = "sony,imx708";
reg = <0x1a>;
devnode = "video2";
physical_w = "3.680";
physical_h = "2.760";
sensor_model = "imx708";
use_sensor_mode_id = "true";
reset-gpios = <&tegra_aon_gpio RPI_CAM_PWDN GPIO_ACTIVE_HIGH>;
/* IMX708_MODE_4608x2592_14fps */
mode0 {
mclk_khz = "24000";
num_lanes = "2";
tegra_sinterface = "serial_g";
phy_mode = "DPHY";
discontinuous_clk = "yes";
dpcm_enable = "false";
cil_settletime = "0";
lane_polarity = "6";
active_w = "4608";
active_h = "2592";
mode_type = "bayer";
pixel_phase = "rggb";
csi_pixel_bit_depth = "10";
readout_orientation = "90";
line_length = "4808";
inherent_gain = "1";
pix_clk_hz = "297600000";
gain_factor = "16";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16";
max_gain_val = "256";
step_gain_val = "1";
default_gain = "16";
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000";
max_framerate = "14000000";
step_framerate = "1";
default_framerate = "14000000";
min_exp_time = "500";
max_exp_time = "65487";
step_exp_time = "1";
default_exp_time = "1600";
embedded_metadata_height = "4";
};
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
vs_cam_out2: endpoint {
port-index = <5>; /* CSI G */
bus-width = <2>;
remote-endpoint = <&vs_csi_in2>;
};
};
};
};
};
(There are also other cameras/sensors connected to CSI A (4 lanes) and CSI E (1 lane)).
The testing :
- Maximize clocks
tuxvision@ubuntu:~$ sudo su
root@ubuntu:/home/tuxvision# echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
root@ubuntu:/home/tuxvision# echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
root@ubuntu:/home/tuxvision# echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
root@ubuntu:/home/tuxvision# cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
460800000
root@ubuntu:/home/tuxvision# cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
576000000
root@ubuntu:/home/tuxvision# cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
314000000
root@ubuntu:/home/tuxvision# exit
exit
- Verify the bound device :
tuxvision@ubuntu:~$ v4l2-ctl --all -d /dev/video2
Driver Info:
Driver name : tegra-video
Card type : vi-output, imx708 2-001a
Bus info : platform:tegra-capture-vi:5
Driver version : 5.10.104
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : tegra-camrtc-ca
Model : NVIDIA Tegra Video Input Device
Serial :
Bus info :
Media version : 5.10.104
Hardware revision: 0x00000003 (3)
Driver version : 5.10.104
Interface Info:
ID : 0x0300002c
Type : V4L Video
Entity Info:
ID : 0x0000002a (42)
Name : vi-output, imx708 2-001a
Function : V4L2 I/O
Pad 0x0100002b : 0: Sink
Link 0x02000030: from remote pad 0x1000009 of entity '13e10000.host1x:nvcsi@15a00000-': Data, Enabled
Priority: 2
Video input : 0 (Camera 5: ok)
Format Video Capture:
Width/Height : 4608/2592
Pixel Format : 'RG10' (10-bit Bayer RGRG/GBGB)
Field : None
Bytes per Line : 9216
Size Image : 23887872
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
Camera Controls
group_hold 0x009a2003 (bool) : default=0 value=0 flags=execute-on-write
sensor_mode 0x009a2008 (int64) : min=0 max=1 step=1 default=0 value=0 flags=slider
gain 0x009a2009 (int64) : min=16 max=256 step=1 default=16 value=16 flags=slider
exposure 0x009a200a (int64) : min=500 max=65487 step=1 default=1600 value=500 flags=slider
frame_rate 0x009a200b (int64) : min=2000000 max=14000000 step=1 default=14000000 value=2000000 flags=slider
sensor_configuration 0x009a2032 (u32) : min=0 max=4294967295 step=1 default=0 [22] flags=read-only, volatile, has-payload
sensor_mode_i2c_packet 0x009a2033 (u32) : min=0 max=4294967295 step=1 default=0 [1026] flags=read-only, volatile, has-payload
sensor_control_i2c_packet 0x009a2034 (u32) : min=0 max=4294967295 step=1 default=0 [1026] flags=read-only, volatile, has-payload
bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
0: 0 (0x0)
1: 1 (0x1)
override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
0: 0 (0x0)
1: 1 (0x1)
height_align 0x009a2066 (int) : min=1 max=16 step=1 default=1 value=1
size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
0: 1 (0x1)
1: 65536 (0x10000)
2: 131072 (0x20000)
write_isp_format 0x009a2068 (int) : min=1 max=1 step=1 default=1 value=1
sensor_signal_properties 0x009a2069 (u32) : min=0 max=4294967295 step=1 default=0 [30][18] flags=read-only, has-payload
sensor_image_properties 0x009a206a (u32) : min=0 max=4294967295 step=1 default=0 [30][16] flags=read-only, has-payload
sensor_control_properties 0x009a206b (u32) : min=0 max=4294967295 step=1 default=0 [30][36] flags=read-only, has-payload
sensor_dv_timings 0x009a206c (u32) : min=0 max=4294967295 step=1 default=0 [30][16] flags=read-only, has-payload
low_latency_mode 0x009a206d (bool) : default=0 value=0
preferred_stride 0x009a206e (int) : min=0 max=65535 step=1 default=0 value=0
sensor_modes 0x009a2082 (int) : min=0 max=30 step=1 default=30 value=1 flags=read-only
- Stream with V4L2 but FAILED as the below v4l2-ctl command hung :
tuxvision@ubuntu:~$ v4l2-ctl --set-fmt-video=width=4608,height=2592,pixelformat=RG10 --stream-mmap --stream-count=100 -d /dev/video2
^C^C
and dmesg gave :
[ 213.994160] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 213.994503] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 213.995268] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 213.995431] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=5, csi_port=6
[ 213.995680] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 213.995838] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 5 vc- 0
[ 213.996436] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 216.554148] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 216.554425] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 216.555254] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 216.555404] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=5, csi_port=6
[ 216.555636] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 216.555778] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 5 vc- 0
[ 216.556363] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 219.114115] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 219.114353] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 219.115248] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 219.115419] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=5, csi_port=6
[ 219.115652] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 219.115943] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 5 vc- 0
[ 219.116528] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 221.674132] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 221.674413] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 221.675339] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 221.675512] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=5, csi_port=6
[ 221.675738] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 221.675910] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 5 vc- 0
[ 221.676501] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 224.202113] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 224.202341] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 224.203245] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 224.203466] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=5, csi_port=6
[ 224.203730] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 224.203897] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 5 vc- 0
[ 224.204433] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
Could you help to point out the issue, please?
Note : I am using Jetpack-5.1(L4T-R35.2.1).
Thanks in advance and best regards,
Khang