We have designed a custom carrier with RPi-style 15-pin interface connected to CSI4 interface and a max96724 GMSL chip to two 4-lane MIPI interfaces: CSI0-1 and CSI2-3. We’d like to understand how to adapt the Device Tree files to get them to work. There is no I2C mux chip on this baseboard.
The following questions arise:
Where can we find DTS docs describing exactly how to configure the CSI-MIPI core (thru DTS and drivers, nvcsi block, naming A to G in some places) and its many modes to set number of lanes, aggregating CSI0 and 1 interfaces for 4-lane operation, etc.? Is there a DTS code example?
How can we configure the MIPI interface to support virtual channels?
Is there any driver support for max96724?
How to configure DTS to remove I2C multiplexer? IMX477 RPi camera is found on startup, so i2c connection looks ok. But when we want to stream data from the camera, we don’t get any information back – presumably because we haven’t setup the MIPI interface correctly yet. Our guess is that we need to switch the CSI lane to #4 but we could not figure out csi interface indexing so far. How can that be done? (using camera feed on MIPI CSI4, 2 lane MIPI, single camera).
Q1.
do you total have three cameras in the system? i.e. 2x 4-lane camera with VC supports, and 1x IMX477?
please refer to Jetson Xavier NX Product Design Guide. Jetson Xavier NX brings fourteen MIPI CSI lanes to the connector. you may also check [Figure 9-2. CSI Connection Options] for reference. and, please also refer to Port Index section for the CSI/VI port bindings definition.
Q2
you may see developer guide, Jetson Virtual Channel with GMSL Camera Framework, and reference driver for using virtual channel supports.
for instance, $public_sources/kernel_src/hardware/nvidia/platform/t19x/common/kernel-dts/t19x-common-modules/tegra194-camera-imx390-a00.dtsi
Q3.
by default it only support max9295/max9296.
Q4.
just be sure you’ve correct i2c assignment to the camera node, then you may update device tree to exclude that cam_i2cmux.
For starters we only have one camera attached to mipi connector. GMSL is not used for now. Later, we would like to use either GMSL with two cameras or mipi with 1 camera.
We have this device tree config:
/ {
tegra-capture-vi {
compatible = "nvidia,tegra-camrtc-capture-vi";
nvidia,vi-devices = <0x23b>;
nvidia,vi-mapping-size = <0x06>;
nvidia,vi-mapping = <0x00 0x00 0x01 0x00 0x02 0x00 0x03 0x00 0x04 0x00 0x05 0x00>;
nvidia,vi-mapping-names = "csi-stream-id\0vi-unit-id";
nvidia,vi-max-channels = <0x24>;
status = "okay";
num-channels = <0x02>;
ports {
#address-cells = <0x01>;
#size-cells = <0x00>;
port@0 {
reg = <0x00>;
phandle = <0x3f9>;
endpoint {
status = "okay";
port-index = <0x00>;
bus-width = <0x02>;
remote-endpoint = <0x247>;
phandle = <0x268>;
};
};
};
};
host1x@13e00000 {
compatible = "nvidia,tegra194-nvcsi";
clocks = <0x04 0x51>;
clock-names = "nvcsi";
num-channels = <0x02>;
#address-cells = <0x01>;
#size-cells = <0x00>;
phandle = <0x23c>;
prod-settings {
#prod-cells = <0x04>;
prod {
prod = <0x00 0x11018 0x70000 0x20000 0x00 0x1109c 0xe000e000 0xc000c000 0x00 0x110a8 0xfefe 0x4c4c 0x00 0x11128 0xe000e000 0xc000c000 0x00 0x11134 0xfefe 0x4c4c 0x00 0x21018 0x70000 0x20000 0x00 0x2109c 0xe000e000 0xc000c000 0x00 0x210a8 0xfefe 0x4c4c 0x00 0x21128 0xe000e000 0xc000c000 0x00 0x21134 0xfefe 0x4c4c 0x00 0x31018 0x70000 0x20000 0x00 0x3109c 0xe000e000 0xc000c000 0x00 0x310a8 0xfefe 0x4c4c 0x00 0x31128 0xe000e000 0xc000c000 0x00 0x31134 0xfefe 0x4c4c 0x00 0x41018 0x70000 0x20000 0x00 0x4109c 0xe000e000 0xc000c000 0x00 0x410a8 0xfefe 0x4c4c 0x00 0x41128 0xe000e000 0xc000c000 0x00 0x41134 0xfefe 0x4c4c 0x00 0x1102c 0xfc000000 0xe0000000 0x00 0x110b8 0xfc000000 0xe0000000 0x00 0x2102c 0xfc000000 0xe0000000 0x00 0x210b8 0xfc000000 0xe0000000 0x00 0x3102c 0xfc000000 0xe0000000 0x00 0x310b8 0xfc000000 0xe0000000 0x00 0x4102c 0xfc000000 0xe0000000 0x00 0x410b8 0xfc000000 0xe0000000>;
};
prod_c_cphy_mode {
prod = <0x00 0x11030 0x7f7f7f 0x444444 0x00 0x110bc 0x7f7f7f 0x444444 0x00 0x21030 0x7f7f7f 0x444444 0x00 0x210bc 0x7f7f7f 0x444444 0x00 0x31030 0x7f7f7f 0x444444 0x00 0x310bc 0x7f7f7f 0x444444 0x00 0x41030 0x7f7f7f 0x444444 0x00 0x410bc 0x7f7f7f 0x444444 0x00 0x110a8 0xffff 0xfcfc 0x00 0x1109c 0xffffffff 0xd848d848 0x00 0x110a0 0x03 0x00 0x00 0x11098 0x03 0x03 0x00 0x11134 0xffff 0xfcfc 0x00 0x11128 0xffffffff 0xd848d848 0x00 0x1112c 0x03 0x00 0x00 0x11124 0x03 0x03 0x00 0x210a8 0xffff 0xfcfc 0x00 0x2109c 0xffffffff 0xd848d848 0x00 0x210a0 0x03 0x00 0x00 0x21098 0x03 0x03 0x00 0x21134 0xffff 0xfcfc 0x00 0x21128 0xffffffff 0xd848d848 0x00 0x2112c 0x03 0x00 0x00 0x21124 0x03 0x03 0x00 0x310a8 0xffff 0xfcfc 0x00 0x3109c 0xffffffff 0xd848d848 0x00 0x310a0 0x03 0x00 0x00 0x31098 0x03 0x03 0x00 0x31134 0xffff 0xfcfc 0x00 0x31128 0xffffffff 0xd848d848 0x00 0x3112c 0x03 0x00 0x00 0x31124 0x03 0x03 0x00 0x410a8 0xffff 0xfcfc 0x00 0x4109c 0xffffffff 0xd848d848 0x00 0x410a0 0x03 0x00 0x00 0x41098 0x03 0x03 0x00 0x41134 0xffff 0xfcfc 0x00 0x41128 0xffffffff 0xd848d848 0x00 0x4112c 0x03 0x00 0x00 0x41124 0x03 0x03>;
};
prod_c_dphy_mode {
prod = <0x00 0x11030 0x7f7f7f 0x00 0x00 0x110bc 0x7f7f7f 0x00 0x00 0x21030 0x7f7f7f 0x00 0x00 0x210bc 0x7f7f7f 0x00 0x00 0x31030 0x7f7f7f 0x00 0x00 0x310bc 0x7f7f7f 0x00 0x00 0x41030 0x7f7f7f 0x00 0x00 0x410bc 0x7f7f7f 0x00>;
};
};
channel@0 {
reg = <0x00>;
phandle = <0x410>;
ports {
#address-cells = <0x01>;
#size-cells = <0x00>;
port@0 {
reg = <0x00>;
phandle = <0x411>;
endpoint@0 {
status = "okay";
port-index = <0x00>;
bus-width = <0x02>;
remote-endpoint = <0x460>;
phandle = <0x2b8>;
};
};
port@4 {
reg = <0x04>;
phandle = <0x412>;
endpoint@1 {
status = "okay";
remote-endpoint = <0x268>;
phandle = <0x247>;
};
};
};
};
};
};
cam_i2cmux {
i2c@0 {
reg = <0x00>;
#address-cells = <0x01>;
#size-cells = <0x00>;
phandle = <0x45e>;
rbpcv3_imx477_a@1a {
compatible = "ridgerun,imx477";
reg = <0x1a>;
devnode = "video0";
physical_w = "3.680";
physical_h = "2.760";
sensor_model = "imx477";
use_sensor_mode_id = "true";
status = "okay";
reset-gpios = <0x0c 0x7c 0x00>;
phandle = <0x45f>;
mode0 {
...
};
mode1 {
...
};
ports {
#address-cells = <0x01>;
#size-cells = <0x00>;
port@0 {
reg = <0x00>;
endpoint {
port-index = <0x00>;
bus-width = <0x02>;
remote-endpoint = <0x2b8>;
phandle = <0x460>;
};
};
};
};
};
};
};
/ {
tegra-camera-platform {
compatible = "nvidia, tegra-camera-platform";
/**
* Physical settings to calculate max ISO BW
*
* num_csi_lanes = <>;
* Total number of CSI lanes when all cameras are active
*
* max_lane_speed = <>;
* Max lane speed in Kbit/s
*
* min_bits_per_pixel = <>;
* Min bits per pixel
*
* vi_peak_byte_per_pixel = <>;
* Max byte per pixel for the VI ISO case
*
* vi_bw_margin_pct = <>;
* Vi bandwidth margin in percentage
*
* max_pixel_rate = <>;
* Max pixel rate in Kpixel/s for the ISP ISO case
*
* isp_peak_byte_per_pixel = <>;
* Max byte per pixel for the ISP ISO case
*
* isp_bw_margin_pct = <>;
* Isp bandwidth margin in percentage
*/
num_csi_lanes = <4>;
max_lane_speed = <1500000>;
min_bits_per_pixel = <10>;
vi_peak_byte_per_pixel = <2>;
vi_bw_margin_pct = <25>;
max_pixel_rate = <7500000>;
isp_peak_byte_per_pixel = <5>;
isp_bw_margin_pct = <25>;
/**
* The general guideline for naming badge_info contains 3 parts, and is as follows,
* The first part is the camera_board_id for the module; if the module is in a FFD
* platform, then use the platform name for this part.
* The second part contains the position of the module, ex. "rear" or "front".
* The third part contains the last 6 characters of a part number which is found
* in the module's specsheet from the vendor.
*/
modules {
module0 {
badge = "jakku_front_IMX477";
position = "front";
orientation = "1";
drivernode0 {
pcl_id = "v4l2_sensor";
devname = "imx477 9-001a";
proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@0/rbpcv3_imx477_a@1a";
};
};
};
};
};
If you look at Port Index this would be an imx477 camera connected to CSI-E. Do you know how to change the device tree to get that solution?
there’re port-index settings to assign the CSI/VI port settings.
as you can see… you should use port-index = <4>; to assign CSI with CSI-E, and also stream-4 for VI.
Setting port indexes was not quite enough.
But after a trying many combinations of the seeting we figured it also needs to be changed in the i2c driver settings of the camera chip. It does not really makse sense for us, but problem was solved that ways.
We’ve matched the ports in tegra-capture-vi and nvcsi@15a00000. But the final trick was setting tegra_sinterface = "serial_e"; in the camera mode at cam_i2cmux / i2c@0 / rbpcv3_imx477_a@1a / mode[x].
that make sense. port-index is definitions for the sensor port connection.
for camera streaming, it’s checking mode-specific settings in the device tree. that’s why you should also configure tegra_sinterface match to the CSI port where the sensor is connected.