CSI camera driver porting to Jetpack 4.4.1 (TX2i)

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,
We have a working custom driver based on ov5693 template, for a (TX2 + JP 3.1) combination. Our video path includes a pre-configured video stream coming from an FPGA, for which the i2c is disabled in the ov5693 driver. Now we are looking to port the driver to newer Jetpack releases.

First we ported the driver to Jetpack 3.3.3 on a TX2i. Following the similar modifications from Jetpack 3.1 on the original source file of Jetpack 3.3.3, and making appropriate device tree changes, we were able to get the video ingested from the FPGA o (TX2i + JP3.3.3) combination.

Lastly, we made similar changes to the original source of Jetpack 4.4.1, to get the video ingested from the FPGA on TX2i. But in this configuration, we are not able to get the video data on the lines.

Sharing the debug information below:
dmesg_init.txt (1.4 KB)
dmesg_v4l2-ctl.txt (11.8 KB)

Kernel trace obtained by following the debugging steps from Jetson_TX2_Camera_Bringup, along with boosting the clocks.
kernel_trace.txt (131.6 KB)

Please note that the same hardware is able to give video data if TX2+JP3.1 OR TX2i+JP3.3.3 is used on it.

Thanks

3

The trace log shows didn’t get any validated data from MIPI bus.

Jetson Linux Driver Package (L4T) release 31 upgraded the Linux kernel from version 4.4 to 4.9. Sensor drivers developed on L4T Release 28 do not work with the current release. If you have such a driver, you must modify it to work with the release 31 driver interface. Detail can be found here: https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/camera_sensor_prog.html#wwpID0E0FB0HA

1 Like
4

Hi sluo,
For each of the Jetpack versions, we started with the respective stock L4T sources.
So initially, for Jetpack 3.3.3, we started with original stock v28.4 L4T source, and modified ov5693.c and ov5693_mode_tables.h files, along with addition of YUYV support in sensor_common.c and camera_common.h files. This way we successfully got the video ingested in the Jetpack 3.3.3 based TX2i.

In the similar manner, for Jetpack 4.4.1 we started with original stock v32.4.4 L4T sources, and removed the i2c part from ov5693.c and tried the video ingestion. But in this case, we are not even getting messages in the kernel trace.

Please note that, same carrier board, and video electronics is used for all the tests. The only thing changing is the TX2i SOM with Jetpack versions.

Thanks

5

Are you able to verify on TX2 to clarify the root cause?

1 Like
6

Due to our product constraints, we needed to use TX2i for this migration, and hence we haven’t had a chance to check on TX2.

Also, we thought that as on Jetpack 3.3.3, the video ingestion worked fine even with TX2i, so the TX2 vs TX2i dynamics might not be at play for our current deadlock.

In any case, we’ll check TX2 + JP4.4.1 combination as well, as suggested, but our end goal is to use TX2i.
Thanks

7

Will the following solution work for your case?

If not, please provide more information including device tree node, v4l2-compliance test output, v4l2-ctl test output and checklist based on the following link. Thanks

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/camera_sensor_prog.html#wwpID0E0YE0HA

1 Like
8

Based on the solution of the above-mentioned post, the discontinuous_clk parameter was the root cause. For our case, we have verified the discontinuous_clk parameter, and it is in accordance with the video input.

Please find the other information below:

  • Device tree node:
ov5693_b@10 {
    compatible = "nvidia,ov5693";
    reg = <0x10>;
    devnode = "video0";
    status = "okay";
    physical_w = "3.674";
    physical_h = "2.738";
    vertical-flip = "true";
    vana-supply = <0x29>;
    vif-supply = <0x2b>;
    avdd-reg = "vana";
    iovdd-reg = "vif";
    reset-gpios = <0x1b 0x89 0x0>;
    pwdn-gpios = <0x1b 0x6a 0x0>;
    clocks = <0x10 0x59 0x10 0x10d>;
    clock-names = "extperiph1", "pllp_grtba";
    mclk = "extperiph1";
    clock-frequency = <0x16e3600>;

    mode0 {
        tegra_sinterface = "serial_b";
        pix_clk_hz = "80000000";
        active_h = "720";
        active_w = "2560";
        line_length = "2560";
        pixel_t = "yuv_yuyv16";
        //mode_type = "yuv";
        //pixel_phase = "yuyv";
        //csi_pixel_bit_depth = "16";
        //dynamic_pixel_bit_depth = "16";
        phy_mode = "DPHY";
        discontinuous_clk = "yes";
        num_lanes = "2";
        default_framerate = "30000000";
        min_framerate = "1816577";
        max_framerate = "30000000";
        framerate_factor = "1000000";
        step_framerate = [31 00];
        cil_settletime = "0";
        default_gain = "10";
        min_gain_val = "10";
        max_gain_val = "160";
        gain_factor = "10";
        step_gain_val = [31 00];
        min_hdr_ratio = [31 00];
        max_hdr_ratio = [31 00];
        default_exp_time = "33334";
        min_exp_time = "34";
        max_exp_time = "550385";
        exposure_factor = "1000000";
        step_exp_time = [31 00];
        mclk_khz = "24000";
        mclk_multiplier = "6.67";
        readout_orientation = "90";
        dpcm_enable = "false";
        inherent_gain = [31 00];
        embedded_metadata_height = [30 00];
    };

    ports {
        #address-cells = <0x1>;
        #size-cells = <0x0>;
        status = "okay";

        port@0 {
            reg = <0x0>;
            status = "okay";

            ov5693_b_out: endpoint {
                status = "okay";
                port-index = <0x1>;
                bus-width = <0x2>;
                remote-endpoint = <0x2d>;
                linux,phandle = <0x1e1>;
                phandle = <0x1e1>;
            };
        };
    };
};
  • v4l2-compliance test:
    v4l2-compliance.txt (2.6 KB)
  • v4l2-ctl test: Did not print anything on terminal
  • Checklist from Debugging Tips section of L4T documentation
    • Driver name matches with our intended driver
    • Device names match in the device tree
    • Device tree values are correct
    • Not seeing any obvious errors in function runs, in kernel logs, apart from the PXL_SOF syncpt error
    • The I2C related points do not apply to our case
    • Mode specific settings are correct: (these parameters work for our 3.1 and 3.3.3 Jetpack’s driver
      • num_lanes = 2
      • tegara_sinterface = serial_b
      • discontinous_clk = yes
      • mclk_multiplier > (80000000/24000000) 6.67 (for our case, pix_clk_hz = 80000000)

Thanks,
Vedant

9

Could you probe the MIPI signal to confirm it.

1 Like
10

Also make sure the sensor output is continuous clock mode or discontinuous clocks to set the correct.(discontinous_clk )

1 Like
11

We have sufficient confidence in the MIPI signal validity, as the same signal works with the other combinations (namely, TX2+JP3.1 and TX2i+JP3.3.3).
Also, the clock is discontinuous, which is what is set in the device tree as well.

Thanks

12

Few information only the CILA_ERR_INTR_STATUS 0x00000089 tell it could be the settle time or sensor set as continuous clocks but dts set discontinuous_clk as yes.

1 Like
13

Although the clock discontinuous_clk setting worked properly in the previous jetpacks, will confirm this and post the observations in a couple of days.

EDIT:
We reconfirmed the discontiuous clk parameter. Even upon trying both “yes” and “no” as the value, the same error is seen. Also, cil_settletime is set as “0” for our case.

Thanks

14

Some Additional information:
We interfacing one more video signal here, which also works with the earlier jetpack versions. The signal parameters:

  • Width: 640
  • Height: 480
  • Lanes: 1
  • Pix Clk: 54000000 Hz
  • Color format: YUYV
  • Continuous clock

For this signal too, we are facing similar errors. Dmesg and kernel trace logs attached below:
dmesg.txt (5.4 KB)
kernel.trace (76.3 KB)

Device tree node:

ov5693_c@36 {
    compatible = "nvidia,ov5693";
    clocks = <0x10 0x59 0x10 0x10d>;
    physical_w = "3.674";
    iovdd-reg = "vif";
    devnode = "video1";
    avdd-reg = "vana";
    clock-names = "extperiph1", "pllp_grtba";
    mclk = "extperiph1";
    status = "okay";
    reset-gpios = <0x1b 0x8d 0x0>;
    phandle = <0x123>;
    reg = <0x36>;
    pwdn-gpios = <0x1b 0x88 0x0>;
    clock-frequency = <0x16e3600>;
    vana-supply = <0x29>;
    vertical-flip = "true";
    linux,phandle = <0x123>;
    vif-supply = <0x2b>;
    physical_h = "2.738";

    mode0 {
        default_framerate = "30000000";
        step_exp_time = [31 00];
        line_length = "1280";
        active_w = "1280";
        embedded_metadata_height = [30 00];
        pixel_phase = "yuyv";
        pix_clk_hz = "54000000";
        dpcm_enable = "false";
        mode_type = "yuv";
        num_lanes = [31 00];
        inherent_gain = [31 00];
        max_gain_val = "160";
        min_hdr_ratio = [31 00];
        discontinuous_clk = "no";
        readout_orientation = "90";
        min_gain_val = "10";
        max_hdr_ratio = [31 00];
        tegra_sinterface = "serial_c";
        step_gain_val = [31 00];
        default_gain = "10";
        csi_pixel_bit_depth = "16";
        min_framerate = "1816577";
        default_exp_time = "33334";
        max_framerate = "30000000";
        exposure_factor = "1000000";
        step_framerate = [31 00];
        phy_mode = "DPHY";
        mclk_khz = "24000";
        cil_settletime = [30 00];
        gain_factor = "10";
        framerate_factor = "1000000";
        active_h = "480";
        max_exp_time = "550385";
        mclk_multiplier = "6.67";
        min_exp_time = "34";
        dynamic_pixel_bit_depth = "16";
    };

    ports {
        #address-cells = <0x1>;
        #size-cells = <0x0>;
        status = "okay";

        port@0 {
            reg = <0x0>;
            status = "okay";

            endpoint {
                port-index = <0x2>;
                remote-endpoint = <0x36>;
                bus-width = <0x1>;
                phandle = <0x125>;
                linux,phandle = <0x125>;
                status = "okay";
            };
        };
    };
};

One thing to note is that the “CIL_” prefixed errors are not seen in this one. Only the PXL_SOF sycpt errors are seen. The kernel trace seems exactly the same as the earlier ones.

Thanks

1 Like
15

The CIL_ not show could be the recovery code try to recovery the capture but still failed.
Remove the recovery code may show the CIL status.

1 Like
17

Hi,
We we were able to get the MIPI video ingested when we used Jetpack 4.5 instead of 4.4.1.
The interesting thing to note is that the same device tree parameters and the modifications to the ov5693 driver worked for Jetpack 4.5, that we were trying for 4.4.1

Thanks