Porting GSML OV10635 Camera driver from JetPack 4.2 to JetPack 4.2.2

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,
We have developed OV10635 camera driver for TX2, the driver works ok on L4T 32.1 (JetPack 4.2)

Now we have ported the driver to L4T 32.2.1 (JetPack 4.2.2), carefully following the recommendations in the documentation. The driver probes ok, mounts the device to /dev/video0 but does not stream:

We tested with:

a) v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1
b) DISPLAY=:0 gst-launch-1.0 v4l2src device=/dev/video0 ! ‘video/x raw,format=UYVY,width=1280,height=800’ ! xvimagesink

VIDIOC_DQBUF: failed: Input/output error

  1. Trying to debug with dmesg we got the following output:
nvidia@nvidia-desktop:~$ dmesg
[  149.446033] ov10635 2-0031: Power on
[  149.453511] ov10635 2-0031: Power off
[  149.460931] ov10635 2-0031: Power on
[  149.468134] ov10635 2-0031: Setting format
[  149.468151] ov10635 2-0031: Setting format
[  149.476027] ov10635 2-0031: Setting stream to 1
[  149.679097] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  149.685509] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  149.899283] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  149.905739] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.119276] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.125755] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.339276] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.345745] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.559275] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.565753] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.779328] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.785800] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.796686] ov10635 2-0031: Setting stream to 0
[  150.826088] ov10635 2-0031: Power off
  1. Trying to debug with kernel trace, the streaming seems to start with the following errors, which means it does not receive buffers:
nvidia@nvidia-desktop:~$ sudo cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 555/555   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/5:2-4762  [005] ....    59.242119: rtos_queue_peek_from_isr_failed: tstamp:2222730175 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    59.354099: rtos_queue_peek_from_isr_failed: tstamp:2227730183 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    59.522136: rtos_queue_peek_from_isr_failed: tstamp:2232730190 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    59.690097: rtos_queue_peek_from_isr_failed: tstamp:2237730196 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    59.858094: rtos_queue_peek_from_isr_failed: tstamp:2242730202 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    60.026101: rtos_queue_peek_from_isr_failed: tstamp:2247730197 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    60.194092: rtos_queue_peek_from_isr_failed: tstamp:2252730216 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    60.362103: rtos_queue_peek_from_isr_failed: tstamp:2257730212 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    60.474133: rtos_queue_peek_from_isr_failed: tstamp:2262730231 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    60.642124: rtos_queue_peek_from_isr_failed: tstamp:2267730228 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    60.810098: rtos_queue_peek_from_isr_failed: tstamp:2272730243 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    60.978098: rtos_queue_peek_from_isr_failed: tstamp:2277730250 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    61.146131: rtos_queue_peek_from_isr_failed: tstamp:2282730247 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    61.314094: rtos_queue_peek_from_isr_failed: tstamp:2287730253 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    61.482182: rtos_queue_peek_from_isr_failed: tstamp:2292730259 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    61.594107: rtos_queue_peek_from_isr_failed: tstamp:2297730267 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    61.762093: rtos_queue_peek_from_isr_failed: tstamp:2302730284 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    61.930131: rtos_queue_peek_from_isr_failed: tstamp:2307730279 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    62.098109: rtos_queue_peek_from_isr_failed: tstamp:2312730323 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    62.266116: rtos_queue_peek_from_isr_failed: tstamp:2317730294 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    62.434098: rtos_queue_peek_from_isr_failed: tstamp:2322730308 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    62.602111: rtos_queue_peek_from_isr_failed: tstamp:2327730306 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    62.714118: rtos_queue_peek_from_isr_failed: tstamp:2332730324 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    62.882097: rtos_queue_peek_from_isr_failed: tstamp:2337730331 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    63.050098: rtos_queue_peek_from_isr_failed: tstamp:2342730328 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    63.218102: rtos_queue_peek_from_isr_failed: tstamp:2347730334 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    63.386095: rtos_queue_peek_from_isr_failed: tstamp:2352730340 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    63.554096: rtos_queue_peek_from_isr_failed: tstamp:2357730346 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    63.722094: rtos_queue_peek_from_isr_failed: tstamp:2362730366 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    63.834102: rtos_queue_peek_from_isr_failed: tstamp:2367730360 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    64.002097: rtos_queue_peek_from_isr_failed: tstamp:2372730369 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    64.114110: rtos_queue_peek_from_isr_failed: tstamp:2375417619 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    68.762133: rtos_queue_peek_from_isr_failed: tstamp:2520262081 queue:0x0b4b4500
     kworker/5:2-4762  [005] ....    68.762138: rtcpu_start: tstamp:2520263181
     kworker/5:2-4762  [005] ....    68.762140: rtos_queue_send_from_isr_failed: tstamp:2520271508 queue:0x0b4a7258
     kworker/5:2-4762  [005] ....    68.762141: rtos_queue_send_from_isr_failed: tstamp:2520271614 queue:0x0b4aad68
     kworker/5:2-4762  [005] ....    68.762141: rtos_queue_send_from_isr_failed: tstamp:2520271722 queue:0x0b4ac998
     kworker/5:2-4762  [005] ....    68.762142: rtos_queue_send_from_isr_failed: tstamp:2520271828 queue:0x0b4ae518
     kworker/5:2-4762  [005] ....    68.762142: rtos_queue_send_from_isr_failed: tstamp:2520271933 queue:0x0b4af2d8
     kworker/5:2-4762  [005] ....    68.762143: rtos_queue_send_from_isr_failed: tstamp:2520272038 queue:0x0b4b0098
     kworker/5:2-4762  [005] ....    68.762144: rtos_queue_send_from_isr_failed: tstamp:2520272143 queue:0x0b4b0e58
     kworker/5:2-4762  [005] ....    68.762144: rtos_queue_send_from_isr_failed: tstamp:2520272248 queue:0x0b4b1c18
     kworker/5:2-4762  [005] ....    68.762146: rtos_queue_send_failed: tstamp:2520272713 queue:0x0b4a7258
     kworker/5:2-4762  [005] ....    68.762146: rtos_queue_send_from_isr_failed: tstamp:2520277763 queue:0x0b4a7258
     kworker/5:2-4762  [005] ....    68.762147: rtos_queue_send_from_isr_failed: tstamp:2520277869 queue:0x0b4aad68
     kworker/5:2-4762  [005] ....    68.762148: rtos_queue_send_from_isr_failed: tstamp:2520277975 queue:0x0b4ac998
     kworker/5:2-4762  [005] ....    68.762148: rtos_queue_send_from_isr_failed: tstamp:2520278081 queue:0x0b4ae518
     kworker/5:2-4762  [005] ....    68.762149: rtos_queue_send_from_isr_failed: tstamp:2520278186 queue:0x0b4af2d8
     kworker/5:2-4762  [005] ....    68.762150: rtos_queue_send_from_isr_failed: tstamp:2520278291 queue:0x0b4b0098
     kworker/5:2-4762  [005] ....    68.762150: rtos_queue_send_from_isr_failed: tstamp:2520278405 queue:0x0b4b0e58
     kworker/5:2-4762  [005] ....    68.762151: rtos_queue_send_from_isr_failed: tstamp:2520278511 queue:0x0b4b1c18

Did something change on the last release that we must notice?

Our device tree for the camera has the following configuration for the mode, are we missing something important?:

mode0 {
    mclk_khz = "24000";
    num_lanes = "4";
    tegra_sinterface = "serial_a";
    vc_id = "0";
    discontinuous_clk = "yes";
    dpcm_enable = "false";
    cil_settletime = "0";
    phy_mode = "DPHY";

    active_w = "1280";
    active_h = "800";
    pixel_t = "uyvy";
    readout_orientation = "0";
    line_length = "2560";
    inherent_gain = "1";
    mclk_multiplier = "25";
    pix_clk_hz = "170000000";
    serdes_pix_clk_hz = "833333333";

    min_gain_val = "1.0";
    max_gain_val = "16.0";
    min_hdr_ratio = "1";
    max_hdr_ratio = "64";
    min_framerate = "1.462526";
    max_framerate = "30";
    min_exp_time = "13";
    max_exp_time = "683709";
    embedded_metadata_height = "0";
};
2

hello fabian.solano,

couple of suggestion as below,

  1. since we had update the implementation to represent the float values in fixed point, (even I believe float values should still supported), it’s worth a try to update your device tree settings accordingly. please access Sensor Software Driver Programming Guide, check framerate_factor property as an example.

  2. we had change integrate for adding function to set bytes-per-line, could you please check VI kernel drivers, please dump pix->bytesperline to check it’s same as your expectation.
    $l4t-r32.2/public_sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/channel.c

3

Thanks JerryChang,

I tried with your two suggestions but still no luck. Did something changed from 4.2 to 4.2.1/4.2.2 related to the CSI channels? Doing some tests I found this:

The device tree (the one that works) on 4.2, refers to the VI and the CSI as follows:

vi_base: vi@15700000 {
	num-channels = <1>;
	    ports {
		#address-cells = <1>;
		#size-cells = <0>;
		vi_port0_0: port@0 { /* CSI A - VC 0 */
			reg = <0>;
			status = "okay";
			vi_in0_0: endpoint {
				status = "okay";
				port-index = <0>;
				bus-width = <4>;
				remote-endpoint = <&csi_out0_0>;
			};
		};
	};
};

Please note that port-index is 0. Using the same configuration on 4.2.2 leads to the previously seen error:

nvidia@nvidia-desktop:~$ dmesg
[  149.446033] ov10635 2-0031: Power on
[  149.453511] ov10635 2-0031: Power off
[  149.460931] ov10635 2-0031: Power on
[  149.468134] ov10635 2-0031: Setting format
[  149.468151] ov10635 2-0031: Setting format
[  149.476027] ov10635 2-0031: Setting stream to 1
[  149.679097] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  149.685509] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  149.899283] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  149.905739] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.119276] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.125755] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.339276] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.345745] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.559275] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.565753] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.779328] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  150.785800] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  150.796686] ov10635 2-0031: Setting stream to 0
[  150.826088] ov10635 2-0031: Power off

However, if I change the port-index as follows:

vi_base: vi@15700000 {
	num-channels = <1>;
	    ports {
		#address-cells = <1>;
		#size-cells = <0>;
		vi_port0_0: port@0 { /* CSI A - VC 0 */
			reg = <0>;
			status = "okay";
			vi_in0_0: endpoint {
				status = "okay";
				port-index = <1>;
				bus-width = <4>;
				remote-endpoint = <&csi_out0_0>;
			};
		};
	};
};

The error from dmesg changes to:

nvidia@nvidia-desktop:~$ dmesg
[  181.386709] ov10635 2-0031: Power on
[  181.393545] ov10635 2-0031: Power off
[  181.401449] ov10635 2-0031: Power on
[  181.407817] ov10635 2-0031: Setting format
[  181.407826] ov10635 2-0031: Setting format
[  181.414151] ov10635 2-0031: Setting stream to 1
[  181.614592] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  181.621084] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  181.630910] nvcsi 150c0000.nvcsi: csi4_cil_check_status (1) CILA_INTR_STATUS 0x00000089
[  181.639038] nvcsi 150c0000.nvcsi: csi4_cil_check_status (1) CILA_ERR_INTR_STATUS 0x00000089
[  181.850608] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  181.857080] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  181.868372] nvcsi 150c0000.nvcsi: csi4_cil_check_status (1) CILA_INTR_STATUS 0x00000089
[  181.876423] nvcsi 150c0000.nvcsi: csi4_cil_check_status (1) CILA_ERR_INTR_STATUS 0x00000089
[  182.086618] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  182.093002] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  182.103010] nvcsi 150c0000.nvcsi: csi4_cil_check_status (1) CILA_INTR_STATUS 0x00000089
[  182.111087] nvcsi 150c0000.nvcsi: csi4_cil_check_status (1) CILA_ERR_INTR_STATUS 0x00000089
[  182.322661] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  182.329041] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  183.585231] ov10635 2-0031: Power off

Which makes me think I could be something related to the CSI.

Thanks for your help.

4

hello fabian.solano,

may I know how many hardware device is connected, is it a single camera sensor configuration?
also, could you please share your position property settings in device tree,
thanks

5

Hi JerryChang,

1- Yes, currently it is a single camera sensor configuration, at a time. But later we will use multiple cameras with VCID support.

2- Here is the tegra-camera-platform node of the device tree which includes the position property:

tcp: tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		num_csi_lanes = <4>;
		max_lane_speed = <1500000>;
		min_bits_per_pixel = <10>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		max_pixel_rate = <160000>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;
		modules {
			cam_module0: module0 {
				status = "okay";
				badge = "e3326_front_ov10635";
				position = "front";
				orientation = "1";
				cam_module0_drivernode0: drivernode0 {
					status = "okay";
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "ov10635 2-0030";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/ov10635_c@30";
				};
				cam_module0_drivernode1: drivernode1 {
					status = "disabled";
					pcl_id = "v4l2_focuser_stub";
					proc-device-tree = "/proc/device-tree/e3326_focuser_ov10635@P5V27C/";
				};
			};
			cam_module1: module1 {
				status = "okay";
				badge = "e3326_rear_ov10635";
				position = "rear";
				orientation = "1";
				cam_module1_drivernode0: drivernode0 {
					status = "okay";
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "ov10635 7-0030";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@c250000/ov10635_c@30";
				};
				cam_module1_drivernode1: drivernode1 {
					status = "disabled";
					pcl_id = "v4l2_lens";
				};
			};
		};
	};
6

hello fabian.solano,

looks you had two OV10635 camera module definition in the tegra-camera-platform field.

tegra-camera-platform {
		modules {
			cam_module0: module0 {...}

			cam_module1: module1 {...}

tegra-camera-platform definition declared the camera module which connected to the tegra device.
since there’s only single camera of your actual environment setup,
suggest you have a try to remove 2nd camera node and also specify the position as rear for 1st camera node.
thanks

7

Hello,

Finally we found the error. After a large comparison between the kernel source code of version 4.2 and 4.2.2, we noticed that the discontinuous_clk property on the device tree is being used. Previous versions bypassed the value from the device tree.

If someone is facing the same/similar error, please check the value of this property in the device tree. If it does not match the camera/driver configuration it will not receive buffers at all, and nothing can be debugged with the trace.

In our case we changed the mode settings from:

mode0 {
    .
    .
    .
    tegra_sinterface = "serial_a";
    vc_id = "0";
    <b>discontinuous_clk = "yes";</b>
    dpcm_enable = "false";
    cil_settletime = "0";
    phy_mode = "DPHY";

    active_w = "1280";
    active_h = "800";
    .
    .
    .
}

to:

mode0 {
    .
    .
    .
    tegra_sinterface = "serial_a";
    vc_id = "0";
    <b>discontinuous_clk = "no";</b>
    dpcm_enable = "false";
    cil_settletime = "0";
    phy_mode = "DPHY";

    active_w = "1280";
    active_h = "800";
    .
    .
    .
}

Notice that the only relevant change is discontinuous_clk. Thanks for your help.

Best regards,

Fabian

8

Hi RidgeRun,

We are also plan to use OV10635 camera on our TX2 project.
I tried to find one simlar driver for my reference on exist Jetpack 3.3 source tree.
But nothig to get on that.
Would you please share me some useful information on how to port the OV10635 driver?

Thanks,
Martin

9

Martin,

We have a driver for ov10640 which is likely very similar to the ov10640 model. I’m not familiar with the differences in the image sensors. You might find our source code helpful. You can find it on GitHub at the link below. In the next week we will be publishing new source code as we recently updated our board support packages to work with JetPack 4.2.2 / R32.2.1.

ov10640 driver code:
[url]https://github.com/D3Engineering/d3-jetson-bsp/tree/d3/1.3.3/kernel/d3/drivers/d3/ov10640[/url]

device tree template:
[url]https://github.com/D3Engineering/d3-jetson-bsp/blob/d3/1.3.3/hardware/d3/d3-cam-ov10640-template.dtsi[/url]

Regards,
Greg

10

Hi Martin,

RidgeRun provides software development kits, board bring-up services, device drivers (such as the OV10635 camera driver), media server development and application programming, among others.

We have the OV10635 working for Jetpack 3.3, 4.2 and 4.2.2. You can send an email to support@ridgerun.com with your question.

If you want to port the driver by yourself, you can review the Nvidia Porting guide [url]Tegra Linux Driver