Porting AR0135 + DS90UB964 driver to Jetson TX2 (Chansel Fault?)

AS the #17 the correct way is to get the data and clip it by tools or customize APP.

hello,
@azeps
I am using DS90UB964 and DS90UB913A-Q1 and AP0101.
The output of AP0101 is YUV422 8bit 1280x720.
The LIN_LEN of DS90UB964 is 0xf00, which is 38400. According to the CSI-2 specifiction, the Data Size of YUV422 8bit is 32bits for each two pixels.
38400 seems not to be a correct value for LIN_LEN.
The virtual channel I set is 0, and the Data Type is 0x1E.
I am facing these error:
[ 8241.170652] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 8242.174653] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 8243.178657] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!

Can you help me with this?

Hello zhaozoz,

to be honest I only have experience with imagers giving me a RAW data format.
Also a lot time happened after doing the project with this FPD-Link stuff. But I’ll try my best.

According to the datasheet of DS90UB913A there is a mode for 12 bit and 10 bit. I’ll hope that your imager is compatible to that. Especially with the pixel clock.

I’ve never heard about LIN_LEN. You mean LINE_LEN?
If I remember correctly LINE_LEN was the same as WORD_COUNT.
We had LINE_LEN = 1920 with 1280 pixels in width as RAW12 data which does make sense.

And yes I agree. With YUV422 you have 2 Bytes for every Pixel.
For 1280 pixels that would be line_len 2560.

3840 is far more than that. Maybe you had a misconfiguration of the deserializer and it’s not the fault of the TX2 in this case. 3840 would be 1280 * 3.
But even YUV422 with 10 bit would lead to 5 bytes per 2 pixels which would be 3200 bytes in total.

I wonder a few things:
Did you configure the imager correctly to have the data type you need?
Did you configure the deserializer correctly to have a mode that suits the type of signal coming from the imager?
Does the TX2 even support YUV pixel format for input? (I’ve never tried)

Also:
“PXL_SOF syncpt timeout” is something about the worst that could happen as it could mean anything.
Do you get something like this?

[  721.731011] tegra_channel_notify_status_callback 223
    [  721.731045] tegra-vi4 15700000.vi: Status:  7 channel:00 frame:0002
    [  721.737422] tegra-vi4 15700000.vi:          timestamp sof 1385830822 eof 1386390795 data 0x00000001
    [  721.746539] tegra-vi4 15700000.vi:          capture_id 0 stream  2 vchan  0
    [  721.753568] tegra_channel_handle_error 205
    [  722.712657] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11

Or only the timeout? I’ll highly suggest placing debug output inside the kernel.
There is vi4_fops.c in the kernel. Put some printouts inside tegra_channel_capture_setup(…) to check if virtual_ch and data_type are configured correctly.

PXL_SOF without vi: Status usually does mean that channel or data_type are wrong as the received seems to just discard the package.

Hello azeps,
Thank you for your time!

  1. It’s my mistake. That should be LINE_LEN, and it is same as WORD_COUNT.
  2. The engineer of the image sensor supplier said they have tested it with DS90UB913A and 914, and they successfully got the image(not with TX2). After checking the register of AP0101, I find the output format of image sensor is YUV422 8bit indeed.
    a). If I change the register to make the AP0101 output YUV422 10bit, the LINE_LEN of DS90UB964 is still 38400. I’m wondering which factor determines the LINE_LEN?
    b). The sensor supplier said they have tested it with DS90UB913A and 914 in RAW12 HF mode.
    If I use the RAW10 mode, the LINE_LEN of DS90UB964 is 4260.
    If I use the RAW12 LF mode, the LINE_LEN of DS90UB964 is 2558(It is not the same each time the board powered on. It may be 2559 or 2565).
  3. I used to make the TX2 receive YUV pixel format successfully.

There is no reply in the TI E2E Community for now.

Thank you again!

Hello zhaozoz,

  1. The 964 does officially offer RAW12 and RAW10 support. There is PORT_CONFIG2 register that offers RAW10_8BIT mode. Maybe you have to activate this one. But sadly I don’t know. This might be a topic for an FAE of Texas.
    2a) Sadly I believe the 913A doesn’t detect the input format. It has to be preconfigured using the deserializer. This can also be checked by reading the configuration registers.
    2b) Don’t use RAW12LF if the pixel clock requirements are not met. Every mode supports a different range of pixel clocks. Maybe this is the reason why RAW12 LF has strange values.
    And for RAW10 mode. Do you have configured the AP0101 to actually give you RAW10 data? Maybe you have configured YUV422 data but it’s interpreted as RAW10 data by the serializer.

I’ll also highly suggest to have printouts of status registers of the 964 every second or so.
Especially LINE_LEN and LINE_COUNT can give you lots of information. Also it should not change every second which can also be a bad indicator.

Hi,azeps, glad to find that you guys have solved this issue.
I am facing the similay situation here.
Here is another topic I have posted to discuss:

https://devtalk.nvidia.com/default/topic/1025863/jetson-tx2/how-to-grab-pre-configured-csi-video-stream-without-i2c

when I use the v4l2-ctl to grab the frame, I always got these errors:

[   31.849063] [OV5693]: ov5693_power_on.
[   31.868313] [OV5693]: ov5693_s_stream.
[   32.872152] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[   33.875972] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[   34.879943] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[   35.883944] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[   36.887942] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
[   36.894201] [OV5693]: ov5693_s_stream.
[   36.898218] [OV5693]: ov5693_update_ctrl_range.
[   36.902966] [OV5693]: wait for one frame.
[   36.909943] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) ERROR_STATUS2VI_VC0 = 0x0000000e
[   36.918735] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) ERROR_STATUS2VI_VC1 = 0x0000000a
[   36.927547] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) ERROR_STATUS2VI_VC2 = 0x00000008
[   36.936324] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) INTR_STATUS 0x000108ae
[   36.944238] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) ERR_INTR_STATUS 0x000108ae
[   36.969129] [OV5693]: ov5693_power_off.

and the notify info also got some errors as follows.

kworker/5:1-1870  [005] ...1   407.317788: rtcpu_vinotify_handle_msg: tstamp:13029782115 tag:CHANSEL_PXL_SOF channel:0x00 frame:24879 vi_tstamp:144879430 data:0x00000001
     kworker/5:1-1870  [005] ...1   407.317792: rtcpu_vinotify_handle_msg: tstamp:13029782386 tag:ATOMP_FS channel:0x00 frame:24879 vi_tstamp:144879436 data:0x00000000
     kworker/5:1-1870  [005] ...1   407.317795: rtcpu_vinotify_handle_msg: tstamp:13029782558 tag:CHANSEL_FAULT channel:0x00 frame:24879 vi_tstamp:144879827 data:0x00000100
     kworker/5:1-1870  [005] ...1   407.317798: rtcpu_vinotify_handle_msg: tstamp:13029783128 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:24879 vi_tstamp:144880834 data:0x08000000
     kworker/5:1-1870  [005] ...1   407.369804: rtcpu_vinotify_handle_msg: tstamp:13030823898 tag:CSIMUX_FRAME channel:0x8c frame:24879 vi_tstamp:145921551 data:0x00000402
     kworker/5:1-1870  [005] ...1   407.421817: rtos_queue_peek_from_isr_failed: tstamp:13032591611 queue:0x0b4a3c58
     kworker/5:1-1870  [005] ...1   407.525905: rtcpu_vinotify_handle_msg: tstamp:13035511298 tag:CSIMUX_FRAME channel:0x00 frame:24879 vi_tstamp:150608630 data:0x000004a2
     kworker/5:1-1870  [005] ...1   407.525915: rtcpu_vinotify_handle_msg: tstamp:13035511529 tag:CHANSEL_SHORT_FRAME channel:0x04 frame:24879 vi_tstamp:150608630 data:0x00000001
     kworker/5:1-1870  [005] ...1   407.525918: rtcpu_vinotify_handle_msg: tstamp:13035511735 tag:ATOMP_FE channel:0x00 frame:24879 vi_tstamp:150608634 data:0x00000000
  1. what did you mean by "The virtual channel needs to be 0 ".

here is the dtsi I used, is there anything wrong with it?

/ {
	host1x {
		vi@15700000 {
			num-channels = <1>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					status = "okay";
					e3326_vi_in0: endpoint {
						csi-port = <2>;
						bus-width = <4>;
						status = "okay";
						remote-endpoint = <&e3326_csi_out0>;
					};
				};
			};
		};

		nvcsi@150c0000 {
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;
			channel@0 {
				reg = <0>;
				status = "okay";
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						status = "okay";
						e3326_csi_in0: endpoint@0 {
							csi-port = <2>;
							bus-width = <4>;
							status = "okay";
							remote-endpoint = <&e3326_ov5693_out0>;
						};
					};
					port@1 {
						reg = <1>;
						status = "okay";
						e3326_csi_out0: endpoint@1 {
						    status = "okay";
							remote-endpoint = <&e3326_vi_in0>;
						};
					};
				};
			};
		};
	};

	i2c@3180000 {		
		status = "okay";
		ov5693_c@36 {
			compatible = "nvidia,ov5693";
			/* I2C device address */
			reg = <0x36>;
			status = "okay";

			/* V4L2 device node location */
			devnode = "video0";

			/* Physical dimensions of sensor */
			physical_w = "3.674";
			physical_h = "2.738";

			/* Define any required hw resources needed by driver */
			/* ie. clocks, io pins, power sources */
			avdd-reg = "vana";
			iovdd-reg = "vif";

			/* Sensor output flip settings */
			vertical-flip = "true";

			/**
			* A modeX node is required to support v4l2 driver
			* implementation with NVIDIA camera software stack
			*
			* mclk_khz = "";
			* Standard MIPI driving clock, typically 24MHz
			*
			* num_lanes = "";
			* Number of lane channels sensor is programmed to output
			*
			* tegra_sinterface = "";
			* The base tegra serial interface lanes are connected to
			*
			* discontinuous_clk = "";
			* The sensor is programmed to use a discontinuous clock on MIPI lanes
			*
			* dpcm_enable = "true";
			* The sensor is programmed to use a DPCM modes
			*
			* cil_settletime = "";
			* MIPI lane settle time value.
			* A "0" value attempts to autocalibrate based on mclk_multiplier
			*
			*
			*
			*
			* active_w = "";
			* Pixel active region width
			*
			* active_h = "";
			* Pixel active region height
			*
			* pixel_t = "";
			* The sensor readout pixel pattern
			*
			* readout_orientation = "0";
			* Based on camera module orientation.
			* Only change readout_orientation if you specifically
			* Program a different readout order for this mode
			*
			* line_length = "";
			* Pixel line length (width) for sensor mode.
			* This is used to calibrate features in our camera stack.
			*
			* mclk_multiplier = "";
			* Multiplier to MCLK to help time hardware capture sequence
			* TODO: Assign to PLL_Multiplier as well until fixed in core
			*
			* pix_clk_hz = "";
			* Sensor pixel clock used for calculations like exposure and framerate
			*
			*
			*
			*
			* inherent_gain = "";
			* Gain obtained inherently from mode (ie. pixel binning)
			*
			* min_gain_val = ""; (floor to 6 decimal places)
			* max_gain_val = ""; (floor to 6 decimal places)
			* Gain limits for mode
			*
			* min_exp_time = ""; (ceil to integer)
			* max_exp_time = ""; (ceil to integer)
			* Exposure Time limits for mode (us)
			*
			*
			* min_hdr_ratio = "";
			* max_hdr_ratio = "";
			* HDR Ratio limits for mode
			*
			* min_framerate = "";
			* max_framerate = "";
			* Framerate limits for mode (fps)
			*/
			mode0 { // OV5693_MODE_2592X1944
				mclk_khz = "24000";
				num_lanes = "4";
				tegra_sinterface = "serial_c";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
				cil_settletime = "0";

				active_w = "1920";
				active_h = "1080";
				pixel_t = "UYVY";
				readout_orientation = "90";
				line_length = "2200";
				inherent_gain = "1";
				mclk_multiplier = "6.67";
				pix_clk_hz = "160000000";

				min_gain_val = "1.0";
				max_gain_val = "16";
				min_hdr_ratio = "1";
				max_hdr_ratio = "64";
				min_framerate = "1.816577";
				max_framerate = "60";
				min_exp_time = "34";
				max_exp_time = "550385";
			};
			
			mode1 { //OV5693_MODE_2592X1458
				mclk_khz = "24000";
				num_lanes = "4";
				tegra_sinterface = "serial_c";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
				cil_settletime = "0";

				active_w = "1920";//"2592";
				active_h = "1080";//"1458";
				pixel_t = "UYVY";//"bayer_bggr";
				readout_orientation = "90";
				line_length = "2200";//"2688";
				inherent_gain = "1";
				mclk_multiplier = "6.67";
				pix_clk_hz = "160000000";

				min_gain_val = "1.0";
				max_gain_val = "16";
				min_hdr_ratio = "1";
				max_hdr_ratio = "64";
				min_framerate = "1.816577";
				max_framerate = "60";//"30";
				min_exp_time = "34";
				max_exp_time = "550385";
			};

			mode2 { //OV5693_MODE_1280X720
				mclk_khz = "24000";
				num_lanes = "4";
				tegra_sinterface = "serial_c";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
				cil_settletime = "0";

				active_w = "1280";
				active_h = "720";
				pixel_t = "UYVY";
				readout_orientation = "90";
				line_length = "1752";
				inherent_gain = "1";
				mclk_multiplier = "6.67";
				pix_clk_hz = "160000000";

				min_gain_val = "1.0";
				max_gain_val = "16";
				min_hdr_ratio = "1";
				max_hdr_ratio = "64";
				min_framerate = "2.787078";
				max_framerate = "120";
				min_exp_time = "22";
				max_exp_time = "358733";
			};
			
			ports {
				#address-cells = <1>;
				#size-cells = <0>;

				port@0 {
					reg = <0>;
					status = "okay";
					e3326_ov5693_out0: endpoint {
						csi-port = <2>;
						bus-width = <4>;
						status = "okay";
						remote-endpoint = <&e3326_csi_in0>;
					};
				};
			};
		};
	};

	e3326_lens_ov5693@P5V27C {
		min_focus_distance = "0.0";
		hyper_focal = "0.0";
		focal_length = "2.67";
		f_number = "2.0";
		aperture = "2.0";
	};

	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		/**
		* Physical settings to calculate max ISO BW
		*
		* num_csi_lanes = <>;
		* Total number of CSI lanes when all cameras are active
		*
		* max_lane_speed = <>;
		* Max lane speed in Kbit/s
		*
		* min_bits_per_pixel = <>;
		* Min bits per pixel
		*
		* vi_peak_byte_per_pixel = <>;
		* Max byte per pixel for the VI ISO case
		*
		* vi_bw_margin_pct = <>;
		* Vi bandwidth margin in percentage
		*
		* max_pixel_rate = <>;
		* Max pixel rate in Kpixel/s for the ISP ISO case
		*
		* isp_peak_byte_per_pixel = <>;
		* Max byte per pixel for the ISP ISO case
		*
		* isp_bw_margin_pct = <>;
		* Isp bandwidth margin in percentage
		*/
		num_csi_lanes = <12>;
		max_lane_speed = <1500000>;
		min_bits_per_pixel = <10>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		max_pixel_rate = <160000>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		/**
		* The general guideline for naming badge_info contains 3 parts, and is as follows,
		* The first part is the camera_board_id for the module; if the module is in a FFD
		* platform, then use the platform name for this part.
		* The second part contains the position of the module, ex. “rear” or “front”.
		* The third part contains the last 6 characters of a part number which is found
		* in the module's specsheet from the vender.
		*/
		modules {
			module0 {
				badge = "e3326_front_P5V27C";
				position = "rear";
				orientation = "1";
				status = "okay";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "ov5693 2-0036";		
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/ov5693_c@36";
				};
				drivernode1 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_lens";
					proc-device-tree = "/proc/device-tree/e3326_lens_ov5693@P5V27C/";
				};
			};
		};
	};
};

And in the camera_common.c, I added thesee codes to use UYUV format:

{
		MEDIA_BUS_FMT_UYVY8_2X8,
		V4L2_COLORSPACE_SRGB,
		V4L2_PIX_FMT_UYVY,
	},
	{
		MEDIA_BUS_FMT_UYVY8_1X16,
		V4L2_COLORSPACE_SRGB,
		V4L2_PIX_FMT_UYVY,
	},

And in sensor_common.c, I added this:

else if (strncmp(pixel_t, "UYVY", size) == 0)
		*format = V4L2_PIX_FMT_UYVY;

In ov5693.c, I used this

#define OV5693_DEFAULT_MODE	OV5693_MODE_1920X1080
#define OV5693_DEFAULT_HDR_MODE	OV5693_MODE_1920X1080_HDR
#define OV5693_DEFAULT_WIDTH	1920 //2592
#define OV5693_DEFAULT_HEIGHT	1080 //1944
#define OV5693_DEFAULT_DATAFMT	MEDIA_BUS_FMT_UYVY8_2X8//MEDIA_BUS_FMT_SRGGB10_1X10
#define OV5693_DEFAULT_CLK_FREQ	24000000
  1. the video stream was 1920X1080, YUV422-8bits, so what is the LINE_LEN should be used?
1 Like

Hi, azeps. I printed some log in tegra_channel_capture_setup() function. And it shows that the virtual_channel is 1. But I have no idea about where to modify this. could you please tell me how?

Here is the whole info:

height=1080
width=1920
format=203
data_type=30
csi_port=2
stream=4
virtual_ch=1

and where can I find data_type definition?

thanks a lot.

Already have below topic for DaLT’s problem. Close this topic.
https://devtalk.nvidia.com/default/topic/1025863