Capture Image with meta data from IMX566 on TX2 NX

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,
I am currently developing a driver for the IMX566. I have a working I2C communication to the sensor, configured the sensor according to the suggestions from Sony, get Signals on the CSI2-Lanes, and see in the trace log, that I am receiving packets over CSI2.

     kworker/4:0-36    [004] ....    90.125066: rtos_queue_send_from_isr_failed: tstamp:3056869333 queue:0x0b4b04d8
     kworker/4:0-36    [004] ....    90.125067: rtos_queue_send_from_isr_failed: tstamp:3056869599 queue:0x0b4b1298
     kworker/4:0-36    [004] ....    90.125068: rtos_queue_send_from_isr_failed: tstamp:3056869870 queue:0x0b4b2058
     kworker/4:0-36    [004] ....    90.125069: rtos_queue_send_failed: tstamp:3056873087 queue:0x0b4a7698
     kworker/4:0-36    [004] ....    90.181122: rtcpu_vinotify_event: tstamp:3058239096 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:3058238592 data:0x00000001
 vi-output, imx5-7922  [003] ....    90.211275: tegra_channel_capture_frame: sof:90.102124224
     kworker/4:0-36    [004] ....    90.237041: rtcpu_vinotify_event: tstamp:3059618819 tag:ATOMP_FS channel:0x00 frame:112 vi_tstamp:3059618320 data:0x00000000
     kworker/4:0-36    [004] ....    90.237048: rtcpu_vinotify_event: tstamp:3059650524 tag:CHANSEL_PXL_SOF channel:0x00 frame:112 vi_tstamp:3059650052 data:0x00000001
     kworker/4:0-36    [004] ....    90.237051: rtcpu_vinotify_event: tstamp:3059657220 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:112 vi_tstamp:3059656736 data:0x08000000
     kworker/4:0-36    [004] ....    90.293126: rtcpu_vinotify_event: tstamp:3060980334 tag:CHANSEL_PXL_EOF channel:0x00 frame:112 vi_tstamp:3060979346 data:0x0b1f0002
     kworker/4:0-36    [004] ....    90.293132: rtcpu_vinotify_event: tstamp:3060980667 tag:CHANSEL_SHORT_FRAME channel:0x01 frame:112 vi_tstamp:3060979360 data:0x00001000
     kworker/4:0-36    [004] ....    90.293134: rtcpu_vinotify_event: tstamp:3060981038 tag:ATOMP_FE channel:0x00 frame:112 vi_tstamp:3060979362 data:0x00000000
     kworker/4:0-36    [004] ....    90.293139: rtos_queue_peek_from_isr_failed: tstamp:3061159211 queue:0x0b4b4940
 vi-output, imx5-7922  [003] ....    90.430240: tegra_channel_capture_setup: vnc_id 0 W 2856 H 2848 fmt 20
 vi-output, imx5-7922  [003] ....    90.430319: tegra_channel_capture_frame: sof:90.102124224
     kworker/4:0-36    [004] ....    90.461140: rtos_queue_peek_from_isr_failed: tstamp:3066159147 queue:0x0b4b4940
     kworker/4:0-36    [004] ....    90.461152: rtos_queue_send_from_isr_failed: tstamp:3066468210 queue:0x0b4a7698
     kworker/4:0-36    [004] ....    90.461155: rtos_queue_send_from_isr_failed: tstamp:3066468498 queue:0x0b4ab1a8
     kworker/4:0-36    [004] ....    90.461159: rtos_queue_send_from_isr_failed: tstamp:3066468771 queue:0x0b4acdd8
     kworker/4:0-36    [004] ....    90.461162: rtos_queue_send_from_isr_failed: tstamp:3066469049 queue:0x0b4af718

If I am interpreting the messages correctly, I get a SOF at line 0x0000(0), get some data, get an EOF at line 0x0b1f(2847). Until here, I think all went well, I expect 2848 lines with data.

Then I get a CHANSEL_SHORT_FRAME with data: 0x00001000
If I interpret the data correctly, I get EMBED_SPURIOUS.
“Jetson_TX2_Parker_TRM_DP07821001p.pdf” states:

EMBED_SPURIOUS: Embedded bytes found between line start and line end

I expect 4 lines of meta data (configured in the device tree) with the same line length as image data (CSI2 Header of first line: 0x12, CSI2 Header of 2nd to 4th line: 0x13) which are transmitted before the data, but after the SOF.

Can you help me finding the wrong configuration?

If you need more information to help me, I will try to provide it as soon as possible.

3

hello bernhard.muellner,

do you have pixel readout diagram for reference? where’s the location of the embedded metadata, FYI, we only support embedded metadata located before the frame (i.e. pixel active region).

4

Thank you for your fast response.
Here is the readout diagram:
image

5

hello bernhard.muellner,

thanks for sharing readout diagram, this should be supported.
according to tracing logs, there’s CHANSEL_SHORT_FRAME failure. this usually caused by the mismatch of active_h and your sensor signaling. could you also share your device tree settings for reference?

BTW, is it possible to have a quick try to disable outputting metadata for issue narrow down?

6

Thank you for your answer.
I already tried to brute force the problem, by trying to sweep active_h in the device tree and imx566_frmfmt in the imx566_mode_tbls.h device driver file.

From active_h 2840 to 2844, I got the following sequence in the trace data:
CHANSEL_PXL_SOF
CHANSEL_LOAD_FRAMED
CHANSEL_PXL_EOF (last transmitted line as data)
CHANSEL_FAULT (next expected line as data and PIX_RUNAWAY)

From active_h 2845 to 2847, I got the following sequence in the trace data:
CHANSEL_PXL_SOF
CHANSEL_LOAD_FRAMED
CHANSEL_PXL_EOF (last transmitted line as data)
CHANSEL_FAULT (next expected line as data and PIX_RUNAWAY)
CHANSEL_SHORT_FRAME
It is not a typo here, I get both PIX_RUNAWAY and SHORT_FRAME at these settings

The output from 2848 was posted before

From 2849 upwards I got the following sequence
CHANSEL_PXL_SOF
CHANSEL_LOAD_FRAMED
CHANSEL_SHORT_FRAME (data: 0x00001001)

Apart from this, I check the sensor configuration, where I expect an output from the following lines (hex number is the CSI2 Header):
0x00 Frame Start 1x
0x12 Embedded Data Line 1x
0x13 Emeedded Data Line 3x
0x37 Vertical effective Opical Black 64x
0x2C Effective pixel area 2848x (Margin 4 + Effective 2840 + Margin 4)
0x01 End Frame

As requested here is my device tree:

		i2c_0: i2c@0 {
			imx566_cam0: becom_imx566_a@1a {
				compatible = "becom,imx566";
				/* I2C device address */
				reg = <0x1a>;

				///* V4L2 device node location */
			    devnode = "video0";

				///* Physical dimensions of sensor */
				physical_w = "2.740";
				physical_h = "2.740";

				sensor_model = "imx566";

				//use_sensor_mode_id = "true";

				mode0 { /* IMX566_MODE_2840x2840 */
                    //mclk_multiplier must be equal or greater than pix_clk_hz / mclk to ensure that the ISP runs fast enough to process data from sensor.
					mclk_khz = "37125";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					phy_mode = "DPHY";
					discontinuous_clk = "yes";
					dpcm_enable = "false";
					cil_settletime = "0";

					active_w = "2856";
					active_h = "2848"; // 2848(Active+Margin)
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "12";
					readout_orientation = "0";
					line_length = "2912";
					inherent_gain = "1";
					mclk_multiplier = "20";
                    //pixel_clk_hz = sensor data rate per lane (Mbps) * number of lanes / bits per pixel
					pix_clk_hz = "198000000"; //1188Mbps * 2 / 12

					//gain_factor = "16";
					//framerate_factor = "1000000";
					//exposure_factor = "1000000";
					//min_gain_val = "16"; /* 1.00x */
					//max_gain_val = "356"; /* 22x */
					//step_gain_val = "1";
					//default_gain = "16"; /* 1.00x */
					//min_hdr_ratio = "1";
					//max_hdr_ratio = "1";
					//min_framerate = "2000000"; /* 2.0 fps */
					//max_framerate = "30000000"; /* 30.0 fps */
					//step_framerate = "1";
					//default_framerate = "30000000"; /* 30.0 fps */
					//min_exp_time = "13"; /* us */
					//max_exp_time = "683709"; /* us */
					//step_exp_time = "1";
					//default_exp_time = "2495"; /* us */

					embedded_metadata_height = "4";
				};
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						becom_imx566_dual_out0: endpoint {
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&becom_imx566_csi_in0>;
						};
					};
				};
			};
		};

Regarding the disabling of the meta data: I haven’t found a possibility to do it in the datasheet. However, I will try to get a definitive answer regarding this from where we purchase the sensors.

Do you have any other suggestions, which I can try if disabling the meta data is not possible?

Thank you for your help. It’s very appreciated!

7

Just stumbeld upon this entry:

I configured the embedded data lines wrong. Only Embedded data with CSI-2 Header 0x12 is counted. Therefore, I had to change it from 4 to 1.

Thank you for your help!

8

hello bernhard.muellner,

does it meant you’re now able to enable the stream correctly?

9

I am able to receive a frame. I have to check if the data I receive is correct, however, my current problem with receiving seems to be fixed. I will mark it as solved, and start a new thread when I encounter the next problem :-D

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.