TX2 CSI Capture with Jetpack 4.5.1

Hi,

I’m trying to update our system, switching from Jetpack 3.2.x to Jetpack 4.5.1. I’m having issues ingesting video with our ported driver though.

We ingest video via an FPGA, which we have to tell (from userspace) to start sending video via CSI. The FPGA waits for the start of an external video (SDI) frame before it actually starts sending video over CSI as YUV422. We had to be careful to get FPGA video sending enabled very soon after the camera driver has been enabled, to avoid timeouts. We have found that if we fail to do this quickly then the CSI/VI never seem to find the start of the frame. This has actually worked well for us on 3.2.x.

I’m struggling to get this to work on 4.5.1. Dmesg says:

[ 1045.562417] v4fpga 2-0010: ext_camera_power_on: power on
[ 1045.575874] v4fpga 2-0010: ext_camera_power_on: powered on
[ 1045.789529] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 1045.795895] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[ 1045.805593] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000004
[ 1045.814318] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[ 1045.822174] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004
[ 1046.037601] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 1046.043966] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[ 1046.053769] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000004
[ 1046.062483] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[ 1046.070394] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004
[ 1046.103195] tegra-vi4 15700000.vi: Status:  2 channel:00 frame:0002
[ 1046.109476] tegra-vi4 15700000.vi:      timestamp sof 1054280353632 eof 1054297017760 data 0x000000a0
[ 1046.118715] tegra-vi4 15700000.vi:      capture_id 101 stream  0 vchan  0
[ 1046.289532] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
[ 1046.295113] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[ 1046.305611] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000004
[ 1046.314340] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[ 1046.322184] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004
[ 1046.330938] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000004

As you can see, there is a fairly quick timeout which we don’t see this on 3.2.x. There is then a 16ms(?) sof to eof, which make sense given the 720p60 input. I don’t know what the ATOMP_FE sync point time out is trying to tell me though. From this point it just fails to get any further capture done as the retries are not visible to our userspace code and the FPGA is never resynchronised.

I tried playing around with set_mode_delay_ms to change the timeout period. this had very little effect. The Camera/FPGA dts stanza looks like:

	v4fpga@10 {
		reg = <0x10>;
		devnode = "video0";
		mclk = "extperiph1";

		compatible = "vision4ce,v4fpga";
		clock-names = "extperiph1", "pllp_grtba";
		reset-gpios = <0x27 0x0 0x0>;
		physical_h = "4.930";
		physical_w = "5.095";
		clocks = <0x10 0x59 0x10 0x10d>;
		sensor_model = "v4fpga";
		status = "okay";
		vif-supply = <&en_vdd_cam>;
		iovdd-reg = "vif";
		vana-supply = <&en_vdd_cam_hv_2v8>;
		avdd-reg = "vana";
		vdig-supply = <&en_vdd_cam_1v2>;
		dvdd-reg = "vdig";
		vvcm-supply = <&en_vdd_vcm_2v8>;
		vcmvdd-reg = "vvcm";
		use_sensor_mode_id = <0>;
		set_mode_delay_ms = "10000";

I set the timeout delay to 10seconds here, but did try other smaller values (100ms).

As the FPGA is fully powered by it’s own supplies, the driver actually ignores the regulators and I assume I can remove this later.

There are two obvious differences in our setup for 4.5.1.

1: the camera driver is an external module. Previously we had to included it as a built-in and recompile the kernel, which we really don’t want to do this time around. This leads into the second one…

2: As we have to ingest YUV422, and the nvidia kernel only accepts bayer sensors, we lie to the camera block and then immediately “correct” the pixel formats:

err = camera_common_initialize(common_data, "v4fpga");
if (err) {
	dev_err(&client->dev, "Failed to initialize v4fpga.\n");
	return err;
}

/* hard code the pixel format as uyvy is not supported by nvidia dt parser */
sensor = &common_data->sensor_props;
for (i = 0; i < sensor->num_modes; ++i)
	sensor->sensor_modes[i].image_properties.pixel_format = V4L2_PIX_FMT_UYVY;

The mode for this 720p60 input looks like:

		mode1 {
			inherent_gain = "1";
			pix_clk_hz = "74250000";
			max_gain_val = "16.0";
			min_hdr_ratio = "1";
			min_framerate = "24";
			cil_settletime = "0";
			max_exp_time = "683709";
			active_h = "720";
			active_w = "1280";
			mclk_khz = "37125";
			min_gain_val = "1.0";
			max_hdr_ratio = "64";
			max_framerate = "60";
			tegra_sinterface = "serial_a";
			phy_mode = "DPHY";
			line_length = "1980";
			mode_type = "yuv";
			pixel_phase = "uyvy";
			pixel_t = "bayer_xrggb10p";
			csi_pixel_bit_depth = "8";
			dynamic_pixel_bit_depth = "8";
			readout_orientation = "0";
			mclk_multiplier = "2";
			num_lanes = "4";
			discontinuous_clk = "no";
			min_exp_time = "13";
			embedded_metadata_height = "0";
		};

Does this approach of overriding the pixel formats sound workable? Any ideas why the set_mode_delay_ms seems to have no effect?

Thanks,
Ratbert

Have apply below patch to try.

Thanks for the reply Shane. It not 100% clear why you are suggesting applying that patch. I assume you have seen something in my message that indicates ECC errors have been detected?

These where not detected with the same hardware on the 3.2.x Jetpacks. Is this because that version of the kernel masked the ECC errors by default?

The FPGA I mentioned is ours and we suspect that our ECC codes might be wrong. We will try to correct them as ECC error detection and possible bit corrections sound desirable to me.

For completness can you please tell me exactly what you saw in my message that indicates that bad ECCs are detected. Is it this:

ERROR_STATUS2VI_VC0 = 0x00000004

Knowing this will help us check our corrected ECC codes are working.

Thanks,
Ratbert

Please have a check the TRM find the REG NVCSI_STREAM_0_ERROR_STATUS2VI_VC2_0 for the detail.
You can download TRM from download center.

Shane,

I applied the patch and it has changed something, but still no video.

The dmesg output has changed to:

[  289.882573] v4fpga 2-0010: charm100_camera_power_on: power on
[  289.896067] v4fpga 2-0010: charm100_camera_power_on: powered on
[  290.115623] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  290.122085] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  290.132798] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000004
[  290.347617] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  290.354028] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  290.364827] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000004
[  290.575617] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  290.582071] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  290.591818] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000006
[  290.803612] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  290.810020] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  290.819999] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000006
[  291.047643] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  291.054018] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  291.063801] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000006
[  291.073042] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x00000006

I had to modify the patch slightly to take account of the port number variable being called csi_port, not port_num.

Some of the errors are now not reported, but the VI is still not happy.

Do you have any other ideas where to look?

Thanks,
Ratbert

What’s the trace log.

https://elinux.org/Jetson_TX2_Camera_BringUp

It repeats, so this is 1 and a bit cycles:

 kworker/4:0-36    [004] ....  3861.575179: rtos_queue_send_failed: tstamp:120820256795 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.575181: rtcpu_vinotify_event: tstamp:120820505607 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:120820505092 data:0x00000001
 kworker/4:0-36    [004] ....  3861.575181: rtcpu_vinotify_event: tstamp:120820505766 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:120820505107 data:0x00000000
 kworker/4:0-36    [004] ....  3861.575182: rtcpu_vinotify_event: tstamp:120820507408 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:2 vi_tstamp:120820507041 data:0x08000000
 kworker/4:0-36    [004] ....  3861.575182: rtcpu_vinotify_event: tstamp:120821026556 tag:CSIMUX_FRAME channel:0x00 frame:2 vi_tstamp:120821025847 data:0x000000a0
 kworker/4:0-36    [004] ....  3861.575183: rtcpu_vinotify_event: tstamp:120821026758 tag:CHANSEL_SHORT_FRAME channel:0x01 frame:2 vi_tstamp:120821025847 data:0x00000001
 kworker/4:0-36    [004] ....  3861.575183: rtcpu_vinotify_event: tstamp:120821026895 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:120821025850 data:0x00000000
 kworker/4:0-36    [004] ....  3861.631211: rtos_queue_peek_from_isr_failed: tstamp:120823100342 queue:0x0b4b4500
 kworker/4:0-36    [004] ....  3861.799170: rtos_queue_send_from_isr_failed: tstamp:120827341731 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799174: rtos_queue_send_from_isr_failed: tstamp:120827341846 queue:0x0b4aad68
 kworker/4:0-36    [004] ....  3861.799174: rtos_queue_send_from_isr_failed: tstamp:120827341951 queue:0x0b4ac998
 kworker/4:0-36    [004] ....  3861.799175: rtos_queue_send_from_isr_failed: tstamp:120827342055 queue:0x0b4ae518
 kworker/4:0-36    [004] ....  3861.799176: rtos_queue_send_from_isr_failed: tstamp:120827342155 queue:0x0b4af2d8
 kworker/4:0-36    [004] ....  3861.799176: rtos_queue_send_from_isr_failed: tstamp:120827342257 queue:0x0b4b0098
 kworker/4:0-36    [004] ....  3861.799177: rtos_queue_send_from_isr_failed: tstamp:120827342357 queue:0x0b4b0e58
 kworker/4:0-36    [004] ....  3861.799178: rtos_queue_send_from_isr_failed: tstamp:120827342459 queue:0x0b4b1c18
 kworker/4:0-36    [004] ....  3861.799179: rtos_queue_send_failed: tstamp:120827343074 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799180: rtos_queue_send_from_isr_failed: tstamp:120827346196 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799180: rtos_queue_send_from_isr_failed: tstamp:120827346299 queue:0x0b4aad68
 kworker/4:0-36    [004] ....  3861.799181: rtos_queue_send_from_isr_failed: tstamp:120827346401 queue:0x0b4ac998
 kworker/4:0-36    [004] ....  3861.799182: rtos_queue_send_from_isr_failed: tstamp:120827346506 queue:0x0b4ae518
 kworker/4:0-36    [004] ....  3861.799182: rtos_queue_send_from_isr_failed: tstamp:120827346607 queue:0x0b4af2d8
 kworker/4:0-36    [004] ....  3861.799183: rtos_queue_send_from_isr_failed: tstamp:120827346710 queue:0x0b4b0098
 kworker/4:0-36    [004] ....  3861.799183: rtos_queue_send_from_isr_failed: tstamp:120827346811 queue:0x0b4b0e58
 kworker/4:0-36    [004] ....  3861.799184: rtos_queue_send_from_isr_failed: tstamp:120827346912 queue:0x0b4b1c18
 kworker/4:0-36    [004] ....  3861.799184: rtos_queue_send_failed: tstamp:120827347328 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799185: rtos_queue_send_from_isr_failed: tstamp:120827623868 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799186: rtos_queue_send_from_isr_failed: tstamp:120827623973 queue:0x0b4aad68
 kworker/4:0-36    [004] ....  3861.799186: rtos_queue_send_from_isr_failed: tstamp:120827624078 queue:0x0b4ac998
 kworker/4:0-36    [004] ....  3861.799187: rtos_queue_send_from_isr_failed: tstamp:120827624182 queue:0x0b4ae518
 kworker/4:0-36    [004] ....  3861.799187: rtos_queue_send_from_isr_failed: tstamp:120827624284 queue:0x0b4af2d8
 kworker/4:0-36    [004] ....  3861.799188: rtos_queue_send_from_isr_failed: tstamp:120827624386 queue:0x0b4b0098
 kworker/4:0-36    [004] ....  3861.799188: rtos_queue_send_from_isr_failed: tstamp:120827624487 queue:0x0b4b0e58
 kworker/4:0-36    [004] ....  3861.799189: rtos_queue_send_from_isr_failed: tstamp:120827624588 queue:0x0b4b1c18
 kworker/4:0-36    [004] ....  3861.799190: rtos_queue_send_failed: tstamp:120827625043 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799190: rtos_queue_send_from_isr_failed: tstamp:120827626373 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799191: rtos_queue_send_from_isr_failed: tstamp:120827626477 queue:0x0b4aad68
 kworker/4:0-36    [004] ....  3861.799191: rtos_queue_send_from_isr_failed: tstamp:120827626582 queue:0x0b4ac998
 kworker/4:0-36    [004] ....  3861.799192: rtos_queue_send_from_isr_failed: tstamp:120827626685 queue:0x0b4ae518
 kworker/4:0-36    [004] ....  3861.799193: rtos_queue_send_from_isr_failed: tstamp:120827626786 queue:0x0b4af2d8
 kworker/4:0-36    [004] ....  3861.799193: rtos_queue_send_from_isr_failed: tstamp:120827626887 queue:0x0b4b0098
 kworker/4:0-36    [004] ....  3861.799194: rtos_queue_send_from_isr_failed: tstamp:120827626988 queue:0x0b4b0e58
 kworker/4:0-36    [004] ....  3861.799194: rtos_queue_send_from_isr_failed: tstamp:120827627089 queue:0x0b4b1c18
 kworker/4:0-36    [004] ....  3861.799195: rtos_queue_send_failed: tstamp:120827627979 queue:0x0b4a7258
 kworker/4:0-36    [004] ....  3861.799197: rtcpu_vinotify_event: tstamp:120827797264 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi_tstamp:120827796753 data:0x00000001
 kworker/4:0-36    [004] ....  3861.799197: rtcpu_vinotify_event: tstamp:120827797423 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp:120827796769 data:0x00000000
 kworker/4:0-36    [004] ....  3861.799197: rtcpu_vinotify_event: tstamp:120827799534 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:2 vi_tstamp:120827799168 data:0x08000000
 kworker/4:0-36    [004] ....  3861.799199: rtos_queue_peek_from_isr_failed: tstamp:120828100338 queue:0x0b4b4500
 kworker/4:0-36    [004] ....  3861.799199: rtcpu_vinotify_event: tstamp:120828318221 tag:CSIMUX_FRAME channel:0x00 frame:2 vi_tstamp:120828317508 data:0x000000a0
 kworker/4:0-36    [004] ....  3861.799200: rtcpu_vinotify_event: tstamp:120828318419 tag:CHANSEL_SHORT_FRAME channel:0x01 frame:2 vi_tstamp:120828317509 data:0x00000001
 kworker/4:0-36    [004] ....  3861.799200: rtcpu_vinotify_event: tstamp:120828318557 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:120828317512 data:0x00000000

The trace log show the “tag:CHANSEL_SHORT_FRAME” that tell the output size didn’t as expect.
You can adjust the active_y in device tree to try. Can confirm the modify by v4l2-ctl --list-formats-ext

Interesting. These sizes appear correct (below) and using 3.2.1, we know this video is 720p. 1280x720 is what V4L reports as well. You mentioned the metadata lines above, but I assumed that was a cut and paste error from the original post. Should I try 4 lines as shown there?

root@nvidia-desktop:~# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture	
    Pixel Format: 'UYVY'
    Name        : UYVY 4:2:2
        Size: Discrete 1920x1080
        Size: Discrete 1280x720
        Size: Discrete 720x289
	    Size: Discrete 720x245
        Size: Discrete 1920x541
        Size: Discrete 640x512
        Size: Discrete 640x480
        Size: Discrete 1280x1024

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'NV16'
    Name        : Y/CbCr 4:2:2
        Size: Discrete 1920x1080
        Size: Discrete 1280x720
        Size: Discrete 720x289
        Size: Discrete 720x245
        Size: Discrete 1920x541
        Size: Discrete 640x512
        Size: Discrete 640x480
        Size: Discrete 1280x1024

    Index       : 2
    Type        : Video Capture
    Pixel Format: 'UYVY'
    Name        : UYVY 4:2:2
        Size: Discrete 1920x1080
        Size: Discrete 1280x720
        Size: Discrete 720x289
        Size: Discrete 720x245
        Size: Discrete 1920x541
        Size: Discrete 640x512
        Size: Discrete 640x480
        Size: Discrete 1280x1024

I know some of these sizes are odd looking, but they are correct, for our use case. SDI has interlaced options, which we transport over the CSI as two “frames (fields)” and recombine the fields to get a full frame.

I would suggest to modify the resolution to check and check the trace log.
I only can check the trace and give advice.

I tried 360 lines and saw no difference. I will try some smaller sizes. Above you said “active_y”. Did you mean “active_h”? Am I missing some extra or renamed parameters? grep showed nothing for “active_y”.

Yes, right it’s active_h
Did you confirm the modify by the v4l2-crl --list-formats-ext?

I confirmed with:

nvidia@nvidia-desktop:~/fpga_tools$ cat /proc/device-tree/i2c@3180000/v4fpga@10/mode1/active_h
10

I thought the V4L sizes formats came from the driver (for mode matching)? E.g.:

static const struct camera_common_frmfmt charm100_camera_frmfmt[] = {
	{{1920, 1080}, NULL, 0, 0, CHARM100_MODE_1920x1080},
	{{1280, 720},  NULL, 0, 0, CHARM100_MODE_1280x720},
	{{720, 289},  NULL, 0, 0, CHARM100_MODE_720x576i},
	{{720, 245},  NULL, 0, 0, CHARM100_MODE_720x487i},
	{{1920, 541},  NULL, 0, 0, CHARM100_MODE_1920x1080i},
	{{640, 512},  NULL, 0, 0, CHARM100_MODE_640x512},
	{{640, 480},  NULL, 0, 0, CHARM100_MODE_640x480},
	{{1280, 1024},  NULL, 0, 0, CHARM100_MODE_1280x1024},
};

v4l2-ctl still shows 720 lines.

If v4l2-ctl still show the same that tell you need to modify the width of the sensor mode in sensor driver.

I’ve changed the format in the driver as well and list-formats-ext now shows:

root@nvidia-desktop:~# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'UYVY'
	Name        : UYVY 4:2:2
		Size: Discrete 1920x1080
		Size: Discrete 1280x10
		Size: Discrete 720x289
		Size: Discrete 720x245
		Size: Discrete 1920x541
		Size: Discrete 640x512
		Size: Discrete 640x480
		Size: Discrete 1280x1024

I’ve snipped of the NV16 and repeated UYVY parts above, but they are there.

Using v4l2-ctl to stream the data and requesting 1280x10 as the resolution results in the following trace:

 kworker/5:3-5169  [005] ....   564.549830: rtcpu_vinotify_event: tstamp:17789075823 tag:CSIMUX_STREAM channel:0xff frame:1 vi_tstamp:17789075143 data:0x00000001
 kworker/5:3-5169  [005] ....   564.549834: rtcpu_vinotify_event: tstamp:17789125819 tag:CSIMUX_STREAM channel:0xff frame:2 vi_tstamp:17789125143 data:0x00000001
 kworker/5:3-5169  [005] ....   564.549838: rtcpu_vinotify_event: tstamp:17789295518 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:17789294586 data:0x00000001
 kworker/5:3-5169  [005] ....   564.549842: rtcpu_vinotify_event: tstamp:17789296408 tag:CHANSEL_FAULT channel:0x00 frame:1 vi_tstamp:17789294590 data:0x00000200
 kworker/5:3-5169  [005] ....   564.549846: rtcpu_vinotify_event: tstamp:17789297349 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:17789294593 data:0x00000000
 kworker/5:3-5169  [005] ....   564.549850: rtcpu_vinotify_event: tstamp:17789298289 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:1 vi_tstamp:17789296801 data:0x08000000
 kworker/5:3-5169  [005] ....   564.549854: rtcpu_vinotify_event: tstamp:17789298945 tag:CHANSEL_FAULT_FE channel:0x01 frame:1 vi_tstamp:17789297366 data:0x00000001
 kworker/5:3-5169  [005] ....   564.549859: rtcpu_vinotify_event: tstamp:17789299625 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:17789297369 data:0x00000000
 kworker/5:3-5169  [005] ....   564.605796: rtos_queue_peek_from_isr_failed: tstamp:17791368709 queue:0x0b4b4500
 kworker/5:3-5169  [005] ....   564.777575: rtos_queue_send_from_isr_failed: tstamp:17796242772 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.777579: rtos_queue_send_from_isr_failed: tstamp:17796242951 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   564.777581: rtos_queue_send_from_isr_failed: tstamp:17796243129 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   564.777582: rtos_queue_send_from_isr_failed: tstamp:17796243305 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   564.777583: rtos_queue_send_from_isr_failed: tstamp:17796243479 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   564.777584: rtos_queue_send_from_isr_failed: tstamp:17796243654 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   564.777586: rtos_queue_send_from_isr_failed: tstamp:17796243828 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   564.777587: rtos_queue_send_from_isr_failed: tstamp:17796244003 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   564.777589: rtos_queue_send_failed: tstamp:17796244874 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.777590: rtos_queue_send_from_isr_failed: tstamp:17796247649 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.777591: rtos_queue_send_from_isr_failed: tstamp:17796247867 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   564.777592: rtos_queue_send_from_isr_failed: tstamp:17796248043 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   564.777593: rtos_queue_send_from_isr_failed: tstamp:17796248221 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   564.777595: rtos_queue_send_from_isr_failed: tstamp:17796248396 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   564.777596: rtos_queue_send_from_isr_failed: tstamp:17796248571 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   564.777597: rtos_queue_send_from_isr_failed: tstamp:17796248745 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   564.777598: rtos_queue_send_from_isr_failed: tstamp:17796248918 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   564.777600: rtos_queue_send_failed: tstamp:17796249633 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.777601: rtos_queue_peek_from_isr_failed: tstamp:17796368270 queue:0x0b4b4500
 kworker/5:3-5169  [005] ....   564.829672: rtos_queue_send_from_isr_failed: tstamp:17797694788 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.829693: rtos_queue_send_from_isr_failed: tstamp:17797694964 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   564.829699: rtos_queue_send_from_isr_failed: tstamp:17797695141 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   564.829705: rtos_queue_send_from_isr_failed: tstamp:17797695316 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   564.829714: rtos_queue_send_from_isr_failed: tstamp:17797695488 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   564.829718: rtos_queue_send_from_isr_failed: tstamp:17797695662 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   564.829725: rtos_queue_send_from_isr_failed: tstamp:17797695836 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   564.829729: rtos_queue_send_from_isr_failed: tstamp:17797696015 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   564.829742: rtos_queue_send_failed: tstamp:17797696729 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.829748: rtos_queue_send_from_isr_failed: tstamp:17797703240 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.829754: rtos_queue_send_from_isr_failed: tstamp:17797703413 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   564.829761: rtos_queue_send_from_isr_failed: tstamp:17797703587 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   564.829767: rtos_queue_send_from_isr_failed: tstamp:17797703762 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   564.829773: rtos_queue_send_from_isr_failed: tstamp:17797703935 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   564.829779: rtos_queue_send_from_isr_failed: tstamp:17797704107 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   564.829785: rtos_queue_send_from_isr_failed: tstamp:17797704281 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   564.829791: rtos_queue_send_from_isr_failed: tstamp:17797704453 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   564.829797: rtos_queue_send_failed: tstamp:17797706374 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   564.829810: rtcpu_vinotify_event: tstamp:17797712354 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:17797711950 data:0x00000001
 kworker/5:3-5169  [005] ....   564.829816: rtcpu_vinotify_event: tstamp:17797811214 tag:CSIMUX_STREAM channel:0xff frame:1 vi_tstamp:17797810560 data:0x00000001
 kworker/5:3-5169  [005] ....   564.829820: rtcpu_vinotify_event: tstamp:17797844314 tag:CSIMUX_STREAM channel:0xff frame:2 vi_tstamp:17797843893 data:0x00000001
 kworker/5:3-5169  [005] ....   564.941691: rtos_queue_peek_from_isr_failed: tstamp:17801368699 queue:0x0b4b4500
 kworker/5:3-5169  [005] ....   565.054248: rtos_queue_send_from_isr_failed: tstamp:17804585442 queue:0x0b4a7258

The trace repeats every 16.6ms (60hz is still my video frame rate) but those last few lines repeat more often, continuing until the next frame is due. Here is a fragment containg two:

 kworker/5:3-5169  [005] ....   564.829810: rtcpu_vinotify_event: tstamp:17797712354 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:17797711950 data:0x00000001
 kworker/5:3-5169  [005] ....   564.829816: rtcpu_vinotify_event: tstamp:17797811214 tag:CSIMUX_STREAM channel:0xff frame:1 vi_tstamp:17797810560 data:0x00000001
 kworker/5:3-5169  [005] ....   564.829820: rtcpu_vinotify_event: tstamp:17797844314 tag:CSIMUX_STREAM channel:0xff frame:2 vi_tstamp:17797843893 data:0x00000001
 kworker/5:3-5169  [005] ....   564.941691: rtos_queue_peek_from_isr_failed: tstamp:17801368699 queue:0x0b4b4500
 kworker/5:3-5169  [005] ....   565.054248: rtos_queue_send_from_isr_failed: tstamp:17804585442 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.054252: rtos_queue_send_from_isr_failed: tstamp:17804585728 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   565.054253: rtos_queue_send_from_isr_failed: tstamp:17804585998 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   565.054255: rtos_queue_send_from_isr_failed: tstamp:17804586267 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   565.054256: rtos_queue_send_from_isr_failed: tstamp:17804586533 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   565.054257: rtos_queue_send_from_isr_failed: tstamp:17804586797 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   565.054259: rtos_queue_send_from_isr_failed: tstamp:17804587063 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   565.054260: rtos_queue_send_from_isr_failed: tstamp:17804587329 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   565.054263: rtos_queue_send_failed: tstamp:17804588554 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.054264: rtos_queue_send_from_isr_failed: tstamp:17804594210 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.054266: rtos_queue_send_from_isr_failed: tstamp:17804594477 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   565.054267: rtos_queue_send_from_isr_failed: tstamp:17804594744 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   565.054269: rtos_queue_send_from_isr_failed: tstamp:17804595013 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   565.054270: rtos_queue_send_from_isr_failed: tstamp:17804595277 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   565.054271: rtos_queue_send_from_isr_failed: tstamp:17804595543 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   565.054273: rtos_queue_send_from_isr_failed: tstamp:17804595810 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   565.054274: rtos_queue_send_from_isr_failed: tstamp:17804596077 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   565.054276: rtos_queue_send_failed: tstamp:17804597169 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.109740: rtos_queue_send_from_isr_failed: tstamp:17805958689 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.109757: rtos_queue_send_from_isr_failed: tstamp:17805958862 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   565.109772: rtos_queue_send_from_isr_failed: tstamp:17805959036 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   565.109780: rtos_queue_send_from_isr_failed: tstamp:17805959210 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   565.109789: rtos_queue_send_from_isr_failed: tstamp:17805959382 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   565.109795: rtos_queue_send_from_isr_failed: tstamp:17805959555 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   565.109801: rtos_queue_send_from_isr_failed: tstamp:17805959727 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   565.109810: rtos_queue_send_from_isr_failed: tstamp:17805959900 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   565.109817: rtos_queue_send_failed: tstamp:17805960600 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.109825: rtos_queue_send_from_isr_failed: tstamp:17805965507 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.109831: rtos_queue_send_from_isr_failed: tstamp:17805965775 queue:0x0b4aad68
 kworker/5:3-5169  [005] ....   565.109838: rtos_queue_send_from_isr_failed: tstamp:17805966042 queue:0x0b4ac998
 kworker/5:3-5169  [005] ....   565.109844: rtos_queue_send_from_isr_failed: tstamp:17805966312 queue:0x0b4ae518
 kworker/5:3-5169  [005] ....   565.109850: rtos_queue_send_from_isr_failed: tstamp:17805966637 queue:0x0b4af2d8
 kworker/5:3-5169  [005] ....   565.109859: rtos_queue_send_from_isr_failed: tstamp:17805966906 queue:0x0b4b0098
 kworker/5:3-5169  [005] ....   565.109866: rtos_queue_send_from_isr_failed: tstamp:17805967214 queue:0x0b4b0e58
 kworker/5:3-5169  [005] ....   565.109872: rtos_queue_send_from_isr_failed: tstamp:17805967483 queue:0x0b4b1c18
 kworker/5:3-5169  [005] ....   565.109879: rtos_queue_send_failed: tstamp:17805970969 queue:0x0b4a7258
 kworker/5:3-5169  [005] ....   565.109891: rtcpu_vinotify_event: tstamp:17805973415 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:17805970282 data:0x00000001
 kworker/5:3-5169  [005] ....   565.109895: rtcpu_vinotify_event: tstamp:17806144576 tag:CSIMUX_STREAM channel:0xff frame:1 vi_tstamp:17806143895 data:0x00000001
 kworker/5:3-5169  [005] ....   565.109899: rtcpu_vinotify_event: tstamp:17806177880 tag:CSIMUX_STREAM channel:0xff frame:2 vi_tstamp:17806177226 data:0x00000001

Note that the short frame has gone.

v4l2-ctl doesn’t capture any frames.

Now it show PIXEL_SHORT_LINE, you can use binary search to find the real size.

tag:CHANSEL_FAULT