MIPI issue between FPGA and Xavier NX

Hi,

We are trying to acquire some images from a FPGA by using the MIPI interface of the Xavier NX. The FPGA sends a 640x480,16 bit test pattern to the NX. From NX perspective we are considering a 1280x480 format to be compatible with RAW8 format. We made the following changes to get it working:

  • DTS changes. We are consider the IMX219 dts as a starting point for FPGA support. You can find the configuration below:
mode2 {
  	mclk_khz = "24000";
	num_lanes = [32 00];
	tegra_sinterface = "serial_a";
	phy_mode = "DPHY";
	discontinuous_clk = "yes";
	dpcm_enable = "false";
	cil_settletime = [30 00];
	active_w = "1280";
	active_h = "480";
	pixel_t = "bayer_rggb";
	readout_orientation = "90";
	line_length = "2780";
	inherent_gain = [31 00];
	mclk_multiplier = "9.33";
	pix_clk_hz = "100000000";
	serdes_pix_clk_hz = "535000000";
	gain_factor = "16";
	framerate_factor = "1000000";
	exposure_factor = "1000000";
	min_gain_val = "16";
	max_gain_val = "170";
	step_gain_val = [31 00];
	default_gain = "16";
	min_hdr_ratio = [31 00];
	max_hdr_ratio = [31 00];
	min_framerate = "20000000";
	max_framerate = "60000000";
	step_framerate = [31 00];
	default_framerate = "50000000";
	min_exp_time = "13";
	max_exp_time = "683709";
	step_exp_time = [31 00];
	default_exp_time = "2495";
	embedded_metadata_height = "0";
};

  • Kernel changes. We remove the I2C part from the sensor kernel driver and we modify the file nvidia/drivers/media/platform/tegra/camera/vi/capture.c to not wait the capture_resp. In this way, the Xavier starts the frame capture without I2C handshake. Moreover, we have changed the camera_common_frmfmt imx219_frmfm structure to be compatible with the new format 1280x480.
static const struct camera_common_frmfmt imx219_frmfmt[] = {
	{{3264, 2464},	imx219_21fps, 1, 0, IMX219_MODE_3264x2464_21FPS},
	/* Add modes with no device tree support after below */
	{{3264, 1848},	imx219_28fps, 1, 0, IMX219_MODE_3264x1848_28FPS},
	{{1280, 480},	imx219_30fps, 1, 0, IMX219_MODE_1280x480_30FPS},
	{{1280, 720},	imx219_60fps, 1, 0, IMX219_MODE_1280x720_60FPS},
	{{1280, 720},	imx219_120fps, 1, 0, IMX219_MODE_1280x720_120FPS},
};
  • RCE fw has been updated to remove ECC/CRC check as described in this post. We are using Jetpack 4.4.1 to be compatible with RCE fw.

We are facing an issue since v4l2 can only acquire few frames before blocking. Here is the v4l2 command with the dmesg output:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=480,pixelformat=RGGB --set-ctrl bypass_mode=0,sensor_mode=2 --stream-mmap --stream-count=5
[ 3654.472372] [RCE] vi5_hwinit: firmware CL2018101701 protocol version 2.2
[ 3654.472388] [RCE] error config mask = 3
[ 3654.472396] [RCE] nvcsi_stream_apply_error_config stream_intr_mask ECC/CRC = 3
[ 3654.472413] tegra194-vi5 15c10000.vi: corr_err: discarding frame 56128, flags: 0, err_data 0
[ 3654.481264] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.489179] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.497290] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.505749] tegra194-vi5 15c10000.vi: corr_err: discarding frame 56129, flags: 0, err_data 0
[ 3654.514334] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.522610] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.530624] [RCE] ERROR: capture-scheduler.c:2375 [capture_scheduler_thread] "Error processing event 0x100000"
[ 3654.530644] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.538864] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.546621] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.555008] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.562983] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.571206] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.578990] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.587338] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.595089] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.603555] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.611369] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.619860] tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0
[ 3654.695797] [RCE] error config mask = e59ff018
[ 3654.695874] [RCE] nvcsi_stream_apply_error_config stream_intr_mask ECC/CRC = 0

Checking the trace log, it shows the following output:

rtcpu_nvcsi_intr: tstamp:3600903419 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600903419 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600905158 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600905158 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600906896 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600906896 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600908641 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600908641 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600910372 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600910372 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600912108 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600912108 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600913846 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600913846 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600915583 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600915583 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600917322 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600917322 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600919057 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600919057 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600920795 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600920795 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600922536 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600922536 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600924271 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600924271 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600926009 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600926009 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600927749 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600927749 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600929485 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600929485 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600931224 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600931224 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600932960 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
rtcpu_nvcsi_intr: tstamp:3600932960 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004

Is the Xavier NX device tree correctly configured? Why are we able to only acquire few frames? May it be an hardware issue? Is there a synchronization/clock problem?

Any hints? @ShaneCCC?

Thank you.

1 Like

Did you make status2vi_notify_mask = 0xFFFF ? in csi5_fops.c

Yes. We changed the RCE fw and done the patch to csi5_fops with error_config.status2vi_notify_mask = 0xFFFF.

Thank you

Have configure stream_intr_mask to 0xFFFF too.

error_config.stream_intr_mask = 0xFFFF;

We have changed the error_config.stream_intr_mask = 0xFFFF as you suggested. The previous issue about capturing only few frames still persist but the trace log is changed:

kworker/1:2-1674  [001] ....   416.143046: rtcpu_string: tstamp:13314802054 id:0x04010000 str:"error config mask = ffff"
kworker/1:2-1674  [001] ....   416.143050: rtcpu_string: tstamp:13314802378 id:0x04010000 str:"nvcsi_stream_apply_error_config stream_intr_mask"
kworker/1:2-1674  [001] ....   416.143051: rtcpu_string: tstamp:13314802497 id:0x04010000 str:" ECC/CRC = ffff"
kworker/1:2-1674  [001] ....   416.143056: rtcpu_vinotify_error: tstamp:13314810317 tag:CSIMUX_STREAM channel:0x00 frame:0 vi_tstamp:13314809461 data:0x00000001
kworker/1:2-1674  [001] ....   416.143057: rtos_queue_send_from_isr_failed: tstamp:13314946566 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.143058: rtos_queue_send_from_isr_failed: tstamp:13314946719 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.143058: rtos_queue_send_from_isr_failed: tstamp:13314946872 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.143059: rtos_queue_send_from_isr_failed: tstamp:13314947017 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.143060: rtos_queue_send_from_isr_failed: tstamp:13314947165 queue:0x0bcbc1e0
vi-output, imx2-17414 [002] ....   416.196339: tegra_channel_capture_frame: sof:426.88496384
vi-output, imx2-17414 [002] ....   416.196342: tegra_channel_capture_frame: eof:426.115141664
kworker/1:2-1674  [001] ....   416.198936: rtcpu_vinotify_event: tstamp:13315095817 tag:CSIMUX_STREAM cannel:0x00 frame:0 vi_tstamp:13314809461 data:0x00000001
kworker/1:2-1674  [001] ....   416.198941: rtos_queue_send_from_isr_failed: tstamp:13315189040 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.198942: rtos_queue_send_from_isr_failed: tstamp:13315189189 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.198943: rtos_queue_send_from_isr_failed: tstamp:13315189341 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.198945: rtos_queue_send_from_isr_failed: tstamp:13315189490 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.198946: rtos_queue_send_from_isr_failed: tstamp:13315189638 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.198970: rtcpu_vinotify_event: tstamp:13315434962 tag:FS channel:0x00 frame:16576 vi_tstamp:13315265459 data:0x00000010
kworker/1:2-1674  [001] ....   416.198971: rtcpu_vinotify_event: tstamp:13315435109 tag:ATOMP_FS channel:0x00 frame:16576 vi_tstamp:13315265460 data:0x00000000
kworker/1:2-1674  [001] ....   416.198972: rtcpu_vinotify_event: tstamp:13315435276 tag:CHANSEL_PXL_SOF channel:0x23 frame:16576 vi_tstamp:13315265512 data:0x00000001
kworker/1:2-1674  [001] ....   416.198972: rtcpu_vinotify_event: tstamp:13315435416 tag:RESERVED_19 channel:0x23 frame:192 vi_tstamp:13771638560 data:0x08020001
kworker/1:2-1674  [001] ....   416.198973: rtos_queue_send_from_isr_failed: tstamp:13315455728 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.198974: rtos_queue_send_from_isr_failed: tstamp:13315455885 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.198975: rtos_queue_send_from_isr_failed: tstamp:13315456040 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.198975: rtos_queue_send_from_isr_failed: tstamp:13315456190 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.198976: rtos_queue_send_from_isr_failed: tstamp:13315456339 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.198977: rtos_queue_send_from_isr_failed: tstamp:13315698888 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.198978: rtos_queue_send_from_isr_failed: tstamp:13315699035 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.198979: rtos_queue_send_from_isr_failed: tstamp:13315699184 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.198979: rtos_queue_send_from_isr_failed: tstamp:13315699329 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.198980: rtos_queue_send_from_isr_failed: tstamp:13315699473 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.198981: rtos_queue_send_from_isr_failed: tstamp:13315977352 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.198982: rtos_queue_send_from_isr_failed: tstamp:13315977498 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.198982: rtos_queue_send_from_isr_failed: tstamp:13315977647 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.198983: rtos_queue_send_from_isr_failed: tstamp:13315977795 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.198984: rtos_queue_send_from_isr_failed: tstamp:13315977942 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.198985: rtcpu_vinotify_event: tstamp:13316099696 tag:CHANSEL_PXL_EOF channel:0x23 frame:16576 vi_tstamp:13316098128 data:0x01df0002
kworker/1:2-1674  [001] ....   416.198986: rtcpu_vinotify_event: tstamp:13316099840 tag:ATOMP_FRAME_DONE channel:0x23 frame:16576 vi_tstamp:13316098147 data:0x00000000
kworker/1:2-1674  [001] ....   416.198986: rtcpu_vinotify_event: tstamp:13316100001 tag:RESERVED_19 channel:0x23 frame:192 vi_tstamp:13798281696 data:0x02020001
kworker/1:2-1674  [001] ....   416.198987: rtcpu_vinotify_event: tstamp:13316100140 tag:FE channel:0x00 frame:16576 vi_tstamp:13316098176 data:0x00000020
kworker/1:2-1674  [001] ....   416.198988: rtcpu_vinotify_event: tstamp:13316100302 tag:ATOMP_FE channel:0x00 frame:16576 vi_tstamp:13316098177 data:0x00000000
kworker/1:2-1674  [001] ....   416.198988: rtcpu_vinotify_event: tstamp:13316100437 tag:RESERVED_19 channel:0x23 frame:192 vi_tstamp:13798287936 data:0x00020001
kworker/1:2-1674  [001] ....   416.198991: rtcpu_string: tstamp:13316131838 id:0x04010000 str:"ERROR: capture-scheduler.c:2375 [capture_schedul"
kworker/1:2-1674  [001] ....   416.198992: rtcpu_string: tstamp:13316131959 id:0x04010000 str:"er_thread] ""
kworker/1:2-1674  [001] ....   416.198993: rtcpu_string: tstamp:13316132214 id:0x04010000 str:"Error processing event 0x100000"
kworker/1:2-1674  [001] ....   416.198994: rtcpu_string: tstamp:13316132408 id:0x04010000 str:""
kworker/1:2-1674  [001] ....   416.199011: rtos_queue_send_from_isr_failed: tstamp:13316223957 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.199012: rtos_queue_send_from_isr_failed: tstamp:13316224113 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.199013: rtos_queue_send_from_isr_failed: tstamp:13316224267 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.199013: rtos_queue_send_from_isr_failed: tstamp:13316224412 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.199014: rtos_queue_send_from_isr_failed: tstamp:13316224557 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.199015: rtcpu_vinotify_event: tstamp:13316418261 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:13798295168 data:0x07020002
kworker/1:2-1674  [001] ....   416.199016: rtcpu_vinotify_event: tstamp:13316418402 tag:FS channel:0x00 frame:16577 vi_tstamp:13316129778 data:0x00000010
kworker/1:2-1674  [001] ....   416.199016: rtcpu_vinotify_event: tstamp:13316418564 tag:ATOMP_FS channel:0x00 frame:16577 vi_tstamp:13316129779 data:0x00000000
kworker/1:2-1674  [001] ....   416.199017: rtcpu_vinotify_event: tstamp:13316418704 tag:CHANSEL_PXL_SOF channel:0x23 frame:16577 vi_tstamp:13316129830 data:0x00000001
kworker/1:2-1674  [001] ....   416.199018: rtcpu_vinotify_event: tstamp:13316418861 tag:RESERVED_19 channel:0x23 frame:193 vi_tstamp:13799296768 data:0x08020002
kworker/1:2-1674  [001] ....   416.199019: rtos_queue_send_from_isr_failed: tstamp:13316486208 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.199020: rtos_queue_send_from_isr_failed: tstamp:13316486360 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.199021: rtos_queue_send_from_isr_failed: tstamp:13316486510 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.199025: rtos_queue_send_from_isr_failed: tstamp:13316486657 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.199026: rtos_queue_send_from_isr_failed: tstamp:13316486803 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.199027: rtos_queue_send_from_isr_failed: tstamp:13316740181 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.199027: rtos_queue_send_from_isr_failed: tstamp:13316740328 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.199028: rtos_queue_send_from_isr_failed: tstamp:13316740478 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.199029: rtos_queue_send_from_isr_failed: tstamp:13316740625 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.199030: rtos_queue_send_from_isr_failed: tstamp:13316740771 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.199031: rtos_queue_send_from_isr_failed: tstamp:13316750360 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.199031: rtos_queue_send_from_isr_failed: tstamp:13316750507 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.199032: rtos_queue_send_from_isr_failed: tstamp:13316750656 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.199033: rtos_queue_send_from_isr_failed: tstamp:13316750802 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.199034: rtos_queue_send_from_isr_failed: tstamp:13316750949 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.258975: rtos_queue_send_from_isr_failed: tstamp:13316987780 queue:0x0bcb4278
kworker/1:2-1674  [001] ....   416.258979: rtos_queue_send_from_isr_failed: tstamp:13316987937 queue:0x0bcb8ae0
kworker/1:2-1674  [001] ....   416.258980: rtos_queue_send_from_isr_failed: tstamp:13316988090 queue:0x0bcba660
kworker/1:2-1674  [001] ....   416.258981: rtos_queue_send_from_isr_failed: tstamp:13316988237 queue:0x0bcbb420
kworker/1:2-1674  [001] ....   416.258982: rtos_queue_send_from_isr_failed: tstamp:13316988406 queue:0x0bcbc1e0
kworker/1:2-1674  [001] ....   416.258984: rtcpu_vinotify_error: tstamp:13316995009 tag:CHANSEL_NOMATCH channel:0x01 frame:16578 vi_tstamp:13316994149 data:0x00000549
kworker/1:2-1674  [001] ....   416.258986: rtcpu_vinotify_event: tstamp:13317096505 tag:CHANSEL_PXL_EOF channel:0x23 frame:16577 vi_tstamp:13316962447 data:0x01df0002
kworker/1:2-1674  [001] ....   416.258987: rtcpu_vinotify_event: tstamp:13317096667 tag:ATOMP_FRAME_DONE channel:0x23 frame:16577 vi_tstamp:13316962465 data:0x00000000
kworker/1:2-1674  [001] ....   416.258987: rtcpu_vinotify_event: tstamp:13317096806 tag:RESERVED_19 channel:0x23 frame:193 vi_tstamp:13825939872 data:0x02020002
 kworker/1:2-1674  [001] ....   416.258988: rtcpu_vinotify_event: tstamp:13317096964 tag:FE channel:0x00 frame:16577 vi_tstamp:13316962495 data:0x00000020
kworker/1:2-1674  [001] ....   416.258989: rtcpu_vinotify_event: tstamp:13317097104 tag:ATOMP_FE channel:0x00 frame:16577 vi_tstamp:13316962496 data:0x00000000
kworker/1:2-1674  [001] ....   416.258990: rtcpu_vinotify_event: tstamp:13317097261 tag:RESERVED_19 channel:0x23 frame:193 vi_tstamp:13825946176 data:0x00020002
kworker/1:2-1674  [001] ....   416.258990: rtcpu_vinotify_event: tstamp:13317097397 tag:FS channel:0x00 frame:16578 vi_tstamp:13316994097 data:0x00000010
kworker/1:2-1674  [001] ....   416.258991: rtcpu_vinotify_event: tstamp:13317097551 tag:CHANSEL_NOMATCH channel:0x01 frame:16578 vi_tstamp:13316994149 data:0x00000549

Moreover, we have removed the start/stop streaming mechanism from the kernel since the FPGA sends a constant images flow. Thus, if we start a v4l2 acquisition, dmesg returns tons of messages like.

tegra194-vi5 15c10000.vi: corr_err: discarding frame 0, flags: 0, err_data 0

Does this behaviour affects the acquisition? Must start/stop mechanism be added?

Thank you.

Check if embedded data output to define the correct embedded_metadata_height in device tree.

embedded_metadata_height in the device tree is equal to 0. Setting it to 1 or 2 leads to a performance worse since v4l2 is no longer able to capture frames. Moreover, our FPGA project is not sending metadata to MIPI interface.

Thank you.

Looking more into detail at the trace log, I can see that the problem might come from a wrong end frame / start frame condition. According to this guide, the error

tag:CHANSEL_NOMATCH channel:0x01 frame:22698 vi_tstamp:47309916670 data:0x00000549

corresponds to a LS problem with data type 0x2A (RAW8). It may be related to a synchronization issue.

In this post, it is suggested to adjust the pix_clk_hz and serdes_pix_clk_hz.
I have calculated the pix_clk_hz with the following equations, starting from a MIPI clock of 200MHz:
Line_rate = Mipi_clk * 2 = 200MHz * 2 = 400 Mbps (MIPI data lane is a Double Data Rate interface)
Bandwidth (total data rate) = Line_rate * N_lines = 400Mbps * 2 = 800Mbps
pic_clk_hz = Bandwidth / N_bit = 800Mbps / 8bit = 100 MHz

Is this calculation correct?

Moreover, how can I calculate the correct value of serdes_pix_clk_hz? Are there some equation to calculate it starting from pix_clk_hz?

Thank you.

You can boost the clocks to check if pix_clk_hz cause the NVCSI/VI bandwidth too low.

sudo su

echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

We have already tested with the clocks boosted but it does not change anything.
It could be an issue on cil_settletime? Do you know how we correctly calculate the serdes_pix_clk_hz? I did not find any documentation on how to calculate the correct values for DTB parameters related to MIPI timing.

Thank you.

serdes_pix_clk_hz only for some SER/DES output clocks fast than pix_clk_hz cause the NVCSI/VI acquire lower speed cause the FIFO error, don’t struggle on the calculation.

Also may not be the cil_settletime due to able receive package from the MIPI bus.

NOMATCH means an incoming pixel line is dropped, because of there’s no VI channel enabled to capture that stream;
It could be FPS issue – if frame rate is too high, and RTCPU cannot enable VI channel properly in streaming mode, could also causes nomatch

We have reduced the frame rate but the problem persists. My idea is that we have a data misalignment due to bad timing. May this concern be correct?
Moreover, our FPGA design supports 4 different modes for MIPI clock:

  • Continuous mode: the clock on the clock lane is running all the time
  • Non-Continuous mode: the clock stops and the clock lane enters Stop-State between every CSI-2 packet
  • Frame-Continuous mode: the clock lane is running during all packets of a frame, from Frame Start packet, through all the long packets with video line data, until the Frame End packet. After the Frame End packet, the clock lane stops and enters Stop-State
  • Line-Continuous mode, the clock lane keeps running between a Frame Start packet and the first long packet with video line data. The clock lanes stops after all long packets and after Frame End packet.

Which is the correct configuration for the MIPI Xavier receiver? In the device tree, discontinuous_clk=yes thus we are considering the clock in non-continuous mode.

Thank you

“discontinuous_clk=yes” is non continuous mode.

After a deeper debug, we found the issue.
The FPGA continuously sends the images via MIPI interface without the possibility of controlling the stream (e.g., no start/stop). This probably causes errors at the Xavier side. Launching the v4l2 command before resetting the FPGA resolved our issue. We are trying to integrate the I2C control to avoid this behavior.

Thank you for your support @ShaneCCC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.