Jetson Orin Nano - CSI/VI Timeout in Raw12 mode (works in raw10)

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
1

Hello all,

I’m currently facing an issue on my Jetson Orin Nano dev kit (JetPack 6.1/ L4T 36.4) trying to capture a MIPI CSI stream from 2 FPGA-based cameras. So far, I have modified original IMX477 sensor driver to have four main modes : SXGA and FHD, each declined in raw10 and raw12 pixel formats. The FPGAs emulate two IMX477 cameras and change formats with the same i2c register adresses, sending a test pattern for the CSI to read. The software changes have been made in imx477_modes_tbls.h and the DT, with some additionnal printk statements in the driver itself.

Everything looks good while capturing in both raw10 modes using v4l2-ctl (and gstreamer). But I have timeouts errors for the raw12 modes with v4l2-ctl in bypass mode. See below :

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=1024,pixelformat=RG12 --stream-mmap --verbose --set-ctrl sensor_mode=2,bypass_mode=0 
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1280/1024
	Pixel Format      : 'RG12' (12-bit Bayer RGRG/GBGB)
	Field             : None
	Bytes per Line    : 2560
	Size Image        : 2621440
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 
		VIDIOC_REQBUFS returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_STREAMON returned 0 (Success)
cap dqbuf: 0 seq:      0 bytesused: 2621440 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq:      0 bytesused: 2621440 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      1 bytesused: 2621440 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq:      2 bytesused: 2621440 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq:      3 bytesused: 2621440 ts: 0.000000 (error, ts-monotonic, ts-src-eof)

Looking further I followed these topics : https://forums.developer.nvidia.com/t/how-to-get-vi-and-csi-traces-in-jetson-nano/161648 and https://docs.nvidia.com/drive/drive-os-5.2.0.0L/drive-os/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/NvMedia/nvmedia_debugging_csi_errors.html
to see and figure out what was happening at VI and CSI level. Trace log looks like this :

<...>-9397    [001] .......  4174.687536: tegra_channel_open: vi-output, imx477 9-001a
        v4l2-ctl-9397    [000] .......  4174.703956: tegra_channel_set_power: imx477 9-001a : 0x1
        v4l2-ctl-9397    [000] .......  4174.704081: camera_common_s_power: status : 0x1
        v4l2-ctl-9397    [000] .......  4175.004458: tegra_channel_set_power: 13e00000.host1x:nvcsi@15a00000- : 0x1
        v4l2-ctl-9397    [000] .......  4175.004476: csi_s_power: enable : 0x1
        v4l2-ctl-9397    [000] .......  4175.005042: tegra_channel_capture_setup: vnc_id 0 W 1280 H 1024 fmt c4
 vi-output, imx4-9398    [000] .......  4175.006779: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
 vi-output, imx4-9398    [000] .......  4175.006796: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
 vi-output, imx4-9398    [000] .......  4175.006797: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
 vi-output, imx4-9398    [000] .......  4175.006798: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
        v4l2-ctl-9397    [001] .......  4175.006831: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-9397    [001] .......  4175.011272: tegra_channel_set_stream: 13e00000.host1x:nvcsi@15a00000- : 0x1
        v4l2-ctl-9397    [001] .......  4175.011281: csi_s_stream: enable : 0x1
        v4l2-ctl-9397    [001] .......  4175.012085: tegra_channel_set_stream: imx477 9-001a : 0x1
     kworker/2:2-8370    [002] .......  4175.027425: rtcpu_vinotify_event: tstamp:131667852423 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4213354453152 data:0x799e300010000000
     kworker/2:2-8370    [002] .......  4175.027433: rtcpu_vinotify_event: tstamp:131667852683 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4213354510112 data:0x0000000031000001
     kworker/2:2-8370    [002] .......  4175.027434: rtcpu_vinotify_event: tstamp:131667852969 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4213354528224 data:0x799e2d0010000000
     kworker/2:2-8370    [002] .......  4175.027435: rtcpu_vinotify_event: tstamp:131667853217 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:4213354548544 data:0x0000000007020001
     kworker/2:2-8370    [002] .......  4175.027435: rtcpu_vinotify_event: tstamp:131667853558 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4213354590912 data:0x0000000031000002
 vi-output, imx4-9399    [002] .......  4177.540462: tegra_channel_capture_setup: vnc_id 0 W 1280 H 1024 fmt c4
 vi-output, imx4-9398    [000] .......  4177.540825: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
 vi-output, imx4-9398    [000] .......  4177.540845: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
 vi-output, imx4-9398    [000] .......  4177.540849: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
 vi-output, imx4-9398    [000] .......  4177.540852: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:522 pid:9398 tid:9398
     kworker/2:2-8370    [002] .......  4177.551339: rtcpu_vinotify_event: tstamp:131746522436 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4215888456032 data:0x799e300010000000
     kworker/2:2-8370    [002] .......  4177.551351: rtcpu_vinotify_event: tstamp:131746522729 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4215888533504 data:0x0000000031000001
     kworker/2:2-8370    [002] .......  4177.551352: rtcpu_vinotify_event: tstamp:131746522982 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:4215888555168 data:0x0000000007020001
     kworker/2:2-8370    [002] .......  4177.607323: rtcpu_vinotify_event: tstamp:131747063292 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4215888748256 data:0x799e2d0010000000
     kworker/2:2-8370    [002] .......  4177.607337: rtcpu_vinotify_event: tstamp:131747063542 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:4215888791008 data:0x0000000031000002
[...]

Activating dev_dbg() on csi5_fops.c and vi5_fops.c did not show me much additionnal info in dmesg, as it seems FS and FE packets are not recognized by CSI :

imx477 power on
[  +0,300175] t194-nvcsi 13e00000.host1x:nvcsi@15a00000: csi5_power_on
[  +0,016461] t194-nvcsi 13e00000.host1x:nvcsi@15a00000: csi5_start_streaming: csi_pt=2, st_id=2, vc_id=0, pg_mode=0x0
[  +0,000015] t194-nvcsi 13e00000.host1x:nvcsi@15a00000: csi5_stream_set_config: stream_id=2, csi_port=2
[  +0,000003] t194-nvcsi 13e00000.host1x:nvcsi@15a00000: cil_settingtime is pulled from device
[  +0,000574] imx477: writing table
[  +0,053764] imx477: Setting mode to mode2
[  +0,000009] imx477: writing table
[  +0,017859] imx477: start streaming
[  +0,000007] imx477: writing table
[  +2,499283] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  +0,000032] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  +0,000952] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  +2,558914] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  +0,000031] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  +0,000949] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  +2,558930] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  +0,000026] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  +0,001148] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel 
[...]
[  +0,000078] imx477: stop streaming
[  +0,000002] imx477: writing table
[  +0,000195] t194-nvcsi 13e00000.host1x:nvcsi@15a00000: csi5_stop_streaming: csi_pt=2, st_id=2, vc_id=0, pg_mode=0x0
[  +0,003336] imx477 power off
[  +0,000056] t194-nvcsi 13e00000.host1x:nvcsi@15a00000: csi5_power_off

I have read the CSI Debug Counters as well using devmem, particularly events 0-8. The counters did not increase value either in raw12 capture, but did in raw10 capture. I ended up reading registers indicated by TRM, namely NVCSI_STREAM_X_ERROR_STATUS2VI_VCY_0 and NVCSI_STREAM_X_INTR_STATUS_VCY_0 where X and Y are indicated by kernel debug logs. The values read are always 0.

So my main questions are :

  • Is there another way to debug this kind of issue ? Using an oscilloscope I could confirm there were FS, Frames and FS packets in the electric signal (which was about ~20% larger), so I should be able to see something at register level. I am yet to find a trace of data handled (or at least discarded) by NVCSI.
  • Could this silent error be caused by base timeout value, or any kind of specific raw12 requirements in the DT or driver ?
  • Am I missing an important detail here that renders me unable to handle data ? If I understood correctly, bypassing the ISP should make V4L2 handle raw12 signal and it is compatible.

I really would appreciate guidance on this, as I am very new to camera dev and am quite out of clues right now. Thanks a lot.
Best regards.

notes_nv.txt (17.2 KB)

3

Does your sensor driver support raw10 and raw12 in the same driver?

v4l2-ctl --list-formats-ext

4

I believe it does :

$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'RG10' (10-bit Bayer RGRG/GBGB)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
	[1]: 'RG12' (12-bit Bayer RGRG/GBGB)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)

However it there are 4 modes per pixel format. I never paid attention as to why, so maybe it’s causing my problem ? As I said I didn’t change much in driver code.

5

Could you remove raw10 only report raw12 to verify.

Thanks

6

Thank you for your response, I have made appropriate changes to DT and driver to only support raw12

v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'RG12' (12-bit Bayer RGRG/GBGB)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)

Tried again the capture w mode 0 being previous mode 2 :

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=1024,pixelformat=RG12 --stream-mmap --verbose --set-ctrl sensor_mode=0,bypass_mode=0 
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: failed: Invalid argument
		VIDIOC_REQBUFS returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QUERYBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_QBUF returned 0 (Success)
		VIDIOC_STREAMON returned -1 (Invalid argument)

Does this error indicate that the Raw12 format is not supported by driver ?

7

Any error message from the “sudo dmesg”

8

Yes, I get this trace while launching capture with the same command :

[  363.209665] imx477 9-001a: Error updating control ranges -22
[  363.226900] imx477: writing table
[  363.283339] imx477: Setting mode to mode2
[  363.283350] imx477: writing table
[  363.301684] imx477 9-001a: Error updating control ranges
[  365.784434] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  365.784455] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  365.785226] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  365.785988] ------------[ cut here ]------------
[  365.785994] WARNING: CPU: 2 PID: 4749 at drivers/media/common/videobuf2/videobuf2-core.c:1607 vb2_start_streaming+0xe4/0x160 [videobuf2_common]
[  365.786027] Modules linked in: nv_imx477(OE) xt_MASQUERADE xt_tcpudp xt_mark nvidia_drm(OE) nvidia_modeset(OE) lzo_rle lzo_compress zram zsmalloc nvme_fabrics nvme_core ramoops reed_solomon nf_tables nfnetlink ip6table_nat ip6table_filter ip6_tables iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter r8168(O) bridge stp llc algif_hash algif_skcipher af_alg usb_f_ncm usb_f_mass_storage bnep usb_f_acm u_serial usb_f_rndis u_ether libcomposite tegra_pcie_dma_test(O) tegra_pcie_edma(O) rtl8822ce(O) snd_soc_tegra186_asrc(O) snd_soc_tegra210_admaif(O) snd_soc_tegra_pcm snd_soc_tegra210_afc(O) snd_soc_tegra186_arad(O) cfg80211 snd_soc_tegra210_mixer(O) snd_soc_tegra210_ope(O) snd_soc_tegra210_mvc(O) snd_soc_tegra210_amx(O) snd_soc_tegra210_sfc(O) snd_soc_tegra186_dspk(O) snd_soc_tegra210_adx(O) snd_soc_tegra210_i2s(O) snd_soc_tegra210_dmic(O) rtk_btusb(O) snd_soc_tegra210_ahub(O) tegra210_adma spidev btusb btrtl btintel btbcm mttcan(O) fusb301(O) nvpps(O)
[  365.786131]  can_dev snd_hda_codec_hdmi tegra234_aon(O) tegra_aconnect tegra_cactmon_mc_all(O) snd_hda_tegra at24 tegra_xudc pwm_tegra_tachometer(O) snd_hda_codec phy_tegra194_p2u snd_hda_core host1x_fence(O) pcie_tegra194 spi_tegra114 mc_hwpm(O) nvhost_vi5(O) nvhost_nvcsi_t194(O) nvhost_isp5(O) tegra_camera(O) v4l2_dv_timings nvhost_nvcsi(O) capture_ivc(O) v4l2_fwnode v4l2_async governor_userspace videobuf2_dma_contig videobuf2_memops tegra_drm(O) videobuf2_v4l2 nvvrs_pseq_rtc(O) videobuf2_common videodev tegra_camera_rtcpu(O) tegra_wmark(O) nvidia(OE) snd_soc_tegra_machine_driver(O) nvhwpm(O) snd_soc_tegra_utils(O) mc ivc_bus(O) cec snd_soc_simple_card_utils tegra_camera_platform(O) hsp_mailbox_client(O) tsecriscv(O) drm_kms_helper crct10dif_ce nvpmodel_clk_cap(O) tegra_dce(O) nvhost_capture(O) ivc_ext(O) thermal_trip_event(O) tegra234_oc_event(O) host1x_nvhost(O) nvidia_vrs_pseq(O) nvidia_p2p(O) ina3221 nvgpu(O) governor_pod_scaling(O) host1x(O) mc_utils(O) nvmap(O) nvsciipc(O) drm
[  365.786231]  fuse ip_tables x_tables ipv6 pwm_fan pwm_tegra tegra_bpmp_thermal [last unloaded: nv_imx477]
[  365.786249] CPU: 2 PID: 4749 Comm: v4l2-ctl Tainted: G        W  OE     5.15.148-tegra #3
[  365.786256] Hardware name: NVIDIA NVIDIA Jetson Orin Nano Developer Kit/Jetson, BIOS 36.4.0-gcid-37537400 09/13/2024
[  365.786260] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  365.786266] pc : vb2_start_streaming+0xe4/0x160 [videobuf2_common]
[  365.786281] lr : vb2_start_streaming+0x74/0x160 [videobuf2_common]
[  365.786292] sp : ffff800014453b00
[  365.786294] x29: ffff800014453b00 x28: ffff000080370000 x27: 0000000040045612
[  365.786302] x26: ffffc8c7933771d0 x25: 0000000000000000 x24: ffff800014453cd8
[  365.786308] x23: ffff00008911e1f8 x22: ffff800014453cd8 x21: ffff00008911e858
[  365.786313] x20: ffff00008911e870 x19: 00000000ffffffea x18: 0000000000000000
[  365.786319] x17: 0000000000000000 x16: ffffc8c7f13cc020 x15: 0000000000000000
[  365.786324] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[  365.786329] x11: 0000000000000040 x10: ffffc8c7f26b6038 x9 : ffff3739ff279000
[  365.786335] x8 : 0000000000000097 x7 : 0000000000000500 x6 : 0000000000000400
[  365.786340] x5 : ffff0001f1267ea0 x4 : 0000000000000006 x3 : 0000000000000000
[  365.786345] x2 : ffffc8c7f00b1f18 x1 : ffff000143781f00 x0 : ffff0000fb785df0
[  365.786351] Call trace:
[  365.786354]  vb2_start_streaming+0xe4/0x160 [videobuf2_common]
[  365.786365]  vb2_core_streamon+0xa4/0x1b0 [videobuf2_common]
[  365.786377]  vb2_ioctl_streamon+0x6c/0xc0 [videobuf2_v4l2]
[  365.786390]  v4l_streamon+0x44/0x60 [videodev]
[  365.786429]  __video_do_ioctl+0x190/0x400 [videodev]
[  365.786460]  video_usercopy+0x278/0x660 [videodev]
[  365.786489]  video_ioctl2+0x44/0x80 [videodev]
[  365.786516]  v4l2_ioctl+0x6c/0xa0 [videodev]
[  365.786544]  __arm64_sys_ioctl+0xb4/0x100
[  365.786554]  invoke_syscall+0x5c/0x130
[  365.786565]  el0_svc_common.constprop.0+0x64/0x110
[  365.786573]  do_el0_svc+0x74/0xa0
[  365.786580]  el0_svc+0x28/0x80
[  365.786587]  el0t_64_sync_handler+0xa4/0x130
[  365.786592]  el0t_64_sync+0x1a4/0x1a8
[  365.786597] ---[ end trace 7b322d9a070de9cb ]---

I don’t know what most of it means, but the mode is set to mode2 table which doesn’t exist.

9

Get the trace log to check.


sudo su
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 3 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace

v4l2-ctl --stream-mmap -c bypass_mode=0

cat /sys/kernel/debug/tracing/trace
10

The trace log is the same as it was with the previous DT and driver modes :

cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 41/41   #P:6
#
#                                _-------=> irqs-off
#                               / _------=> need-resched
#                              | / _-----=> need-resched-lazy
#                              || / _----=> hardirq/softirq
#                              ||| / _---=> preempt-depth
#                              |||| / _--=> preempt-lazy-depth
#                              ||||| / _-=> migrate-disable
#                              |||||| /     delay
#           TASK-PID     CPU#  |||||||  TIMESTAMP  FUNCTION
#              | |         |   |||||||      |         |
     kworker/3:2-134     [003] .......   182.080558: rtcpu_string: tstamp:7064527411 id:0x04010000 str:"VM0 deactivating."
        v4l2-ctl-4595    [000] .......   191.220315: tegra_channel_open: vi-output, imx477 9-001a
        v4l2-ctl-4595    [000] .......   191.239195: tegra_channel_set_power: imx477 9-001a : 0x1
        v4l2-ctl-4595    [000] .......   191.239321: camera_common_s_power: status : 0x1
        v4l2-ctl-4595    [000] .......   191.539476: tegra_channel_set_power: 13e00000.host1x:nvcsi@15a00000- : 0x1
        v4l2-ctl-4595    [000] .......   191.539485: csi_s_power: enable : 0x1
        v4l2-ctl-4595    [000] .......   191.540861: tegra_channel_capture_setup: vnc_id 0 W 1920 H 1080 fmt c4
 vi-output, imx4-4596    [001] .......   191.551936: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [001] .......   191.551947: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [001] .......   191.551950: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [001] .......   191.551952: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
        v4l2-ctl-4595    [003] .......   191.551971: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-4595    [003] .......   191.554452: tegra_channel_set_stream: 13e00000.host1x:nvcsi@15a00000- : 0x1
        v4l2-ctl-4595    [003] .......   191.554456: csi_s_stream: enable : 0x1
        v4l2-ctl-4595    [003] .......   191.554864: tegra_channel_set_stream: imx477 9-001a : 0x1
     kworker/3:2-134     [003] .......   191.568204: rtcpu_string: tstamp:7361694212 id:0x04010000 str:"VM0 activating."
     kworker/3:2-134     [003] .......   191.624193: rtcpu_vinotify_event: tstamp:7362425464 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:235580510336 data:0x799e300010000000
     kworker/3:2-134     [003] .......   191.624196: rtcpu_vinotify_event: tstamp:7362425733 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:235580567264 data:0x0000000031000001
     kworker/3:2-134     [003] .......   191.624196: rtcpu_vinotify_event: tstamp:7362426020 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:235580585472 data:0x799e2d0010000000
     kworker/3:2-134     [003] .......   191.624197: rtcpu_vinotify_event: tstamp:7362426270 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:235580605728 data:0x0000000007020001
     kworker/3:2-134     [003] .......   191.624197: rtcpu_vinotify_event: tstamp:7362426550 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:235580648160 data:0x0000000031000002
 vi-output, imx4-4597    [002] .......   194.273190: tegra_channel_capture_setup: vnc_id 0 W 1920 H 1080 fmt c4
 vi-output, imx4-4596    [000] .......   194.273377: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [000] .......   194.273392: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [000] .......   194.273394: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [000] .......   194.273395: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
     kworker/3:2-134     [003] .......   194.324443: rtcpu_vinotify_event: tstamp:7447062166 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:238301929824 data:0x799e300010000000
     kworker/3:2-134     [003] .......   194.324447: rtcpu_vinotify_event: tstamp:7447062459 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:238301986720 data:0x0000000031000001
     kworker/3:2-134     [003] .......   194.324448: rtcpu_vinotify_event: tstamp:7447062713 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:238302004960 data:0x799e2d0010000000
     kworker/3:2-134     [003] .......   194.324449: rtcpu_vinotify_event: tstamp:7447062992 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:238302025216 data:0x0000000007020001
     kworker/3:2-134     [003] .......   194.324450: rtcpu_vinotify_event: tstamp:7447063242 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:238302067616 data:0x0000000031000002
 vi-output, imx4-4597    [002] .......   196.833146: tegra_channel_capture_setup: vnc_id 0 W 1920 H 1080 fmt c4
 vi-output, imx4-4596    [003] .......   196.833350: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [003] .......   196.833363: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [003] .......   196.833366: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
 vi-output, imx4-4596    [003] .......   196.833368: vi_task_submit: class_id:48 ch:0 syncpt_id:26 syncpt_thresh:5 pid:4596 tid:4596
     kworker/3:2-134     [003] .......   196.848055: rtcpu_vinotify_event: tstamp:7527358110 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:240861914816 data:0x799e300010000000
     kworker/3:2-134     [003] .......   196.848056: rtcpu_vinotify_event: tstamp:7527358371 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:240861971808 data:0x0000000031000001
     kworker/3:2-134     [003] .......   196.848057: rtcpu_vinotify_event: tstamp:7527358657 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:240861989952 data:0x799e2d0010000000
     kworker/3:2-134     [003] .......   196.848057: rtcpu_vinotify_event: tstamp:7527358906 cch:0 vi:1 tag:VIFALC_ACTIONLST channel:0x23 frame:0 vi_tstamp:240862010304 data:0x0000000007020001
     kworker/3:2-134     [003] .......   196.848057: rtcpu_vinotify_event: tstamp:7527359190 cch:0 vi:1 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:240862052640 data:0x0000000031000002

V4l2 looks the same and dmesg log doesn’t print a call trace anymore but resets capture channel indefinitely once again.

As I understand, there are buffers reserved but no data getting to VI.
Any chance the initial Ridgerun driver does not in fact support raw 12 ?

11

This tell the NVCSI/VI didn’t receive any validate data from the MIPI bus.

12

OK so my initial guess is correct, but as I explained there is signal coming through the MIPI so there must be some sort of timeout event counter or CRC / malformed packet counter in memory to help me understand the issue ? Or does NVCSI just discards silently ?

I have tried NVCSI_STREAM_X_ERROR_STATUS2VI_VCY_0 and NVCSI_STREAM_X_INTR_STATUS_VCY_0 according to TRM but there is no flag raised. I have also tried with different camera hardware with same results.

13

You can have the driver report RAW12 but configuration the output as RAW10 to check the log if received data to confirm the NVCSI/VI configuration.

Thanks

14

Thank you so much for your suggestion. It worked !

In fact I declared 2 new modes with pixel format in DT (csi_pixel_bit_depth) contradicting with real output by FPGA :

  • 1st mode tells driver it’s Raw12 but in reality outputs Raw10.
  • 2nd mode tells driver it’s Raw10 but really outputs Raw12

Now 1st mode did not work but second mode allowed me to capture image with :

v4l2-ctl -d /dev/video1 --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl sensor_mode=1,bypass_mode=0 --stream-count=1 --stream-to=test12.raw

I have then checked with hexdump to confirm the image is indeed raw12.

That raises the question of why the image could not be captured using csi_pixel_bit_depth=12. The only change made is this field in DT, and the i2c value to tell sensor to output raw12.
The most logical explanation is a wrong MIPI data type code, but since then I have checked with logic analyzer and there is no mismatch : for raw10 it’s 0x2B and for raw12 it’s 0x2C.
So why is the data getting through ? And why it isn’t when I tell the DT the correct pixel format ?

This may need another topic, please let me know.
Thanks.

15

Please file new topic and tell the much detail for the problem.

Thanks