Mipi capture hangs on video format change

Hi,

My use-case is an avionics DVR. The DVR has to be tolerant of input disconnect/reconnect AND video input format changes.
I am using Linux R35.5.0 (Jetpack 5.1.3) with custom carrier board.
I have modified the device tree and ov5693 driver to support camera capture from an FPGA.

This is a continuation from issue as I described in Kernel panic when mipi input is disconnected and reconnected - #3 by JerryChang
I have applied the patches as per Topic 280731 and that has resolved the initial issue of kernel panic on input connect/disconnect.
However, when the input format changes from 1080p60 to 720p60 v4l2-ctl hangs (I cannot stop it using CRL C or kill -9) and the only way to fix it is reboot Jetson.

In our normal use-case if video changes from 1080p60 to 720p60 the FPGA detects it and reports it to our software which will stop the gstreamer pipeline and restart a 720p pipeline. This fails because of 1080p pipeline hang (all CSI capture broken).

Any advice to make mipi capture tolerant to format changes and not hang until we can stop capture?

I run:
v4l2-ctl --device /dev/video1 --set-fmt-video=width=1920,height=1080,pixelformat=YUYV --set-ctrl bypass_mode=0 --stream-mmap
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
Then switch input to 720p60 to cause issue.

This is dumped on kernel log when iput switches from1080p60 to 720p60:
��BUG: camera-ip/vi5/vi5.c:415 [vi5_check_falcon_failure] “VI FALCON FAILURE: 0x40000000”
[ 2874.428045] Camera-FW on t194-rce-safe started
TCU early console enabled.
[ 2874.494764] Camera-FW on t194-rce-safe ready SHA1=571b1d9f (crt 0.744 ms, total boot 67.492 ms)
��[ 2856.166441] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 2856.166728] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 2856.168158] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[ 2856.170944] tegra194-vi5 15c10000.vi: vi_capture_release: control failed, errno 1

And trace log of tegra_rtcpu::
kworker/0:1-1454 [000] … 1092.341083: rtcpu_vinotify_event: tstamp:34747791628 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1111912806432 data:0xcd9ce50010000000
kworker/0:1-1454 [000] … 1092.341084: rtcpu_vinotify_event: tstamp:34747791786 cch:0 vi:0 tag:VIFALC_TDSTATE channel:0x23 frame:0 vi_tstamp:1111912849888 data:0x000000003100025b
kworker/0:1-1454 [000] … 1092.397060: rtcpu_nvcsi_intr: tstamp:34747896976 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397062: rtcpu_nvcsi_intr: tstamp:34747896976 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397063: rtcpu_nvcsi_intr: tstamp:34747906775 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397064: rtcpu_nvcsi_intr: tstamp:34747906775 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397082: rtcpu_nvcsi_intr: tstamp:34747916572 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397083: rtcpu_nvcsi_intr: tstamp:34747916572 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397084: rtcpu_nvcsi_intr: tstamp:34747926373 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397084: rtcpu_nvcsi_intr: tstamp:34747926373 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397085: rtcpu_nvcsi_intr: tstamp:34747936172 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397086: rtcpu_nvcsi_intr: tstamp:34747936172 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000008
kworker/0:1-1454 [000] … 1092.397087: rtcpu_string: tstamp:34747955254 id:0x04010000 str:“BUG: camera-ip/vi5/vi5.c:415 [vi5_check_falcon_f”
kworker/0:1-1454 [000] … 1092.397090: rtcpu_string: tstamp:34747955361 id:0x04010000 str:“ailure] “”
kworker/0:1-1454 [000] … 1092.397091: rtcpu_string: tstamp:34747958484 id:0x04010000 str:“VI FALCON FAILURE: 0x40000000”
kworker/0:1-1454 [000] … 1092.397092: rtcpu_string: tstamp:34747969417 id:0x04010000 str:”"
"
kworker/0:1-1454 [000] … 1092.509062: rtcpu_start: tstamp:34751995522

hello alanser,

may I have more details…
is it you kept sending frames, but it sometime output different format types to VI engine?

I don’t think current CSI/VI driver able to handle output size change in runtime.

Maybe modify the vi5_fops.c to break when capture failed to check if able to start new capture without system reboot.

Vi engine is setup and capturing1920x1080 frames. Then video input format changes on the fly from 1920x1080 to 1280x720. The VI engine is expecting 1920x1080 frames but getting 1280x720 frames until my software can stop capture and restart it as 1280x720.

hello alanser,

as mentioned in above comment #4. failure is expected since VI engine configured as 1920x1080, but, it’s getting 1280x720 frames.

why video input format changes on the fly? is it possible to output consistent formats?

We don’t handle change size on fly. Usually case is stop streaming then change the size then start streaming again.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.