Discard additional data lines with chansel?

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi,

I’m currently working on a driver for a sony imx296 sensor connected to a TX2. This is for the 4.4.38 kernel on L4T r28.2.1. The driver already works fine with different resolutions as long as the trigger timing is defined by the sensor itself (“normal mode”). I can grab frames with v4l2-ctrl or with argus_camera in this case.

However I want to use the sensor in a hardware triggered mode to synchronize it with lighting and I also have to reduce the vertical resolution to achieve higher framerates. The sensor supports two triggered modes of which “fast trigger mode” seems more reasonable to me. When I configure the sensor to this mode, apply a trigger and request a frame on the TX2 I get an i/o error:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=RG10 --set-ctrl bypass_mode=0 --set-ctrl gain=$((48*256)) --set-ctrl coarse_time=10000 --stream-mmap --stream-count=1 --stream-to=test257-1280x720.raw --stream-skip=2
VIDIOC_DQBUF: failed: Input/output error

syslog:

Aug 14 18:20:53 tegra-ubuntu kernel: [ 2069.513766] imx296 30-001a: imx296_power_on:++
Aug 14 18:20:53 tegra-ubuntu kernel: [ 2069.557490] nvcsi 150c0000.nvcsi: csi port:0
Aug 14 18:20:53 tegra-ubuntu kernel: [ 2069.562144] nvcsi 150c0000.nvcsi: csi port:1
Aug 14 18:20:53 tegra-ubuntu kernel: [ 2069.566971] nvcsi 150c0000.nvcsi: csi port:2
Aug 14 18:20:53 tegra-ubuntu kernel: [ 2069.571622] nvcsi 150c0000.nvcsi: csi port:3
Aug 14 18:20:53 tegra-ubuntu kernel: [ 2069.576271] nvcsi 150c0000.nvcsi: csi port:4
Aug 14 18:20:53 tegra-ubuntu kernel: [ 2069.580823] nvcsi 150c0000.nvcsi: csi port:5
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.597051] nvcsi 150c0000.nvcsi: csi4_start_streaming ports index=0, lanes=1
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.604231] nvcsi 150c0000.nvcsi: csi4_stream_init
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.609054] nvcsi 150c0000.nvcsi: csi4_stream_config
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.614069] nvcsi 150c0000.nvcsi: csi4_stream_config (0) read VC0_DPCM_CTRL = 00000000
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.621992] nvcsi 150c0000.nvcsi: csi4_phy_config
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.626701] nvcsi 150c0000.nvcsi: NVCSI_CIL_CONFIG = 00000000
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.632491] nvcsi 150c0000.nvcsi: cil_settingtime is pulled from device
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.639108] nvcsi 150c0000.nvcsi: cil core clock: 204, csi clock: 102
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.645549] nvcsi 150c0000.nvcsi: cil_settingtime was autocalculated
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.651938] nvcsi 150c0000.nvcsi: csi settle time: 33, cil settle time: 25
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.688918] imx296 30-001a: imx296_configure_triggering_pins: Sensor is in Slave mode
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.697083] imx296 30-001a: imx296_configure_triggering_pins: XVS_XHS driver register: 30
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.705290] imx296 30-001a: imx296_configure_shutter: Sensor is in Lowlag Trigger Mode
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.713826] imx296_configure_shutter(): trigen=1, lowlag=1
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.786283] tegra-vi4 15700000.vi: Status:  4 channel:00 frame:0002
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.792629] tegra-vi4 15700000.vi:          timestamp sof 2097881019936 eof 2097891686592 data 0x 
Aug 14 18:20:54 tegra-ubuntu kernel: [ 2069.802320] tegra-vi4 15700000.vi:          capture_id 21 stream  0 vchan  0
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.771107] host1x 13e10000.host1x: vi-output, imx2: syncpoint id 22 (tegra-vi4) stuck waiting 22 
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.781568] ---- syncpts ----
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.785310] id 4 (disp_d) min 167 max 167 refs 1 (previous client : )
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.792991] id 5 (disp_e) min 2 max 2 refs 1 (previous client : )
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.799236] id 7 (vblank1) min 124365 max 0 refs 1 (previous client : )
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.805909] id 18 (17000000.gp10b_507) min 4556 max 4556 refs 1 (previous client : )
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.813752] id 19 (17000000.gp10b_506) min 22 max 22 refs 1 (previous client : )
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.821209] id 21 (17000000.gp10b_505) min 868 max 868 refs 1 (previous client : 17000000.gp10b_5 
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.830387] id 22 (tegra-vi4) min 21 max 22 refs 1 (previous client : tegra-vi4)
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.837841] id 23 (tegra-vi4) min 20 max 22 refs 1 (previous client : tegra-vi4)
Aug 14 18:21:00 tegra-ubuntu kernel: [ 2075.845927] 
Aug 14 18:21:04 tegra-ubuntu kernel: [ 2079.847110] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
Aug 14 18:21:05 tegra-ubuntu kernel: [ 2080.851128] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
Aug 14 18:21:05 tegra-ubuntu kernel: [ 2080.900743] nvcsi 150c0000.nvcsi: csi4_stop_streaming ports index=0, lanes=1
Aug 14 18:21:05 tegra-ubuntu kernel: [ 2080.907949] nvcsi 150c0000.nvcsi: csi4_phy_config
Aug 14 18:21:05 tegra-ubuntu kernel: [ 2080.912772] nvcsi 150c0000.nvcsi: NVCSI_CIL_CONFIG = 00000001
Aug 14 18:21:05 tegra-ubuntu kernel: [ 2080.918639] nvcsi 150c0000.nvcsi: csi4_stream_check_status
Aug 14 18:21:05 tegra-ubuntu kernel: [ 2080.924223] nvcsi 150c0000.nvcsi: csi4_cil_check_status 340
Aug 14 18:21:05 tegra-ubuntu kernel: [ 2080.936116] imx296 30-001a: imx296_power_off:++

sudo cat /sys/kernel/debug/tracing/trace

# tracer: nop
#
# entries-in-buffer/entries-written: 81/81   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/5:2-223   [005] ...1  2069.622926: rtos_queue_peek_from_isr_failed: tstamp:65553261835 queue:0x0b4a3c58
     kworker/5:2-223   [005] ...1  2069.622929: rtcpu_start: tstamp:65553263815
     kworker/5:2-223   [005] ...1  2069.778948: rtos_queue_peek_from_isr_failed: tstamp:65558262786 queue:0x0b4a3c58
     kworker/5:2-223   [005] ...1  2069.778955: rtcpu_vinotify_handle_msg: tstamp:65558776754 tag:ATOMP_FS channel:0x00 frame:2 vi_tstamp 1134266891 data:0x00000000
     kworker/5:2-223   [005] ...1  2069.778957: rtcpu_vinotify_handle_msg: tstamp:65558782284 tag:CHANSEL_PXL_SOF channel:0x00 frame:2 vi tstamp:1134272433 data:0x00000001
     kworker/5:2-223   [005] ...1  2069.778959: rtcpu_vinotify_handle_msg: tstamp:65558784669 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:  vi_tstamp:1134274818 data:0x08000000
     kworker/5:2-223   [005] ...1  2069.830911: rtcpu_vinotify_handle_msg: tstamp:65559115748 tag:CHANSEL_PXL_EOF channel:0x00 frame:2 vi tstamp:1134605637 data:0x02cf0002
     kworker/5:2-223   [005] ...1  2069.830913: rtcpu_vinotify_handle_msg: tstamp:65559115927 tag:CHANSEL_FAULT channel:0x00 frame:2 vi_t tamp:1134605766 data:0x02d00040
     kworker/5:2-223   [005] ...1  2069.830914: rtcpu_vinotify_handle_msg: tstamp:65559116584 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:  vi_tstamp:1134606650 data:0x08000000
     kworker/5:2-223   [005] ...1  2069.830914: rtcpu_vinotify_handle_msg: tstamp:65559218286 tag:CSIMUX_FRAME channel:0x00 frame:2 vi_ts amp:1134708214 data:0x00000220
     kworker/5:2-223   [005] ...1  2069.830915: rtcpu_vinotify_handle_msg: tstamp:65559218488 tag:ATOMP_FE channel:0x00 frame:2 vi_tstamp:1134708217 data:0x00000000
     kworker/5:2-223   [005] ...1  2069.935054: rtos_queue_peek_from_isr_failed: tstamp:65563263326 queue:0x0b4a3c58
     kworker/5:2-223   [005] ...1  2070.091066: rtos_queue_peek_from_isr_failed: tstamp:65568264021 queue:0x0b4a3c58
     kworker/5:2-223   [005] ...1  2070.246991: rtos_queue_peek_from_isr_failed: tstamp:65573264523 queue:0x0b4a3c58

According to https://elinux.org/Jetson_TX2_Camera_BringUp “CHANSEL_FAULT … data:0x02d00040” means: unexpected additional data (0x…40) at line 720 (0x02d0…). (active_h setting in the dtsi is “720”)

I’ve connected my (too slow) oscilloscope to the mipi-n line (through a resistor). In “normal mode” I recognize 2 lines of metadata, 10 lines of black pixels and 720 lines of the visible pixels. I also recognize two short pules, one at the start and one at the end of the frame, which lie in the same 14.8148us line length raster. I guess these are the “Frame Start Packet” and the “Frame End Packet”.

In “fast trigger mode” I see 2+10+721 Lines and what I think is the “Frame End Packet” appears delayed on the next trigger rising edge.

Can I configure the TX2s CHANSEL to just read the first 720 Lines and discard following data?

Additional Info: there’s another hardware triggered mode, which would send frames up to 1088 lines long, independent from the configured ammount of lines / ROI. The ammount of transmitted lines can be interrupted, if a second trigger occurs early enough. In this mode I can receive frames, if I configure active_h to 1088 in the dts, but the lines 721 to 1088 all contain the same data as line #720. And of course this works only, if I trigger slow enough. So in this case it would also be usefull to have the ability to discard additional lines from the mipi interface.

Thanks in advance!

imx296-fast-trigger-zoom-start.png
imx296-fast-trigger-zoom-start.png800×503 43.7 KB

imx296-fast-trigger-zoom-end.png
imx296-fast-trigger-zoom-end.png800×503 39.4 KB

2

Out of curiosity I added another resolution with the same sensor initialisation like in 1280x720, but with “active_h” setting of 721. Doesn’t work either, but now I don’t get a CHANSEL_FAULT message in the trace anymore.

/var/log/syslog:

Aug 22 16:51:39 tegra-ubuntu kernel: [89948.729225] imx296 30-001a: imx296_power_on:++
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.776647] nvcsi 150c0000.nvcsi: csi port:0
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.781865] nvcsi 150c0000.nvcsi: csi port:1
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.786756] nvcsi 150c0000.nvcsi: csi port:2
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.791702] nvcsi 150c0000.nvcsi: csi port:3
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.796797] nvcsi 150c0000.nvcsi: csi port:4
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.801719] nvcsi 150c0000.nvcsi: csi port:5
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.827626] nvcsi 150c0000.nvcsi: csi4_start_streaming ports index=0, lanes=1
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.834806] nvcsi 150c0000.nvcsi: csi4_stream_init
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.839622] nvcsi 150c0000.nvcsi: csi4_stream_config
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.844628] nvcsi 150c0000.nvcsi: csi4_stream_config (0) read VC0_DPCM_CTRL = 00000000
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.852564] nvcsi 150c0000.nvcsi: csi4_phy_config
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.857280] nvcsi 150c0000.nvcsi: NVCSI_CIL_CONFIG = 00000000
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.863048] nvcsi 150c0000.nvcsi: cil_settingtime is pulled from device
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.869665] nvcsi 150c0000.nvcsi: cil core clock: 204, csi clock: 102
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.876107] nvcsi 150c0000.nvcsi: cil_settingtime was autocalculated
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.882464] nvcsi 150c0000.nvcsi: csi settle time: 33, cil settle time: 25
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.919418] imx296 30-001a: imx296_configure_triggering_pins: Sensor is in Slave mode
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.927610] imx296 30-001a: imx296_configure_triggering_pins: XVS_XHS driver register: 30
Aug 22 16:51:39 tegra-ubuntu kernel: [89948.935859] imx296 30-001a: imx296_configure_shutter: Sensor is in Lowlag Trigger Mode
Aug 22 16:51:39 tegra-ubuntu kernel: [89949.213702] tegra-vi4 15700000.vi: Status:  2 channel:00 frame:0001
Aug 22 16:51:39 tegra-ubuntu kernel: [89949.219978] tegra-vi4 15700000.vi:          timestamp sof 89977309822688 eof 89977334482016 data 0x00000220
Aug 22 16:51:39 tegra-ubuntu kernel: [89949.229721] tegra-vi4 15700000.vi:          capture_id 47 stream  0 vchan  0
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.189294] host1x 13e10000.host1x: vi-output, imx2: syncpoint id 22 (tegra-vi4) stuck waiting 48, timeout=1000
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.199867] ---- syncpts ----
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.203374] id 4 (disp_d) min 1588 max 1588 refs 1 (previous client : )
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.210348] id 5 (disp_e) min 2 max 2 refs 1 (previous client : )
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.216769] id 7 (vblank1) min 449197 max 0 refs 1 (previous client : )
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.223462] id 18 (17000000.gp10b_507) min 21360 max 21360 refs 1 (previous client : )
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.231486] id 19 (17000000.gp10b_506) min 32 max 32 refs 1 (previous client : )
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.239000] id 21 (17000000.gp10b_505) min 21754 max 21754 refs 1 (previous client : 17000000.gp10b_505)
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.248550] id 22 (tegra-vi4) min 47 max 48 refs 1 (previous client : tegra-vi4)
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.256009] id 23 (tegra-vi4) min 46 max 48 refs 1 (previous client : tegra-vi4)
Aug 22 16:51:45 tegra-ubuntu kernel: [89955.264140] 
Aug 22 16:51:49 tegra-ubuntu kernel: [89959.265321] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
Aug 22 16:51:50 tegra-ubuntu kernel: [89960.269304] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
Aug 22 16:51:50 tegra-ubuntu kernel: [89960.327294] nvcsi 150c0000.nvcsi: csi4_stop_streaming ports index=0, lanes=1
Aug 22 16:51:50 tegra-ubuntu kernel: [89960.334576] nvcsi 150c0000.nvcsi: csi4_phy_config
Aug 22 16:51:50 tegra-ubuntu kernel: [89960.339558] nvcsi 150c0000.nvcsi: NVCSI_CIL_CONFIG = 00000001
Aug 22 16:51:50 tegra-ubuntu kernel: [89960.346255] nvcsi 150c0000.nvcsi: csi4_stream_check_status
Aug 22 16:51:50 tegra-ubuntu kernel: [89960.351953] nvcsi 150c0000.nvcsi: csi4_cil_check_status 340
Aug 22 16:51:50 tegra-ubuntu kernel: [89960.380974] imx296 30-001a: imx296_power_off:++

/sys/kernel/debug/tracing/trace:

# tracer: nop
#
# entries-in-buffer/entries-written: 80/80   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/5:0-11045 [005] ...1 89948.834793: rtos_queue_peek_from_isr_failed: tstamp:2811779585236 queue:0x0b4a3c58
     kworker/5:0-11045 [005] ...1 89948.834797: rtcpu_start: tstamp:2811779587592
     kworker/5:0-11045 [005] ...1 89948.993076: rtos_queue_peek_from_isr_failed: tstamp:2811784586159 queue:0x0b4a3c58
     kworker/5:0-11045 [005] ...1 89949.157076: rtos_queue_peek_from_isr_failed: tstamp:2811789586660 queue:0x0b4a3c58
     kworker/5:0-11045 [005] ...1 89949.209077: rtcpu_vinotify_handle_msg: tstamp:2811790926861 tag:ATOMP_FS channel:0x00 frame:1 vi_tsta p:2882314832 data:0x00000000
     kworker/5:0-11045 [005] ...1 89949.209078: rtcpu_vinotify_handle_msg: tstamp:2811790932385 tag:CHANSEL_PXL_SOF channel:0x00 frame:1  i_tstamp:2882320375 data:0x00000001
     kworker/5:0-11045 [005] ...1 89949.209079: rtcpu_vinotify_handle_msg: tstamp:2811790935464 tag:CHANSEL_LOAD_FRAMED channel:0x01 fram :1 vi_tstamp:2882323458 data:0x08000000
     kworker/5:0-11045 [005] ...1 89949.209080: rtcpu_vinotify_handle_msg: tstamp:2811791266059 tag:CHANSEL_PXL_EOF channel:0x00 frame:1  i_tstamp:2882654043 data:0x02d00002
     kworker/5:0-11045 [005] ...1 89949.261083: rtcpu_vinotify_handle_msg: tstamp:2811791703164 tag:CSIMUX_FRAME channel:0x00 frame:1 vi_ stamp:2883090979 data:0x00000220
     kworker/5:0-11045 [005] ...1 89949.261085: rtcpu_vinotify_handle_msg: tstamp:2811791703345 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:2883090983 data:0x00000000
     kworker/5:0-11045 [005] ...1 89949.313077: rtos_queue_peek_from_isr_failed: tstamp:2811794587171 queue:0x0b4a3c58
     kworker/5:0-11045 [005] ...1 89949.469183: rtos_queue_peek_from_isr_failed: tstamp:2811799587964 queue:0x0b4a3c58
     kworker/5:0-11045 [005] ...1 89949.677176: rtos_queue_peek_from_isr_failed: tstamp:2811804588470 queue:0x0b4a3c58
3

hello cbmarco,

The sensor supports two triggered modes of which “fast trigger mode” seems more reasonable to me. When I configure the sensor to this mode, apply a trigger and request a frame on the TX2 I get an i/o error.

may I have more details about what’s two triggered modes of your sensor works, how the streaming actually sending to tegra CSI?
you might also contact with your sensor vendor have configurations.
thanks