IMX219 Camera driver pixel clock settings, error "PXL_SOF syncpt timeout!"

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi,

I’m working on a driver for the IMX219 camera sensor, running on the TX2, using Jetpack 4.2 on an Auvidea J120 carrier board but am currently unable to capture data.

When I execute the command

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=t.raw --verbose

I get the following output

VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG10'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
^C

and when I look at the kernel log (attached) I get lots of “PXL_SOF syncpt timeout! err = -11” errors.

[  151.961876] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  151.968378] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  152.181866] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  152.188367] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  152.402242] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  152.408784] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  152.621870] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  152.628394] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  152.841820] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11

I have attached a trace but there doesn’t seem to be any clues to help resolve the problem.

# tracer: nop
#
# entries-in-buffer/entries-written: 929/929   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/4:2-3173  [004] ....   117.441920: rtos_queue_peek_from_isr_failed: tstamp:3814008271 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   117.609846: rtos_queue_peek_from_isr_failed: tstamp:3819008275 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   117.777833: rtos_queue_peek_from_isr_failed: tstamp:3824008279 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   117.945842: rtos_queue_peek_from_isr_failed: tstamp:3829008284 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   118.113841: rtos_queue_peek_from_isr_failed: tstamp:3834008308 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   118.225837: rtos_queue_peek_from_isr_failed: tstamp:3839008306 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   118.393919: rtos_queue_peek_from_isr_failed: tstamp:3844008311 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   118.561841: rtos_queue_peek_from_isr_failed: tstamp:3849008316 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   118.729868: rtos_queue_peek_from_isr_failed: tstamp:3854008388 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   118.897848: rtos_queue_peek_from_isr_failed: tstamp:3859008338 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   119.065837: rtos_queue_peek_from_isr_failed: tstamp:3864008330 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   119.233836: rtos_queue_peek_from_isr_failed: tstamp:3869008342 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   119.345840: rtos_queue_peek_from_isr_failed: tstamp:3874008350 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   119.513839: rtos_queue_peek_from_isr_failed: tstamp:3879008351 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   119.681843: rtos_queue_peek_from_isr_failed: tstamp:3884008373 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   119.849836: rtos_queue_peek_from_isr_failed: tstamp:3889008364 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   120.017875: rtos_queue_peek_from_isr_failed: tstamp:3894008375 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   120.185915: rtos_queue_peek_from_isr_failed: tstamp:3899008377 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   120.353858: rtos_queue_peek_from_isr_failed: tstamp:3904008385 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   120.465868: rtos_queue_peek_from_isr_failed: tstamp:3909008408 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   120.633835: rtos_queue_peek_from_isr_failed: tstamp:3914008400 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   120.801837: rtos_queue_peek_from_isr_failed: tstamp:3919008410 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   120.969843: rtos_queue_peek_from_isr_failed: tstamp:3924008412 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   121.137847: rtos_queue_peek_from_isr_failed: tstamp:3929008422 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   121.305865: rtos_queue_peek_from_isr_failed: tstamp:3934008445 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   121.473850: rtos_queue_peek_from_isr_failed: tstamp:3939008440 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   121.585833: rtos_queue_peek_from_isr_failed: tstamp:3944008446 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   121.753834: rtos_queue_peek_from_isr_failed: tstamp:3949008456 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   121.921920: rtos_queue_peek_from_isr_failed: tstamp:3954008460 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   122.089838: rtos_queue_peek_from_isr_failed: tstamp:3959008480 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   122.257825: rtos_queue_peek_from_isr_failed: tstamp:3964008495 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   122.425887: rtos_queue_peek_from_isr_failed: tstamp:3969008481 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   122.593874: rtos_queue_peek_from_isr_failed: tstamp:3974008490 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   122.705832: rtos_queue_peek_from_isr_failed: tstamp:3979008492 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   122.873834: rtos_queue_peek_from_isr_failed: tstamp:3984008514 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   122.985863: rtos_queue_peek_from_isr_failed: tstamp:3987166965 queue:0x0b4b4500
          v4l_id-7100  [004] ....   150.295512: tegra_channel_open: vi-output, imx219 7-0010
          v4l_id-7100  [005] ....   150.296806: tegra_channel_set_power: imx219 7-0010 : 0x1
          v4l_id-7100  [005] ....   150.296816: camera_common_s_power: status : 0x1
          v4l_id-7100  [005] ....   150.319989: tegra_channel_set_power: 150c0000.nvcsi--1 : 0x1
          v4l_id-7100  [005] ....   150.319997: csi_s_power: enable : 0x1
          v4l_id-7100  [005] ....   150.321859: tegra_channel_close: vi-output, imx219 7-0010
          v4l_id-7100  [005] ....   150.321869: tegra_channel_set_power: imx219 7-0010 : 0x0
          v4l_id-7100  [005] ....   150.321885: camera_common_s_power: status : 0x0
          v4l_id-7100  [005] ....   150.321967: tegra_channel_set_power: 150c0000.nvcsi--1 : 0x0
          v4l_id-7100  [005] ....   150.321969: csi_s_power: enable : 0x0
        v4l2-ctl-7104  [000] ....   151.674385: tegra_channel_open: vi-output, imx219 7-0010
        v4l2-ctl-7104  [000] ....   151.681827: tegra_channel_set_power: imx219 7-0010 : 0x1
        v4l2-ctl-7104  [000] ....   151.681873: camera_common_s_power: status : 0x1
        v4l2-ctl-7104  [000] ....   151.705583: tegra_channel_set_power: 150c0000.nvcsi--1 : 0x1
        v4l2-ctl-7104  [000] ....   151.705612: csi_s_power: enable : 0x1
        v4l2-ctl-7104  [000] ....   151.726082: tegra_channel_capture_setup: vnc_id 0 W 1920 H 1080 fmt 20
 vi-output, imx2-7105  [005] ....   151.726335: tegra_channel_set_stream: enable : 0x1
 vi-output, imx2-7105  [000] ....   151.735003: tegra_channel_set_stream: 150c0000.nvcsi--1 : 0x1
 vi-output, imx2-7105  [000] ....   151.735005: csi_s_stream: enable : 0x1
 vi-output, imx2-7105  [000] ....   151.735041: tegra_channel_set_stream: imx219 7-0010 : 0x1
     kworker/4:2-3173  [004] ....   151.773834: rtos_queue_peek_from_isr_failed: tstamp:4886029444 queue:0x0b4b4500
     kworker/4:2-3173  [004] ....   151.773852: rtcpu_start: tstamp:4886030558
     kworker/4:2-3173  [004] ....   151.773863: rtos_queue_send_from_isr_failed: tstamp:4886039387 queue:0x0b4a7258
     kworker/4:2-3173  [004] ....   151.773868: rtos_queue_send_from_isr_failed: tstamp:4886039501 queue:0x0b4aad68
     kworker/4:2-3173  [004] ....   151.773873: rtos_queue_send_from_isr_failed: tstamp:4886039619 queue:0x0b4ac998

I understand that the problem could relate to the pixel clock setting, which I calculate as follows.

pixel_clk_hz = sensor data rate per lane (Mbps) * number of lanes / bits per pixel
= 755*4/10
= 302MHz

Could someone please help verify that this setting is correct? I’ve seen lots of different dtsi files for the imx219 sensor with various values applied for this parameter.

According to the IMX219 data sheet, there is a 10 bit A/D on chip converter, therefore I am assuming this means 10bits per pixel, however the other dtsi files I have seen use 12 bits.

The data rates are specified as Max.755Mbps/lane(@4lane), 912Mbps/Lane(@2lane), therefore, for 4 lanes the above value seems correct to me.

I’ve attached my dtsi file and would be grateful if someone can help validate the settings or let me know if there is something obviously wrong.

Thanks
tegra186-my-camera-config-a00.txt (9.35 KB)
trace.txt (104 KB)
dmesg.txt (3.52 KB)

2

From the trace VI/CSI didn’t receive any validate data from sensor output.
Does the vl42-ctl --list-formats-ext show the sensor mode the same with your device tree define?

3

Thanks for the suggestion.
I execute v4l2-ctl -d /dev/video0 --list-formats-ext and got the following result.

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.011s (90.000 fps)

There was in fact a mismatch between my frame count settings, however after fixing the device tree (attached - updated) there was no improvement. I still get PXL_SOF syncpt timeout! err = -11.
tegra186-my-camera-config-a00_1.txt (9.43 KB)

4

I resolved my problem, so thought I’d share in case someone else encounters the same.

I could communicate with the sensor to write settings but could not stream video with the following error being reported.
PXL_SOF syncpt timeout! err = -11

The reason was that the csi port/port index was not set correctly in the device tree.

For the Auvidea J120 carrier board there is one CSI port and this must be configured in the device tree as follows :

port-index = <4>
tegra_sinterface = “serial_e”;