Custom Carrier TX2 CSI Video problems

IceBlackz · October 24, 2018, 8:29am

Hello all,

We’re developing a custom carrier board with 6x2-lane CSI inputs. I’ve come pretty far, getting a feeling almost beeing there. However, when trying to capture a frame with v4l2-ctl I’m encountering the following errors. I’m implementing the Sony IMX290 sensor.

Sometimes I get these:

nvidia@tegra-ubuntu:~$ v4l2-ctl -w --verbose --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=1 -d /dev/video5
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG12'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 0
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

libv4l2: error dequeuing buf: Input/output error
VIDIOC_DQBUF: failed: Input/output error
VIDIOC_STREAMOFF: ok

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tracing/trace
...
     kworker/0:1-114   [000] ...1   556.963028: rtcpu_start: tstamp:17739612326
     kworker/0:1-114   [000] ...1   557.119064: rtos_queue_peek_from_isr_failed: tstamp:17744611750 queue:0x0b4a3c58
     kworker/0:1-114   [000] ...1   557.275090: rtos_queue_peek_from_isr_failed: tstamp:17749612219 queue:0x0b4a3c58
     kworker/0:1-114   [000] ...1   557.327081: rtcpu_vinotify_handle_msg: tstamp:17750922038 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:571052389 data:0x00000000
     kworker/0:1-114   [000] ...1   557.327089: rtcpu_vinotify_handle_msg: tstamp:17750933120 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:571063484 data:0x00000001
     kworker/0:1-114   [000] ...1   557.327092: rtcpu_vinotify_handle_msg: tstamp:17750933916 tag:CHANSEL_FAULT channel:0x00 frame:8 vi_tstamp:571064288 data:0x00000100
     kworker/0:1-114   [000] ...1   557.327095: rtcpu_vinotify_handle_msg: tstamp:17750934866 tag:CHANSEL_LOAD_FRAMED channel:0x20 frame:8 vi_tstamp:571064894 data:0x08000000
     kworker/0:1-114   [000] ...1   557.327098: rtcpu_vinotify_handle_msg: tstamp:17750935034 tag:CHANSEL_FAULT_FE channel:0x20 frame:8 vi_tstamp:571064895 data:0x00000001
     kworker/0:1-114   [000] ...1   557.327101: rtcpu_vinotify_handle_msg: tstamp:17750935250 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:571064898 data:0x00000000
     kworker/0:1-114   [000] ...1   557.431170: rtos_queue_peek_from_isr_failed: tstamp:17754612724 queue:0x0b4a3c58
     kworker/0:1-114   [000] ...1   557.587085: rtos_queue_peek_from_isr_failed: tstamp:17759613485 queue:0x0b4a3c58
...

And sometimes it seems to go a bit further:

nvidia@tegra-ubuntu:~$ sudo su
[sudo] password for nvidia: 
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/tracing_on
root@tegra-ubuntu:/home/nvidia# echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
root@tegra-ubuntu:/home/nvidia# echo 2 > /sys/kernel/debug/camrtc/log-level
root@tegra-ubuntu:/home/nvidia# echo > /sys/kernel/debug/tracing/trace
root@tegra-ubuntu:/home/nvidia# exit
exit
nvidia@tegra-ubuntu:~$ v4l2-ctl -w --verbose --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=1 -d /dev/video5
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG12'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 0
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 1
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 2
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

VIDIOC_STREAMOFF: ok

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 40/40   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/0:2-185   [000] ...1    97.448401: rtos_queue_peek_from_isr_failed: tstamp:3380088899 queue:0x0b4a3c58
     kworker/0:2-185   [000] ...1    97.448409: rtcpu_start: tstamp:3380090817
     kworker/0:2-185   [000] ...1    97.552450: rtcpu_vinotify_handle_msg: tstamp:3384254280 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:3384253795 data:0x00100000
     kworker/0:2-185   [000] ...1    97.604448: rtos_queue_peek_from_isr_failed: tstamp:3385089824 queue:0x0b4a3c58
     kworker/0:2-185   [000] ...1    97.656437: rtcpu_vinotify_handle_msg: tstamp:3387364374 tag:CSIMUX_FRAME channel:0x00 frame:160 vi_tstamp:3387363914 data:0x00e00265
     kworker/0:2-185   [000] ...1    97.760496: rtcpu_vinotify_handle_msg: tstamp:3389447670 tag:CSIMUX_FRAME channel:0x00 frame:160 vi_tstamp:3389447215 data:0x00e00265
     kworker/0:2-185   [000] ...1    97.760509: rtos_queue_peek_from_isr_failed: tstamp:3390090332 queue:0x0b4a3c58
     kworker/0:2-185   [000] ...1    97.812457: rtcpu_vinotify_handle_msg: tstamp:3391530972 tag:CSIMUX_FRAME channel:0x00 frame:160 vi_tstamp:3391530516 data:0x00e00265
     kworker/0:2-185   [000] ...1    97.812466: rtcpu_vinotify_handle_msg: tstamp:3391546737 tag:ATOMP_FS channel:0x00 frame:160 vi_tstamp:3391546291 data:0x00000000
     kworker/0:2-185   [000] ...1    97.812469: rtcpu_vinotify_handle_msg: tstamp:3391557829 tag:CHANSEL_PXL_SOF channel:0x00 frame:160 vi_tstamp:3391557385 data:0x00000001
     kworker/0:2-185   [000] ...1    97.812473: rtcpu_vinotify_handle_msg: tstamp:3391558631 tag:CHANSEL_FAULT channel:0x00 frame:160 vi_tstamp:3391558189 data:0x00000100
     kworker/0:2-185   [000] ...1    97.812476: rtcpu_vinotify_handle_msg: tstamp:3391559665 tag:CHANSEL_LOAD_FRAMED channel:0x20 frame:160 vi_tstamp:3391558793 data:0x08000000
     kworker/0:2-185   [000] ...1    97.812479: rtcpu_vinotify_handle_msg: tstamp:3391559836 tag:CHANSEL_FAULT_FE channel:0x20 frame:160 vi_tstamp:3391559237 data:0x00000001
     kworker/0:2-185   [000] ...1    97.812482: rtcpu_vinotify_handle_msg: tstamp:3391560049 tag:ATOMP_FE channel:0x00 frame:160 vi_tstamp:3391559240 data:0x00000000
     kworker/0:2-185   [000] ...1    97.916447: rtos_queue_peek_from_isr_failed: tstamp:3395091030 queue:0x0b4a3c58
     kworker/0:2-185   [000] ...1    98.072446: rtos_queue_peek_from_isr_failed: tstamp:3400091534 queue:0x0b4a3c58

I’m having a hard time decoding these error messages, the only helpful thing I found this for is this post. This suggests that I might have a DTYPE_MISMATCH if I’m reading this correctly? I’m not sure what that error then means…

Things I verified:

I2C works as expected (seeing packets and seeing sensor being configured correctly)
MIPI clock speed is as expected (thus sensor is configured properly)
Sensor board has been verified on previous hardware designs (non-Jetson)

See attached files for dtsi’s, driver code and error logs

Thanks in advance for all help!
v4l2-compliance-log.txt (2.74 KB)
IO-error-output.txt (9.47 KB)
csi-mux-error-output.txt (7.41 KB)
tegra186-camera-imx290-a00-dtsi.txt (20.5 KB)
tegra186-quill-camera-imx290-a00-dtsi.txt (10.2 KB)
imx290.c (31.6 KB)

JerryChang · October 24, 2018, 9:16am

hello IceBlackz,

may I have your confirmation that you got same failure with each cameras?
I’m suspect with your mclk clock rate settings,

mclk_khz = "74250";

please refer to sensor specification to check the settings,
in generally, we choose mclk as 24Mhz, and increase the mclk_multiplier to reach required pixel clock.

pix_clk_hz = mclk_multiplier * mclk_khz

IceBlackz · October 24, 2018, 9:33am

Hello Jerry,

Thanks for your response!

Yes, also, the camera I’m currently testing with is confirmed working on the other non-jetson platform
I wasn’t quite sure what to do with the MCLK setting, as we don use the MCLK outputs of the Jetson TX2. The sensor is directly connected to a 74,25MHz oscillator.

The same for pix_clk_hz, is this the internal pixel clock from the sensor? The MIPI/CSI clock measured on an oscilloscope is 222MHz, as expected. Because CSI is DDR, this means the data rate per lane is 445gbps, totaling to 890gbps per CSI port. I’ve tried pix_clk_hz=74250000, 222500000 and 445500000 (and editing mclk_multiplier), but each give similar errors. I could also try 891000000?

JerryChang · October 24, 2018, 9:39am

hello IceBlackz,

please also have a try to boost the system performance with the following commands,
thanks

sudo ./jetson_clocks.sh
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate
echo max_rate > /sys/kernel/debug/bpmp/debug/clk/vi/rate
echo max_rate > /sys/kernel/debug/bpmp/debug/clk/isp/rate

IceBlackz · October 24, 2018, 9:52am

I had to disable the do_fan() in the script because we’ve done the cooling in an other way

nvidia@tegra-ubuntu:~$ sudo ./jetson_clocks.sh 
nvidia@tegra-ubuntu:~$ echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
-bash: /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked: Permission denied
nvidia@tegra-ubuntu:~$ sudo echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
-bash: /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked: Permission denied
nvidia@tegra-ubuntu:~$ sudo su
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
root@tegra-ubuntu:/home/nvidia# cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate
1036800000
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate
1126400000
root@tegra-ubuntu:/home/nvidia# echo 1036800000 > /sys/kernel/debug/bpmp/debug/clk/vi/rate
root@tegra-ubuntu:/home/nvidia# echo 1126400000 > /sys/kernel/debug/bpmp/debug/clk/isp/rate
root@tegra-ubuntu:/home/nvidia# exit
exit
nvidia@tegra-ubuntu:~$

First try:

nvidia@tegra-ubuntu:~$ sudo su
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/tracing_on
root@tegra-ubuntu:/home/nvidia# echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
root@tegra-ubuntu:/home/nvidia# echo 2 > /sys/kernel/debug/camrtc/log-level
root@tegra-ubuntu:/home/nvidia# echo > /sys/kernel/debug/tracing/trace
root@tegra-ubuntu:/home/nvidia# exit
exit
nvidia@tegra-ubuntu:~$ v4l2-ctl -w --verbose --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=1 -d /dev/video5
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG12'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 0
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

libv4l2: error dequeuing buf: Input/output error
VIDIOC_DQBUF: failed: Input/output error
VIDIOC_STREAMOFF: ok

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 23/23   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/3:2-2439  [003] ...1  2445.883093: rtos_queue_peek_from_isr_failed: tstamp:76768064873 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2445.883097: rtcpu_start: tstamp:76768065623
     kworker/3:2-2439  [003] ...1  2446.039086: rtos_queue_peek_from_isr_failed: tstamp:76773065745 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2446.195100: rtos_queue_peek_from_isr_failed: tstamp:76778066271 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2446.247093: rtcpu_vinotify_handle_msg: tstamp:76779171461 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:3764727010 data:0x00000000
     kworker/3:2-2439  [003] ...1  2446.247095: rtcpu_vinotify_handle_msg: tstamp:76779182541 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:3764738107 data:0x00000001
     kworker/3:2-2439  [003] ...1  2446.247096: rtcpu_vinotify_handle_msg: tstamp:76779183342 tag:CHANSEL_FAULT channel:0x00 frame:8 vi_tstamp:3764738910 data:0x00000100
     kworker/3:2-2439  [003] ...1  2446.247096: rtcpu_vinotify_handle_msg: tstamp:76779184203 tag:CHANSEL_LOAD_FRAMED channel:0x20 frame:8 vi_tstamp:3764739437 data:0x08000000
     kworker/3:2-2439  [003] ...1  2446.247097: rtcpu_vinotify_handle_msg: tstamp:76779184302 tag:CHANSEL_FAULT_FE channel:0x20 frame:8 vi_tstamp:3764739440 data:0x00000001
     kworker/3:2-2439  [003] ...1  2446.247098: rtcpu_vinotify_handle_msg: tstamp:76779184450 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:3764739441 data:0x00000000
     kworker/3:2-2439  [003] ...1  2446.351088: rtos_queue_peek_from_isr_failed: tstamp:76783066757 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2446.507090: rtos_queue_peek_from_isr_failed: tstamp:76788067264 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2446.663086: rtos_queue_peek_from_isr_failed: tstamp:76793067766 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2446.819086: rtos_queue_peek_from_isr_failed: tstamp:76798068274 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2447.027091: rtos_queue_peek_from_isr_failed: tstamp:76803068804 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2447.183087: rtos_queue_peek_from_isr_failed: tstamp:76808069287 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2447.339088: rtos_queue_peek_from_isr_failed: tstamp:76813069796 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2447.495094: rtos_queue_peek_from_isr_failed: tstamp:76818070305 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2447.651088: rtos_queue_peek_from_isr_failed: tstamp:76823070809 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2447.807087: rtos_queue_peek_from_isr_failed: tstamp:76828071318 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2447.963097: rtos_queue_peek_from_isr_failed: tstamp:76833071822 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2448.119090: rtos_queue_peek_from_isr_failed: tstamp:76838072331 queue:0x0b4a3c58
     kworker/3:2-2439  [003] ...1  2448.223113: rtos_queue_peek_from_isr_failed: tstamp:76841887983 queue:0x0b4a3c58
nvidia@tegra-ubuntu:~$

Now I also can’t get it into the state where it throws the csimux_frame and stream errors

JerryChang · October 24, 2018, 10:33am

hello IceBlackz,

you got an PIXEL_LONG_LINE failure.
could you please review your device tree settings, please confirm the active_w, active_h, and line_length properties.
thanks

IceBlackz · October 26, 2018, 10:33am

I’m not quite sure that is the problem, because even when I reduce line_length to 1937, its still produces the same errors:

#
# entries-in-buffer/entries-written: 23/23   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/3:2-308   [003] ...1   208.748541: rtos_queue_peek_from_isr_failed: tstamp:6857609630 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   208.748545: rtcpu_start: tstamp:6857610390
     kworker/3:2-308   [003] ...1   208.904536: rtos_queue_peek_from_isr_failed: tstamp:6862610504 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   209.060535: rtos_queue_peek_from_isr_failed: tstamp:6867611002 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   209.112543: rtcpu_vinotify_handle_msg: tstamp:6868723551 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:2573755847 data:0x00000000
     kworker/3:2-308   [003] ...1   209.112545: rtcpu_vinotify_handle_msg: tstamp:6868734644 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:2573766945 data:0x00000001
     kworker/3:2-308   [003] ...1   209.112546: rtcpu_vinotify_handle_msg: tstamp:6868735441 tag:CHANSEL_FAULT channel:0x00 frame:8 vi_tstamp:2573767748 data:0x00000100
     kworker/3:2-308   [003] ...1   209.112547: rtcpu_vinotify_handle_msg: tstamp:6868736301 tag:CHANSEL_LOAD_FRAMED channel:0x20 frame:8 vi_tstamp:2573768272 data:0x08000000
     kworker/3:2-308   [003] ...1   209.112548: rtcpu_vinotify_handle_msg: tstamp:6868736399 tag:CHANSEL_FAULT_FE channel:0x20 frame:8 vi_tstamp:2573768275 data:0x00000001
     kworker/3:2-308   [003] ...1   209.112548: rtcpu_vinotify_handle_msg: tstamp:6868736547 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:2573768275 data:0x00000000
     kworker/3:2-308   [003] ...1   209.216546: rtos_queue_peek_from_isr_failed: tstamp:6872611509 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   209.372536: rtos_queue_peek_from_isr_failed: tstamp:6877612017 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   209.528538: rtos_queue_peek_from_isr_failed: tstamp:6882612519 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   209.684535: rtos_queue_peek_from_isr_failed: tstamp:6887613040 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   209.892535: rtos_queue_peek_from_isr_failed: tstamp:6892613532 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   210.048536: rtos_queue_peek_from_isr_failed: tstamp:6897614040 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   210.204536: rtos_queue_peek_from_isr_failed: tstamp:6902614545 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   210.360536: rtos_queue_peek_from_isr_failed: tstamp:6907615067 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   210.516562: rtos_queue_peek_from_isr_failed: tstamp:6912615562 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   210.672538: rtos_queue_peek_from_isr_failed: tstamp:6917616067 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   210.828538: rtos_queue_peek_from_isr_failed: tstamp:6922616575 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   210.984537: rtos_queue_peek_from_isr_failed: tstamp:6927617082 queue:0x0b4a3c58
     kworker/3:2-308   [003] ...1   211.088546: rtos_queue_peek_from_isr_failed: tstamp:6931461224 queue:0x0b4a3c58

I’m quite sure that my active_w and active_h are correct, at least they aren’t bigger than those values.

IceBlackz · October 30, 2018, 10:09am

When I change my settings to the following I’m getting no more CHANSEL errors, only rtos_queue_peek:

mclk_multiplier = “6”;
pix_clk_hz = “445500000”;

nvidia@tegra-ubuntu:~$ sudo su
[sudo] password for nvidia: 
Sorry, try again.
[sudo] password for nvidia: 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# 
root@tegra-ubuntu:/home/nvidia# sudo ./jetson_clocks.sh
Can't access Fan!
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked;
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked;
root@tegra-ubuntu:/home/nvidia# cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate;
1036800000
root@tegra-ubuntu:/home/nvidia# cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate;
1126400000
root@tegra-ubuntu:/home/nvidia# echo 1036800000 > /sys/kernel/debug/bpmp/debug/clk/vi/rate;
root@tegra-ubuntu:/home/nvidia# echo 1126400000 > /sys/kernel/debug/bpmp/debug/clk/isp/rate;
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/tracing_on;
root@tegra-ubuntu:/home/nvidia# echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb;
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable;
root@tegra-ubuntu:/home/nvidia# echo 1 > /sys/kernel/debug/tracing/events/freertos/enable;
root@tegra-ubuntu:/home/nvidia# echo 2 > /sys/kernel/debug/camrtc/log-level;
root@tegra-ubuntu:/home/nvidia# echo > /sys/kernel/debug/tracing/trace;
root@tegra-ubuntu:/home/nvidia# exit
exit
nvidia@tegra-ubuntu:~$ 
nvidia@tegra-ubuntu:~$ 
nvidia@tegra-ubuntu:~$ 
nvidia@tegra-ubuntu:~$ 
nvidia@tegra-ubuntu:~$ 
nvidia@tegra-ubuntu:~$ v4l2-ctl -w --verbose --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=5 -d /dev/video0
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG12'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 0
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 1
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 2
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 3
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 3
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

 1.00 fps
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 4
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

 1.00 fps
	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 5
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

 0.99 fps
	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 6
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

 0.99 fps
VIDIOC_STREAMOFF: ok

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 61/61   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/0:0-4     [000] ...1   110.139900: rtos_queue_peek_from_isr_failed: tstamp:3775987585 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   110.139904: rtcpu_start: tstamp:3775988341
     kworker/0:0-4     [000] ...1   110.295896: rtos_queue_peek_from_isr_failed: tstamp:3780988460 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   110.451888: rtos_queue_peek_from_isr_failed: tstamp:3785988950 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   110.607890: rtos_queue_peek_from_isr_failed: tstamp:3790989452 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   110.763887: rtos_queue_peek_from_isr_failed: tstamp:3795989965 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   110.919887: rtos_queue_peek_from_isr_failed: tstamp:3800990478 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   111.075894: rtos_queue_peek_from_isr_failed: tstamp:3805990973 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   111.283887: rtos_queue_peek_from_isr_failed: tstamp:3810991485 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   111.439884: rtos_queue_peek_from_isr_failed: tstamp:3815991986 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   111.595885: rtos_queue_peek_from_isr_failed: tstamp:3820992495 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   111.751883: rtos_queue_peek_from_isr_failed: tstamp:3825993000 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   111.907881: rtos_queue_peek_from_isr_failed: tstamp:3830993508 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   112.063886: rtos_queue_peek_from_isr_failed: tstamp:3835994016 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   112.219884: rtos_queue_peek_from_isr_failed: tstamp:3840994521 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   112.375879: rtos_queue_peek_from_isr_failed: tstamp:3845995029 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   112.531889: rtos_queue_peek_from_isr_failed: tstamp:3850995533 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   112.687923: rtos_queue_peek_from_isr_failed: tstamp:3855996042 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   112.843877: rtos_queue_peek_from_isr_failed: tstamp:3860996547 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   112.999879: rtos_queue_peek_from_isr_failed: tstamp:3865997055 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   113.155878: rtos_queue_peek_from_isr_failed: tstamp:3870997565 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   113.363877: rtos_queue_peek_from_isr_failed: tstamp:3875998081 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   113.519890: rtos_queue_peek_from_isr_failed: tstamp:3880998573 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   113.675871: rtos_queue_peek_from_isr_failed: tstamp:3885999082 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   113.831874: rtos_queue_peek_from_isr_failed: tstamp:3890999591 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   113.987874: rtos_queue_peek_from_isr_failed: tstamp:3896000095 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   114.143875: rtos_queue_peek_from_isr_failed: tstamp:3901000610 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   114.299873: rtos_queue_peek_from_isr_failed: tstamp:3906001120 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   114.455873: rtos_queue_peek_from_isr_failed: tstamp:3911001617 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   114.611873: rtos_queue_peek_from_isr_failed: tstamp:3916002125 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   114.767871: rtos_queue_peek_from_isr_failed: tstamp:3921002630 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   114.923871: rtos_queue_peek_from_isr_failed: tstamp:3926003150 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   115.079878: rtos_queue_peek_from_isr_failed: tstamp:3931003655 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   115.235869: rtos_queue_peek_from_isr_failed: tstamp:3936004154 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   115.443868: rtos_queue_peek_from_isr_failed: tstamp:3941004658 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   115.599872: rtos_queue_peek_from_isr_failed: tstamp:3946005164 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   115.755863: rtos_queue_peek_from_isr_failed: tstamp:3951005669 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   115.911868: rtos_queue_peek_from_isr_failed: tstamp:3956006189 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   116.067871: rtos_queue_peek_from_isr_failed: tstamp:3961006685 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   116.223867: rtos_queue_peek_from_isr_failed: tstamp:3966007193 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   116.379865: rtos_queue_peek_from_isr_failed: tstamp:3971007699 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   116.535901: rtos_queue_peek_from_isr_failed: tstamp:3976008206 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   116.691861: rtos_queue_peek_from_isr_failed: tstamp:3981008724 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   116.847860: rtos_queue_peek_from_isr_failed: tstamp:3986009221 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   117.003865: rtos_queue_peek_from_isr_failed: tstamp:3991009725 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   117.159864: rtos_queue_peek_from_isr_failed: tstamp:3996010234 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   117.315864: rtos_queue_peek_from_isr_failed: tstamp:4001010739 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   117.523867: rtos_queue_peek_from_isr_failed: tstamp:4006011247 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   117.679863: rtos_queue_peek_from_isr_failed: tstamp:4011011755 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   117.835863: rtos_queue_peek_from_isr_failed: tstamp:4016012260 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   117.991861: rtos_queue_peek_from_isr_failed: tstamp:4021012768 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   118.147862: rtos_queue_peek_from_isr_failed: tstamp:4026013273 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   118.303859: rtos_queue_peek_from_isr_failed: tstamp:4031013781 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   118.459860: rtos_queue_peek_from_isr_failed: tstamp:4036014286 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   118.615856: rtos_queue_peek_from_isr_failed: tstamp:4041014793 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   118.771858: rtos_queue_peek_from_isr_failed: tstamp:4046015298 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   118.927857: rtos_queue_peek_from_isr_failed: tstamp:4051015819 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   119.083860: rtos_queue_peek_from_isr_failed: tstamp:4056016311 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   119.239856: rtos_queue_peek_from_isr_failed: tstamp:4061016820 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   119.395856: rtos_queue_peek_from_isr_failed: tstamp:4066017327 queue:0x0b4a3c58
     kworker/0:0-4     [000] ...1   119.499871: rtos_queue_peek_from_isr_failed: tstamp:4069095039 queue:0x0b4a3c58

Edit: nevermind, I saw jetson_clocks didnt run properly, now it gives the same errors again:

kworker/0:2-285   [000] ...1   486.889538: rtcpu_start: tstamp:15549000235
     kworker/0:2-285   [000] ...1   487.045542: rtos_queue_peek_from_isr_failed: tstamp:15554000339 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   487.201526: rtos_queue_peek_from_isr_failed: tstamp:15559000841 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   487.253532: rtcpu_vinotify_handle_msg: tstamp:15560438538 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:2675536231 data:0x00000000
     kworker/0:2-285   [000] ...1   487.253535: rtcpu_vinotify_handle_msg: tstamp:15560449634 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:2675547328 data:0x00000001
     kworker/0:2-285   [000] ...1   487.253535: rtcpu_vinotify_handle_msg: tstamp:15560450422 tag:CHANSEL_FAULT channel:0x00 frame:8 vi_tstamp:2675548132 data:0x00000100
     kworker/0:2-285   [000] ...1   487.253536: rtcpu_vinotify_handle_msg: tstamp:15560451284 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:8 vi_tstamp:2675548661 data:0x08000000
     kworker/0:2-285   [000] ...1   487.253537: rtcpu_vinotify_handle_msg: tstamp:15560451383 tag:CHANSEL_FAULT_FE channel:0x01 frame:8 vi_tstamp:2675548662 data:0x00000001
     kworker/0:2-285   [000] ...1   487.253538: rtcpu_vinotify_handle_msg: tstamp:15560451511 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:2675548662 data:0x00000000
     kworker/0:2-285   [000] ...1   487.357527: rtos_queue_peek_from_isr_failed: tstamp:15564001352 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   487.513526: rtos_queue_peek_from_isr_failed: tstamp:15569001856 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   487.669531: rtos_queue_peek_from_isr_failed: tstamp:15574002366 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   487.825526: rtos_queue_peek_from_isr_failed: tstamp:15579002870 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   487.981536: rtos_queue_peek_from_isr_failed: tstamp:15584003384 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   488.137529: rtos_queue_peek_from_isr_failed: tstamp:15589003883 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   488.293542: rtos_queue_peek_from_isr_failed: tstamp:15594004387 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   488.501530: rtos_queue_peek_from_isr_failed: tstamp:15599004897 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   488.657530: rtos_queue_peek_from_isr_failed: tstamp:15604005402 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   488.813533: rtos_queue_peek_from_isr_failed: tstamp:15609005914 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   488.969529: rtos_queue_peek_from_isr_failed: tstamp:15614006415 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   489.125533: rtos_queue_peek_from_isr_failed: tstamp:15619006923 queue:0x0b4a3c58
     kworker/0:2-285   [000] ...1   489.235841: rtos_queue_peek_from_isr_failed: tstamp:15623146380 queue:0x0b4a3c58

JerryChang · October 31, 2018, 2:22am

hello IceBlackz,

the sensor timestamp seems not correct from your v4l2 standard control verbose result.

Timestamp: 0.000000s (Monotonic, End-of-Frame)

since there are two ways to access camera sensor, VI-mode and VI-bypass mode.
besides v4l2-ctrl, could you please also try to access with gst-launch.
for example,

gst-launch-1.0 nvcamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=2592, height=1944, framerate=30/1, format=NV12' ! nvoverlaysink -ev

IceBlackz · October 31, 2018, 7:42am

Well, I’ve managed to get rid of the short/long line errors, by editing the actual V4L2 command… so you don’t need to edit your DTSI continuously, you can also just edit the V4L2-CTL command to find the correct width/height.

For future reference; Sony sensors define an effective, active and recommended pixel area. You need to take the effective pixel area, but don’t forget the dummy lines, also defined in the datasheet first pages. Also, for the IMX290, the embedded data height should be 1.

So, my resolution is apparently 1948x1097.

I now get the following results (see attached picture for (mostly black) raw file). It should show a color bar test image…

nvidia@tegra-ubuntu:~$ v4l2-ctl -w --verbose --set-fmt-video=width=1948,height=1097,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=1 -d /dev/video0 --stream-to=test.raw
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1948/1097
	Pixel Format      : 'RG12'
	Field             : None
	Bytes per Line    : 4096
	Size Image        : 4493312
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 0
	Length   : 4493312
	Bytesused: 4493312
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 1
	Length   : 4493312
	Bytesused: 4493312
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 2
	Length   : 4493312
	Bytesused: 4493312
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

VIDIOC_STREAMOFF: ok

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 26/26   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
     kworker/3:2-299   [003] ...1   295.071961: rtos_queue_peek_from_isr_failed: tstamp:9556971133 queue:0x0b4a3c58
     kworker/3:2-299   [003] ...1   295.071965: rtcpu_start: tstamp:9556971963
     kworker/3:2-299   [003] ...1   295.279954: rtos_queue_peek_from_isr_failed: tstamp:9561972018 queue:0x0b4a3c58
     kworker/3:2-299   [003] ...1   295.435963: rtos_queue_peek_from_isr_failed: tstamp:9566972516 queue:0x0b4a3c58
     kworker/3:2-299   [003] ...1   295.487960: rtcpu_vinotify_handle_msg: tstamp:9568556676 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:978621668 data:0x00000000
     kworker/3:2-299   [003] ...1   295.487962: rtcpu_vinotify_handle_msg: tstamp:9568567761 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:978632765 data:0x00000001
     kworker/3:2-299   [003] ...1   295.487963: rtcpu_vinotify_handle_msg: tstamp:9568569858 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:8 vi_tstamp:978634862 data:0x08000000
     kworker/3:2-299   [003] ...1   295.487963: rtcpu_vinotify_handle_msg: tstamp:9569583611 tag:CHANSEL_PXL_EOF channel:0x00 frame:8 vi_tstamp:979648380 data:0x04480002
     kworker/3:2-299   [003] ...1   295.487964: rtcpu_vinotify_handle_msg: tstamp:9569583717 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:979648471 data:0x00000000
     kworker/3:2-299   [003] ...1   295.487965: rtcpu_vinotify_handle_msg: tstamp:9569598309 tag:ATOMP_FS channel:0x00 frame:9 vi_tstamp:979663317 data:0x00000000
     kworker/3:2-299   [003] ...1   295.487966: rtcpu_vinotify_handle_msg: tstamp:9569609402 tag:CHANSEL_PXL_SOF channel:0x00 frame:9 vi_tstamp:979674414 data:0x00000001
     kworker/3:2-299   [003] ...1   295.487966: rtcpu_vinotify_handle_msg: tstamp:9569611294 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:9 vi_tstamp:979676305 data:0x08000000
     kworker/3:2-299   [003] ...1   295.539963: rtcpu_vinotify_handle_msg: tstamp:9570625267 tag:CHANSEL_PXL_EOF channel:0x00 frame:9 vi_tstamp:980690029 data:0x04480002
     kworker/3:2-299   [003] ...1   295.539965: rtcpu_vinotify_handle_msg: tstamp:9570625372 tag:ATOMP_FE channel:0x00 frame:9 vi_tstamp:980690120 data:0x00000000
     kworker/3:2-299   [003] ...1   295.539965: rtcpu_vinotify_handle_msg: tstamp:9570639970 tag:ATOMP_FS channel:0x00 frame:10 vi_tstamp:980704966 data:0x00000000
     kworker/3:2-299   [003] ...1   295.539966: rtcpu_vinotify_handle_msg: tstamp:9570651058 tag:CHANSEL_PXL_SOF channel:0x00 frame:10 vi_tstamp:980716063 data:0x00000001
     kworker/3:2-299   [003] ...1   295.539967: rtcpu_vinotify_handle_msg: tstamp:9570652838 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:10 vi_tstamp:980717851 data:0x08000000
     kworker/3:2-299   [003] ...1   295.591974: rtcpu_vinotify_handle_msg: tstamp:9571666918 tag:CHANSEL_PXL_EOF channel:0x00 frame:10 vi_tstamp:981731678 data:0x04480002
     kworker/3:2-299   [003] ...1   295.591976: rtcpu_vinotify_handle_msg: tstamp:9571667023 tag:ATOMP_FE channel:0x00 frame:10 vi_tstamp:981731770 data:0x00000000
     kworker/3:2-299   [003] ...1   295.591977: rtcpu_vinotify_handle_msg: tstamp:9571681617 tag:ATOMP_FS channel:0x00 frame:11 vi_tstamp:981746615 data:0x00000000
     kworker/3:2-299   [003] ...1   295.591978: rtcpu_vinotify_handle_msg: tstamp:9571692700 tag:CHANSEL_PXL_SOF channel:0x00 frame:11 vi_tstamp:981757713 data:0x00000001
     kworker/3:2-299   [003] ...1   295.591979: rtcpu_vinotify_handle_msg: tstamp:9571695529 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:11 vi_tstamp:981760537 data:0x08000000
     kworker/3:2-299   [003] ...1   295.591981: rtos_queue_peek_from_isr_failed: tstamp:9571973026 queue:0x0b4a3c58
     kworker/3:2-299   [003] ...1   295.591981: rtcpu_vinotify_handle_msg: tstamp:9572708569 tag:CHANSEL_PXL_EOF channel:0x00 frame:11 vi_tstamp:982773328 data:0x04480002
     kworker/3:2-299   [003] ...1   295.591982: rtcpu_vinotify_handle_msg: tstamp:9572708674 tag:ATOMP_FE channel:0x00 frame:11 vi_tstamp:982773419 data:0x00000000
     kworker/3:2-299   [003] ...1   295.591983: rtos_queue_peek_from_isr_failed: tstamp:9572714975 queue:0x0b4a3c58
nvidia@tegra-ubuntu:~$

@JerryChang, I’ve tried your gst-launch command, but that doesn’t work at all. Looking in my DMESG it seems to try to activate all six camera’s, while I have currently only 1 attached.

nvidia@tegra-ubuntu:~$ gst-launch-1.0 nvcamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1948, height=1097, framerate=30/1, format=NV12' ! nvoverlaysink -ev
Setting pipeline to PAUSED ...
Socket read error. Camera Daemon stopped functioning.....
gst_nvcamera_open() failed ret=0
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...
nvidia@tegra-ubuntu:~$

Is there a way it doesn’t try to access the other camera’s gpio’s?

test.raw.txt (4.29 MB)

JerryChang · October 31, 2018, 8:36am

hello IceBlackz,

it’s weird that the output resolution is odd number.
you may check the drive mode for the IMX290 spec, i think you may configure it as 1920x1080.
thanks

IceBlackz · October 31, 2018, 12:57pm

empty post

IceBlackz · October 31, 2018, 12:58pm

hello JerryChang,

Thanks! That solved quite some problems, but I also noted that as soon as I run ./jetson_clocks, I get the weird broken image like before (with only 2 or 3 lines image, the rest beeing black).

I still think some things aren okay, but at least I got my first image out of the whole chain!

When trying a longer stream, more errors appear:

nvidia@tegra-ubuntu:~$ v4l2-ctl -w  --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=90 -d /dev/video0 
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.69 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.50 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<
nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tracing/trace
...
     kworker/0:0-3335  [000] ...1   660.627906: rtos_queue_peek_from_isr_failed: tstamp:20980137147 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   660.627909: rtcpu_start: tstamp:20980137988
     kworker/0:0-3335  [000] ...1   660.835885: rtos_queue_peek_from_isr_failed: tstamp:20985138002 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   660.991886: rtos_queue_peek_from_isr_failed: tstamp:20990138507 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   661.043900: rtcpu_vinotify_handle_msg: tstamp:20991949171 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:3812079559 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.043902: rtcpu_vinotify_handle_msg: tstamp:20991960246 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:3812090654 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.043903: rtcpu_vinotify_handle_msg: tstamp:20991962312 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:8 vi_tstamp:3812092715 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.043904: rtcpu_vinotify_handle_msg: tstamp:20992960375 tag:CHANSEL_PXL_EOF channel:0x00 frame:8 vi_tstamp:3813090520 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.043905: rtcpu_vinotify_handle_msg: tstamp:20992960481 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:3813090627 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.043905: rtcpu_vinotify_handle_msg: tstamp:20992975079 tag:ATOMP_FS channel:0x00 frame:9 vi_tstamp:3813105473 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.043906: rtcpu_vinotify_handle_msg: tstamp:20992986167 tag:CHANSEL_PXL_SOF channel:0x00 frame:9 vi_tstamp:3813116567 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.043907: rtcpu_vinotify_handle_msg: tstamp:20992988409 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:9 vi_tstamp:3813118816 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.095893: rtcpu_vinotify_handle_msg: tstamp:20993986284 tag:CHANSEL_PXL_EOF channel:0x00 frame:9 vi_tstamp:3814116434 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.095894: rtcpu_vinotify_handle_msg: tstamp:20993986389 tag:ATOMP_FE channel:0x00 frame:9 vi_tstamp:3814116541 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.095895: rtcpu_vinotify_handle_msg: tstamp:20994000995 tag:ATOMP_FS channel:0x00 frame:10 vi_tstamp:3814131387 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.095896: rtcpu_vinotify_handle_msg: tstamp:20994012071 tag:CHANSEL_PXL_SOF channel:0x00 frame:10 vi_tstamp:3814142481 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.095897: rtcpu_vinotify_handle_msg: tstamp:20994013937 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:10 vi_tstamp:3814144346 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.147886: rtcpu_vinotify_handle_msg: tstamp:20995012196 tag:CHANSEL_PXL_EOF channel:0x00 frame:10 vi_tstamp:3815142347 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.147888: rtcpu_vinotify_handle_msg: tstamp:20995012301 tag:ATOMP_FE channel:0x00 frame:10 vi_tstamp:3815142454 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.147888: rtcpu_vinotify_handle_msg: tstamp:20995026907 tag:ATOMP_FS channel:0x00 frame:11 vi_tstamp:3815157299 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.147889: rtcpu_vinotify_handle_msg: tstamp:20995037995 tag:CHANSEL_PXL_SOF channel:0x00 frame:11 vi_tstamp:3815168395 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.147890: rtcpu_vinotify_handle_msg: tstamp:20995039989 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:11 vi_tstamp:3815170397 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.147891: rtos_queue_peek_from_isr_failed: tstamp:20995139001 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   661.147892: rtcpu_vinotify_handle_msg: tstamp:20996038109 tag:CHANSEL_PXL_EOF channel:0x00 frame:11 vi_tstamp:3816168259 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.147893: rtcpu_vinotify_handle_msg: tstamp:20996038216 tag:ATOMP_FE channel:0x00 frame:11 vi_tstamp:3816168367 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.147894: rtcpu_vinotify_handle_msg: tstamp:20996052819 tag:ATOMP_FS channel:0x00 frame:12 vi_tstamp:3816183213 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.147894: rtcpu_vinotify_handle_msg: tstamp:20996063912 tag:CHANSEL_PXL_SOF channel:0x00 frame:12 vi_tstamp:3816194308 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.147895: rtcpu_vinotify_handle_msg: tstamp:20996065792 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:12 vi_tstamp:3816196200 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.199908: rtcpu_vinotify_handle_msg: tstamp:20997064023 tag:CHANSEL_PXL_EOF channel:0x00 frame:12 vi_tstamp:3817194172 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.199911: rtcpu_vinotify_handle_msg: tstamp:20997064128 tag:ATOMP_FE channel:0x00 frame:12 vi_tstamp:3817194280 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.199912: rtcpu_vinotify_handle_msg: tstamp:20997078731 tag:ATOMP_FS channel:0x00 frame:13 vi_tstamp:3817209126 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.199913: rtcpu_vinotify_handle_msg: tstamp:20997089810 tag:CHANSEL_PXL_SOF channel:0x00 frame:13 vi_tstamp:3817220220 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.199914: rtcpu_vinotify_handle_msg: tstamp:20997091642 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:13 vi_tstamp:3817222049 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.199914: rtcpu_vinotify_handle_msg: tstamp:20998089953 tag:CHANSEL_PXL_EOF channel:0x00 frame:13 vi_tstamp:3818220085 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.199915: rtcpu_vinotify_handle_msg: tstamp:20998090063 tag:ATOMP_FE channel:0x00 frame:13 vi_tstamp:3818220193 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.251892: rtcpu_vinotify_handle_msg: tstamp:20998104644 tag:ATOMP_FS channel:0x00 frame:14 vi_tstamp:3818235038 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.251893: rtcpu_vinotify_handle_msg: tstamp:20998115723 tag:CHANSEL_PXL_SOF channel:0x00 frame:14 vi_tstamp:3818246133 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.251894: rtcpu_vinotify_handle_msg: tstamp:20998117773 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:14 vi_tstamp:3818248180 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.251895: rtcpu_vinotify_handle_msg: tstamp:20999115866 tag:CHANSEL_PXL_EOF channel:0x00 frame:14 vi_tstamp:3819245998 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.251896: rtcpu_vinotify_handle_msg: tstamp:20999115970 tag:ATOMP_FE channel:0x00 frame:14 vi_tstamp:3819246105 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.251897: rtcpu_vinotify_handle_msg: tstamp:20999130557 tag:ATOMP_FS channel:0x00 frame:15 vi_tstamp:3819260951 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.251897: rtcpu_vinotify_handle_msg: tstamp:20999141648 tag:CHANSEL_PXL_SOF channel:0x00 frame:15 vi_tstamp:3819272045 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.251898: rtcpu_vinotify_handle_msg: tstamp:20999143606 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:15 vi_tstamp:3819274011 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.303887: rtos_queue_peek_from_isr_failed: tstamp:21000139523 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   661.303888: rtcpu_vinotify_handle_msg: tstamp:21000141757 tag:CHANSEL_PXL_EOF channel:0x00 frame:15 vi_tstamp:3820271910 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.303889: rtcpu_vinotify_handle_msg: tstamp:21000141862 tag:ATOMP_FE channel:0x00 frame:15 vi_tstamp:3820272018 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.303890: rtcpu_vinotify_handle_msg: tstamp:21000156473 tag:ATOMP_FS channel:0x00 frame:16 vi_tstamp:3820286864 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.303890: rtcpu_vinotify_handle_msg: tstamp:21000167549 tag:CHANSEL_PXL_SOF channel:0x00 frame:16 vi_tstamp:3820297958 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.303891: rtcpu_vinotify_handle_msg: tstamp:21000169400 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:16 vi_tstamp:3820299808 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.303892: rtcpu_vinotify_handle_msg: tstamp:21001167676 tag:CHANSEL_PXL_EOF channel:0x00 frame:16 vi_tstamp:3821297823 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.303893: rtcpu_vinotify_handle_msg: tstamp:21001167781 tag:ATOMP_FE channel:0x00 frame:16 vi_tstamp:3821297930 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.303893: rtcpu_vinotify_handle_msg: tstamp:21001182380 tag:ATOMP_FS channel:0x00 frame:17 vi_tstamp:3821312777 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.303894: rtcpu_vinotify_handle_msg: tstamp:21001193474 tag:CHANSEL_PXL_SOF channel:0x00 frame:17 vi_tstamp:3821323871 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.303895: rtcpu_vinotify_handle_msg: tstamp:21001195331 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:17 vi_tstamp:3821325738 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.355890: rtcpu_vinotify_handle_msg: tstamp:21002193588 tag:CHANSEL_PXL_EOF channel:0x00 frame:17 vi_tstamp:3822323736 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.355891: rtcpu_vinotify_handle_msg: tstamp:21002193694 tag:ATOMP_FE channel:0x00 frame:17 vi_tstamp:3822323843 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.355892: rtcpu_vinotify_handle_msg: tstamp:21002208294 tag:ATOMP_FS channel:0x00 frame:18 vi_tstamp:3822338689 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.355893: rtcpu_vinotify_handle_msg: tstamp:21002219386 tag:CHANSEL_PXL_SOF channel:0x00 frame:18 vi_tstamp:3822349783 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.355893: rtcpu_vinotify_handle_msg: tstamp:21002221239 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:18 vi_tstamp:3822351647 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.407885: rtcpu_vinotify_handle_msg: tstamp:21003219502 tag:CHANSEL_PXL_EOF channel:0x00 frame:18 vi_tstamp:3823349648 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.407886: rtcpu_vinotify_handle_msg: tstamp:21003219606 tag:ATOMP_FE channel:0x00 frame:18 vi_tstamp:3823349756 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.407887: rtcpu_vinotify_handle_msg: tstamp:21003234205 tag:ATOMP_FS channel:0x00 frame:19 vi_tstamp:3823364601 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.407887: rtcpu_vinotify_handle_msg: tstamp:21003245291 tag:CHANSEL_PXL_SOF channel:0x00 frame:19 vi_tstamp:3823375696 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.407888: rtcpu_vinotify_handle_msg: tstamp:21003247045 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:19 vi_tstamp:3823377451 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.407889: rtcpu_vinotify_handle_msg: tstamp:21004245414 tag:CHANSEL_PXL_EOF channel:0x00 frame:19 vi_tstamp:3824375561 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.407889: rtcpu_vinotify_handle_msg: tstamp:21004245519 tag:ATOMP_FE channel:0x00 frame:19 vi_tstamp:3824375668 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.407890: rtcpu_vinotify_handle_msg: tstamp:21004260122 tag:ATOMP_FS channel:0x00 frame:20 vi_tstamp:3824390514 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.407891: rtcpu_vinotify_handle_msg: tstamp:21004271198 tag:CHANSEL_PXL_SOF channel:0x00 frame:20 vi_tstamp:3824401609 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.407892: rtcpu_vinotify_handle_msg: tstamp:21004273014 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:20 vi_tstamp:3824403407 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.459884: rtos_queue_peek_from_isr_failed: tstamp:21005140015 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   661.459885: rtcpu_vinotify_handle_msg: tstamp:21005271327 tag:CHANSEL_PXL_EOF channel:0x00 frame:20 vi_tstamp:3825401474 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.459886: rtcpu_vinotify_handle_msg: tstamp:21005271434 tag:ATOMP_FE channel:0x00 frame:20 vi_tstamp:3825401581 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.459886: rtcpu_vinotify_handle_msg: tstamp:21005286034 tag:ATOMP_FS channel:0x00 frame:21 vi_tstamp:3825416426 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.459887: rtcpu_vinotify_handle_msg: tstamp:21005297125 tag:CHANSEL_PXL_SOF channel:0x00 frame:21 vi_tstamp:3825427521 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.459888: rtcpu_vinotify_handle_msg: tstamp:21005298970 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:21 vi_tstamp:3825429376 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.511884: rtcpu_vinotify_handle_msg: tstamp:21006297238 tag:CHANSEL_PXL_EOF channel:0x00 frame:21 vi_tstamp:3826427386 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.511885: rtcpu_vinotify_handle_msg: tstamp:21006297343 tag:ATOMP_FE channel:0x00 frame:21 vi_tstamp:3826427493 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.511886: rtcpu_vinotify_handle_msg: tstamp:21006311945 tag:ATOMP_FS channel:0x00 frame:22 vi_tstamp:3826442339 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.511887: rtcpu_vinotify_handle_msg: tstamp:21006323026 tag:CHANSEL_PXL_SOF channel:0x00 frame:22 vi_tstamp:3826453434 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.511888: rtcpu_vinotify_handle_msg: tstamp:21006324847 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:22 vi_tstamp:3826455240 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.511888: rtcpu_vinotify_handle_msg: tstamp:21007323148 tag:CHANSEL_PXL_EOF channel:0x00 frame:22 vi_tstamp:3827453298 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.511889: rtcpu_vinotify_handle_msg: tstamp:21007323253 tag:ATOMP_FE channel:0x00 frame:22 vi_tstamp:3827453406 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.511890: rtcpu_vinotify_handle_msg: tstamp:21007337858 tag:ATOMP_FS channel:0x00 frame:23 vi_tstamp:3827468251 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.511891: rtcpu_vinotify_handle_msg: tstamp:21007348947 tag:CHANSEL_PXL_SOF channel:0x00 frame:23 vi_tstamp:3827479347 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.511891: rtcpu_vinotify_handle_msg: tstamp:21007350788 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:23 vi_tstamp:3827481194 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.563894: rtcpu_vinotify_handle_msg: tstamp:21008349064 tag:CHANSEL_PXL_EOF channel:0x00 frame:23 vi_tstamp:3828479210 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.563897: rtcpu_vinotify_handle_msg: tstamp:21008349169 tag:ATOMP_FE channel:0x00 frame:23 vi_tstamp:3828479319 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.563897: rtcpu_vinotify_handle_msg: tstamp:21008363770 tag:ATOMP_FS channel:0x00 frame:24 vi_tstamp:3828494164 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.563898: rtcpu_vinotify_handle_msg: tstamp:21008374860 tag:CHANSEL_PXL_SOF channel:0x00 frame:24 vi_tstamp:3828505259 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.563899: rtcpu_vinotify_handle_msg: tstamp:21008377083 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:24 vi_tstamp:3828507492 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.563900: rtcpu_vinotify_handle_msg: tstamp:21009374975 tag:CHANSEL_PXL_EOF channel:0x00 frame:24 vi_tstamp:3829505123 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.563900: rtcpu_vinotify_handle_msg: tstamp:21009375080 tag:ATOMP_FE channel:0x00 frame:24 vi_tstamp:3829505231 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.563901: rtcpu_vinotify_handle_msg: tstamp:21009389683 tag:ATOMP_FS channel:0x00 frame:25 vi_tstamp:3829520076 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.563902: rtcpu_vinotify_handle_msg: tstamp:21009400762 tag:CHANSEL_PXL_SOF channel:0x00 frame:25 vi_tstamp:3829531172 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.563903: rtcpu_vinotify_handle_msg: tstamp:21009402728 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:25 vi_tstamp:3829533131 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.615882: rtos_queue_peek_from_isr_failed: tstamp:21010140527 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   661.615886: rtcpu_vinotify_handle_msg: tstamp:21010400888 tag:CHANSEL_PXL_EOF channel:0x00 frame:25 vi_tstamp:3830531036 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.615887: rtcpu_vinotify_handle_msg: tstamp:21010401014 tag:ATOMP_FE channel:0x00 frame:25 vi_tstamp:3830531144 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.615887: rtcpu_vinotify_handle_msg: tstamp:21010415594 tag:ATOMP_FS channel:0x00 frame:26 vi_tstamp:3830545989 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.615888: rtcpu_vinotify_handle_msg: tstamp:21010426675 tag:CHANSEL_PXL_SOF channel:0x00 frame:26 vi_tstamp:3830557084 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.615889: rtcpu_vinotify_handle_msg: tstamp:21010428781 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:26 vi_tstamp:3830559185 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.667885: rtcpu_vinotify_handle_msg: tstamp:21011426821 tag:CHANSEL_PXL_EOF channel:0x00 frame:26 vi_tstamp:3831556949 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.667887: rtcpu_vinotify_handle_msg: tstamp:21011426926 tag:ATOMP_FE channel:0x00 frame:26 vi_tstamp:3831557056 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.667888: rtcpu_vinotify_handle_msg: tstamp:21011441507 tag:ATOMP_FS channel:0x00 frame:27 vi_tstamp:3831571902 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.667889: rtcpu_vinotify_handle_msg: tstamp:21011452599 tag:CHANSEL_PXL_SOF channel:0x00 frame:27 vi_tstamp:3831582996 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.667889: rtcpu_vinotify_handle_msg: tstamp:21011454619 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:27 vi_tstamp:3831585024 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.667890: rtcpu_vinotify_handle_msg: tstamp:21012452713 tag:CHANSEL_PXL_EOF channel:0x00 frame:27 vi_tstamp:3832582861 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.667891: rtcpu_vinotify_handle_msg: tstamp:21012452818 tag:ATOMP_FE channel:0x00 frame:27 vi_tstamp:3832582969 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.667892: rtcpu_vinotify_handle_msg: tstamp:21012467418 tag:ATOMP_FS channel:0x00 frame:28 vi_tstamp:3832597814 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.667892: rtcpu_vinotify_handle_msg: tstamp:21012478512 tag:CHANSEL_PXL_SOF channel:0x00 frame:28 vi_tstamp:3832608909 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.667893: rtcpu_vinotify_handle_msg: tstamp:21012480414 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:28 vi_tstamp:3832610823 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.719884: rtcpu_vinotify_handle_msg: tstamp:21013478625 tag:CHANSEL_PXL_EOF channel:0x00 frame:28 vi_tstamp:3833608774 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.719885: rtcpu_vinotify_handle_msg: tstamp:21013478730 tag:ATOMP_FE channel:0x00 frame:28 vi_tstamp:3833608881 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.719886: rtcpu_vinotify_handle_msg: tstamp:21013493332 tag:ATOMP_FS channel:0x00 frame:29 vi_tstamp:3833623727 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.719887: rtcpu_vinotify_handle_msg: tstamp:21013504411 tag:CHANSEL_PXL_SOF channel:0x00 frame:29 vi_tstamp:3833634821 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.719888: rtcpu_vinotify_handle_msg: tstamp:21013506299 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:29 vi_tstamp:3833636707 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.771882: rtcpu_vinotify_handle_msg: tstamp:21014504542 tag:CHANSEL_PXL_EOF channel:0x00 frame:29 vi_tstamp:3834634686 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.771883: rtcpu_vinotify_handle_msg: tstamp:21014504647 tag:ATOMP_FE channel:0x00 frame:29 vi_tstamp:3834634793 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.771884: rtcpu_vinotify_handle_msg: tstamp:21014519248 tag:ATOMP_FS channel:0x00 frame:30 vi_tstamp:3834649640 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.771885: rtcpu_vinotify_handle_msg: tstamp:21014530327 tag:CHANSEL_PXL_SOF channel:0x00 frame:30 vi_tstamp:3834660734 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.771885: rtcpu_vinotify_handle_msg: tstamp:21014532189 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:30 vi_tstamp:3834662593 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.771887: rtos_queue_peek_from_isr_failed: tstamp:21015141033 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   661.771888: rtcpu_vinotify_handle_msg: tstamp:21015530449 tag:CHANSEL_PXL_EOF channel:0x00 frame:30 vi_tstamp:3835660599 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.771888: rtcpu_vinotify_handle_msg: tstamp:21015530554 tag:ATOMP_FE channel:0x00 frame:30 vi_tstamp:3835660706 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.771889: rtcpu_vinotify_handle_msg: tstamp:21015545157 tag:ATOMP_FS channel:0x00 frame:31 vi_tstamp:3835675552 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.771890: rtcpu_vinotify_handle_msg: tstamp:21015556250 tag:CHANSEL_PXL_SOF channel:0x00 frame:31 vi_tstamp:3835686647 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.771891: rtcpu_vinotify_handle_msg: tstamp:21015558103 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:31 vi_tstamp:3835688508 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.823883: rtcpu_vinotify_handle_msg: tstamp:21016556364 tag:CHANSEL_PXL_EOF channel:0x00 frame:31 vi_tstamp:3836686512 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.823884: rtcpu_vinotify_handle_msg: tstamp:21016556469 tag:ATOMP_FE channel:0x00 frame:31 vi_tstamp:3836686619 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.823885: rtcpu_vinotify_handle_msg: tstamp:21016571069 tag:ATOMP_FS channel:0x00 frame:32 vi_tstamp:3836701465 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.823885: rtcpu_vinotify_handle_msg: tstamp:21016582162 tag:CHANSEL_PXL_SOF channel:0x00 frame:32 vi_tstamp:3836712559 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.823886: rtcpu_vinotify_handle_msg: tstamp:21016584033 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:32 vi_tstamp:3836714441 data:0x08000000
     kworker/0:0-3335  [000] ...1   661.823887: rtcpu_vinotify_handle_msg: tstamp:21017582277 tag:CHANSEL_PXL_EOF channel:0x00 frame:32 vi_tstamp:3837712425 data:0x04370002
     kworker/0:0-3335  [000] ...1   661.823888: rtcpu_vinotify_handle_msg: tstamp:21017582382 tag:ATOMP_FE channel:0x00 frame:32 vi_tstamp:3837712532 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.875881: rtcpu_vinotify_handle_msg: tstamp:21017596982 tag:ATOMP_FS channel:0x00 frame:33 vi_tstamp:3837727377 data:0x00000000
     kworker/0:0-3335  [000] ...1   661.875882: rtcpu_vinotify_handle_msg: tstamp:21017608064 tag:CHANSEL_PXL_SOF channel:0x00 frame:33 vi_tstamp:3837738472 data:0x00000001
     kworker/0:0-3335  [000] ...1   661.875883: rtcpu_vinotify_handle_msg: tstamp:21017609783 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:33 vi_tstamp:3837740179 data:0x08000000
...
tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:80 vi_tstamp:3885958263 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.435874: rtcpu_vinotify_handle_msg: tstamp:21066826109 tag:CHANSEL_PXL_EOF channel:0x00 frame:80 vi_tstamp:3886956256 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.435875: rtcpu_vinotify_handle_msg: tstamp:21066826214 tag:ATOMP_FE channel:0x00 frame:80 vi_tstamp:3886956364 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.435875: rtcpu_vinotify_handle_msg: tstamp:21066840813 tag:ATOMP_FS channel:0x00 frame:81 vi_tstamp:3886971209 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.435876: rtcpu_vinotify_handle_msg: tstamp:21066851895 tag:CHANSEL_PXL_SOF channel:0x00 frame:81 vi_tstamp:3886982304 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.435877: rtcpu_vinotify_handle_msg: tstamp:21066853715 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:81 vi_tstamp:3886984124 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.435877: rtcpu_vinotify_handle_msg: tstamp:21067852022 tag:CHANSEL_PXL_EOF channel:0x00 frame:81 vi_tstamp:3887982170 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.435878: rtcpu_vinotify_handle_msg: tstamp:21067852127 tag:ATOMP_FE channel:0x00 frame:81 vi_tstamp:3887982278 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.435879: rtcpu_vinotify_handle_msg: tstamp:21067866725 tag:ATOMP_FS channel:0x00 frame:82 vi_tstamp:3887997123 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.435880: rtcpu_vinotify_handle_msg: tstamp:21067877808 tag:CHANSEL_PXL_SOF channel:0x00 frame:82 vi_tstamp:3888008218 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.435880: rtcpu_vinotify_handle_msg: tstamp:21067879630 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:82 vi_tstamp:3888010036 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.487878: rtcpu_vinotify_handle_msg: tstamp:21068877937 tag:CHANSEL_PXL_EOF channel:0x00 frame:82 vi_tstamp:3889008084 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.487880: rtcpu_vinotify_handle_msg: tstamp:21068878040 tag:ATOMP_FE channel:0x00 frame:82 vi_tstamp:3889008191 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.487881: rtcpu_vinotify_handle_msg: tstamp:21068892644 tag:ATOMP_FS channel:0x00 frame:83 vi_tstamp:3889023037 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.487881: rtcpu_vinotify_handle_msg: tstamp:21068903734 tag:CHANSEL_PXL_SOF channel:0x00 frame:83 vi_tstamp:3889034132 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.487882: rtcpu_vinotify_handle_msg: tstamp:21068905616 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:83 vi_tstamp:3889036022 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.539886: rtcpu_vinotify_handle_msg: tstamp:21069903850 tag:CHANSEL_PXL_EOF channel:0x00 frame:83 vi_tstamp:3890033998 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.539889: rtcpu_vinotify_handle_msg: tstamp:21069903968 tag:ATOMP_FE channel:0x00 frame:83 vi_tstamp:3890034106 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.539889: rtcpu_vinotify_handle_msg: tstamp:21069918558 tag:ATOMP_FS channel:0x00 frame:84 vi_tstamp:3890048951 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.539890: rtcpu_vinotify_handle_msg: tstamp:21069929637 tag:CHANSEL_PXL_SOF channel:0x00 frame:84 vi_tstamp:3890060046 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.539891: rtcpu_vinotify_handle_msg: tstamp:21069931516 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:84 vi_tstamp:3890061918 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.539893: rtos_queue_peek_from_isr_failed: tstamp:21070146604 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   663.539894: rtcpu_vinotify_handle_msg: tstamp:21070929764 tag:CHANSEL_PXL_EOF channel:0x00 frame:84 vi_tstamp:3891059912 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.539894: rtcpu_vinotify_handle_msg: tstamp:21070929870 tag:ATOMP_FE channel:0x00 frame:84 vi_tstamp:3891060019 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.539895: rtcpu_vinotify_handle_msg: tstamp:21070944470 tag:ATOMP_FS channel:0x00 frame:85 vi_tstamp:3891074865 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.539896: rtcpu_vinotify_handle_msg: tstamp:21070955568 tag:CHANSEL_PXL_SOF channel:0x00 frame:85 vi_tstamp:3891085960 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.539896: rtcpu_vinotify_handle_msg: tstamp:21070957649 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:85 vi_tstamp:3891088048 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.591900: rtcpu_vinotify_handle_msg: tstamp:21071955680 tag:CHANSEL_PXL_EOF channel:0x00 frame:85 vi_tstamp:3892085826 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.591903: rtcpu_vinotify_handle_msg: tstamp:21071955785 tag:ATOMP_FE channel:0x00 frame:85 vi_tstamp:3892085933 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.591903: rtcpu_vinotify_handle_msg: tstamp:21071970385 tag:ATOMP_FS channel:0x00 frame:86 vi_tstamp:3892100779 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.591904: rtcpu_vinotify_handle_msg: tstamp:21071981475 tag:CHANSEL_PXL_SOF channel:0x00 frame:86 vi_tstamp:3892111874 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.591905: rtcpu_vinotify_handle_msg: tstamp:21071983259 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:86 vi_tstamp:3892113667 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.643889: rtcpu_vinotify_handle_msg: tstamp:21072981592 tag:CHANSEL_PXL_EOF channel:0x00 frame:86 vi_tstamp:3893111740 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.643891: rtcpu_vinotify_handle_msg: tstamp:21072981728 tag:ATOMP_FE channel:0x00 frame:86 vi_tstamp:3893111848 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.643892: rtcpu_vinotify_handle_msg: tstamp:21072996298 tag:ATOMP_FS channel:0x00 frame:87 vi_tstamp:3893126693 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.643893: rtcpu_vinotify_handle_msg: tstamp:21073007378 tag:CHANSEL_PXL_SOF channel:0x00 frame:87 vi_tstamp:3893137788 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.643894: rtcpu_vinotify_handle_msg: tstamp:21073009442 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:87 vi_tstamp:3893139849 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.643894: rtcpu_vinotify_handle_msg: tstamp:21074007524 tag:CHANSEL_PXL_EOF channel:0x00 frame:87 vi_tstamp:3894137654 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.643895: rtcpu_vinotify_handle_msg: tstamp:21074007630 tag:ATOMP_FE channel:0x00 frame:87 vi_tstamp:3894137761 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.643896: rtcpu_vinotify_handle_msg: tstamp:21074022210 tag:ATOMP_FS channel:0x00 frame:88 vi_tstamp:3894152607 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.643897: rtcpu_vinotify_handle_msg: tstamp:21074033294 tag:CHANSEL_PXL_SOF channel:0x00 frame:88 vi_tstamp:3894163701 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.643897: rtcpu_vinotify_handle_msg: tstamp:21074035212 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:88 vi_tstamp:3894165618 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.695873: rtcpu_vinotify_handle_msg: tstamp:21075033423 tag:CHANSEL_PXL_EOF channel:0x00 frame:88 vi_tstamp:3895163568 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.695875: rtcpu_vinotify_handle_msg: tstamp:21075033526 tag:ATOMP_FE channel:0x00 frame:88 vi_tstamp:3895163675 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.695876: rtcpu_vinotify_handle_msg: tstamp:21075048127 tag:ATOMP_FS channel:0x00 frame:89 vi_tstamp:3895178521 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.695876: rtcpu_vinotify_handle_msg: tstamp:21075059216 tag:CHANSEL_PXL_SOF channel:0x00 frame:89 vi_tstamp:3895189616 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.695877: rtcpu_vinotify_handle_msg: tstamp:21075061004 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:89 vi_tstamp:3895191413 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.695879: rtos_queue_peek_from_isr_failed: tstamp:21075147113 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   663.695880: rtcpu_vinotify_handle_msg: tstamp:21076059335 tag:CHANSEL_PXL_EOF channel:0x00 frame:89 vi_tstamp:3896189481 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.695880: rtcpu_vinotify_handle_msg: tstamp:21076059442 tag:ATOMP_FE channel:0x00 frame:89 vi_tstamp:3896189589 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.695881: rtcpu_vinotify_handle_msg: tstamp:21076074039 tag:ATOMP_FS channel:0x00 frame:90 vi_tstamp:3896204435 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.695882: rtcpu_vinotify_handle_msg: tstamp:21076085120 tag:CHANSEL_PXL_SOF channel:0x00 frame:90 vi_tstamp:3896215529 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.695883: rtcpu_vinotify_handle_msg: tstamp:21076086953 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:90 vi_tstamp:3896217357 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.747874: rtcpu_vinotify_handle_msg: tstamp:21077085249 tag:CHANSEL_PXL_EOF channel:0x00 frame:90 vi_tstamp:3897215395 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.747875: rtcpu_vinotify_handle_msg: tstamp:21077085353 tag:ATOMP_FE channel:0x00 frame:90 vi_tstamp:3897215503 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.747875: rtcpu_vinotify_handle_msg: tstamp:21077099953 tag:ATOMP_FS channel:0x00 frame:91 vi_tstamp:3897230348 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.747876: rtcpu_vinotify_handle_msg: tstamp:21077111050 tag:CHANSEL_PXL_SOF channel:0x00 frame:91 vi_tstamp:3897241443 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.747877: rtcpu_vinotify_handle_msg: tstamp:21077112899 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:91 vi_tstamp:3897243307 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.799873: rtcpu_vinotify_handle_msg: tstamp:21078111160 tag:CHANSEL_PXL_EOF channel:0x00 frame:91 vi_tstamp:3898241308 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.799875: rtcpu_vinotify_handle_msg: tstamp:21078111263 tag:ATOMP_FE channel:0x00 frame:91 vi_tstamp:3898241416 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.799876: rtcpu_vinotify_handle_msg: tstamp:21078125867 tag:ATOMP_FS channel:0x00 frame:92 vi_tstamp:3898256261 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.799876: rtcpu_vinotify_handle_msg: tstamp:21078136957 tag:CHANSEL_PXL_SOF channel:0x00 frame:92 vi_tstamp:3898267357 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.799877: rtcpu_vinotify_handle_msg: tstamp:21078138776 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:92 vi_tstamp:3898269181 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.799878: rtcpu_vinotify_handle_msg: tstamp:21079137074 tag:CHANSEL_PXL_EOF channel:0x00 frame:92 vi_tstamp:3899267221 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.799878: rtcpu_vinotify_handle_msg: tstamp:21079137179 tag:ATOMP_FE channel:0x00 frame:92 vi_tstamp:3899267329 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.799879: rtcpu_vinotify_handle_msg: tstamp:21079151780 tag:ATOMP_FS channel:0x00 frame:93 vi_tstamp:3899282175 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.799880: rtcpu_vinotify_handle_msg: tstamp:21079162860 tag:CHANSEL_PXL_SOF channel:0x00 frame:93 vi_tstamp:3899293270 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.799881: rtcpu_vinotify_handle_msg: tstamp:21079164658 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:93 vi_tstamp:3899295064 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.851872: rtos_queue_peek_from_isr_failed: tstamp:21080147619 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   663.851873: rtcpu_vinotify_handle_msg: tstamp:21080162985 tag:CHANSEL_PXL_EOF channel:0x00 frame:93 vi_tstamp:3900293134 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.851874: rtcpu_vinotify_handle_msg: tstamp:21080163089 tag:ATOMP_FE channel:0x00 frame:93 vi_tstamp:3900293242 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.851875: rtcpu_vinotify_handle_msg: tstamp:21080177694 tag:ATOMP_FS channel:0x00 frame:94 vi_tstamp:3900308088 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.851877: rtcpu_vinotify_handle_msg: tstamp:21080188773 tag:CHANSEL_PXL_SOF channel:0x00 frame:94 vi_tstamp:3900319182 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.851877: rtcpu_vinotify_handle_msg: tstamp:21080190603 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:94 vi_tstamp:3900321009 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.903873: rtcpu_vinotify_handle_msg: tstamp:21081188899 tag:CHANSEL_PXL_EOF channel:0x00 frame:94 vi_tstamp:3901319047 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.903874: rtcpu_vinotify_handle_msg: tstamp:21081189004 tag:ATOMP_FE channel:0x00 frame:94 vi_tstamp:3901319154 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.903875: rtcpu_vinotify_handle_msg: tstamp:21081203606 tag:ATOMP_FS channel:0x00 frame:95 vi_tstamp:3901334000 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.903876: rtcpu_vinotify_handle_msg: tstamp:21081214695 tag:CHANSEL_PXL_SOF channel:0x00 frame:95 vi_tstamp:3901345094 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.903876: rtcpu_vinotify_handle_msg: tstamp:21081216511 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:95 vi_tstamp:3901346917 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.903877: rtcpu_vinotify_handle_msg: tstamp:21082214810 tag:CHANSEL_PXL_EOF channel:0x00 frame:95 vi_tstamp:3902344960 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.903878: rtcpu_vinotify_handle_msg: tstamp:21082214915 tag:ATOMP_FE channel:0x00 frame:95 vi_tstamp:3902345067 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.903879: rtcpu_vinotify_handle_msg: tstamp:21082229518 tag:ATOMP_FS channel:0x00 frame:96 vi_tstamp:3902359913 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.903879: rtcpu_vinotify_handle_msg: tstamp:21082240597 tag:CHANSEL_PXL_SOF channel:0x00 frame:96 vi_tstamp:3902371007 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.903880: rtcpu_vinotify_handle_msg: tstamp:21082242458 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:96 vi_tstamp:3902372854 data:0x08000000
     kworker/0:0-3335  [000] ...1   663.955872: rtcpu_vinotify_handle_msg: tstamp:21083240723 tag:CHANSEL_PXL_EOF channel:0x00 frame:96 vi_tstamp:3903370872 data:0x04370002
     kworker/0:0-3335  [000] ...1   663.955873: rtcpu_vinotify_handle_msg: tstamp:21083240828 tag:ATOMP_FE channel:0x00 frame:96 vi_tstamp:3903370979 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.955874: rtcpu_vinotify_handle_msg: tstamp:21083255430 tag:ATOMP_FS channel:0x00 frame:97 vi_tstamp:3903385825 data:0x00000000
     kworker/0:0-3335  [000] ...1   663.955875: rtcpu_vinotify_handle_msg: tstamp:21083266535 tag:CHANSEL_PXL_SOF channel:0x00 frame:97 vi_tstamp:3903396920 data:0x00000001
     kworker/0:0-3335  [000] ...1   663.955876: rtcpu_vinotify_handle_msg: tstamp:21083268361 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:97 vi_tstamp:3903398768 data:0x08000000
     kworker/0:0-3335  [000] ...1   664.007893: rtcpu_vinotify_handle_msg: tstamp:21084266635 tag:CHANSEL_PXL_EOF channel:0x00 frame:97 vi_tstamp:3904396785 data:0x04370002
     kworker/0:0-3335  [000] ...1   664.007895: rtcpu_vinotify_handle_msg: tstamp:21084266739 tag:ATOMP_FE channel:0x00 frame:97 vi_tstamp:3904396892 data:0x00000000
     kworker/0:0-3335  [000] ...1   664.007896: rtcpu_vinotify_handle_msg: tstamp:21084281345 tag:ATOMP_FS channel:0x00 frame:98 vi_tstamp:3904411738 data:0x00000000
     kworker/0:0-3335  [000] ...1   664.007897: rtcpu_vinotify_handle_msg: tstamp:21084292433 tag:CHANSEL_PXL_SOF channel:0x00 frame:98 vi_tstamp:3904422832 data:0x00000001
     kworker/0:0-3335  [000] ...1   664.007897: rtcpu_vinotify_handle_msg: tstamp:21084294179 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:98 vi_tstamp:3904424585 data:0x08000000
     kworker/0:0-3335  [000] ...1   664.007899: rtos_queue_peek_from_isr_failed: tstamp:21085148127 queue:0x0b4a3c58
     kworker/0:0-3335  [000] ...1   664.007900: rtcpu_vinotify_handle_msg: tstamp:21085292550 tag:CHANSEL_PXL_EOF channel:0x00 frame:98 vi_tstamp:3905422697 data:0x04370002
     kworker/0:0-3335  [000] ...1   664.007901: rtcpu_vinotify_handle_msg: tstamp:21085292684 tag:ATOMP_FE channel:0x00 frame:98 vi_tstamp:3905422804 data:0x00000000
     kworker/0:0-3335  [000] ...1   664.007902: rtcpu_vinotify_handle_msg: tstamp:21085307258 tag:ATOMP_FS channel:0x00 frame:99 vi_tstamp:3905437650 data:0x00000000
     kworker/0:0-3335  [000] ...1   664.007902: rtcpu_vinotify_handle_msg: tstamp:21085318338 tag:CHANSEL_PXL_SOF channel:0x00 frame:99 vi_tstamp:3905448745 data:0x00000001
     kworker/0:0-3335  [000] ...1   664.007903: rtcpu_vinotify_handle_msg: tstamp:21085320247 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:99 vi_tstamp:3905450653 data:0x08000000
     kworker/0:0-3335  [000] ...1   664.059874: rtcpu_vinotify_handle_msg: tstamp:21086318462 tag:CHANSEL_PXL_EOF channel:0x00 frame:99 vi_tstamp:3906448610 data:0x04370002
     kworker/0:0-3335  [000] ...1   664.059877: rtcpu_vinotify_handle_msg: tstamp:21086318567 tag:ATOMP_FE channel:0x00 frame:99 vi_tstamp:3906448717 data:0x00000000
     kworker/0:0-3335  [000] ...1   664.059878: rtcpu_vinotify_handle_msg: tstamp:21086333170 tag:ATOMP_FS channel:0x00 frame:100 vi_tstamp:3906463563 data:0x00000000
     kworker/0:0-3335  [000] ...1   664.059879: rtcpu_vinotify_handle_msg: tstamp:21086344251 tag:CHANSEL_PXL_SOF channel:0x00 frame:100 vi_tstamp:3906474658 data:0x00000001
     kworker/0:0-3335  [000] ...1   664.059880: rtcpu_vinotify_handle_msg: tstamp:21086347372 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:100 vi_tstamp:3906477776 data:0x08000000
     kworker/0:0-3335  [000] ...1   664.059880: rtcpu_vinotify_handle_msg: tstamp:21087344375 tag:CHANSEL_PXL_EOF channel:0x00 frame:100 vi_tstamp:3907474523 data:0x04370002
     kworker/0:0-3335  [000] ...1   664.059881: rtcpu_vinotify_handle_msg: tstamp:21087344480 tag:ATOMP_FE channel:0x00 frame:100 vi_tstamp:3907474631 data:0x00000000
     kworker/0:0-3335  [000] ...1   664.059883: rtos_queue_peek_from_isr_failed: tstamp:21087350774 queue:0x0b4a3c58

Also, the timestamp when running --verbose is still not showing:

nvidia@tegra-ubuntu:~$ v4l2-ctl -w  --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=10 -d /dev/video0 --verbose
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG12'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 0
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 1
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 2
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 3
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 3
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 4
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 5
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 6
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 3
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 7
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 8
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 9
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 10
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

	Index    : 3
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 11
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 0.000000s (Monotonic, End-of-Frame)

VIDIOC_STREAMOFF: ok

nvidia@tegra-ubuntu:~$

Could that have anything to do with this?

test_beeld_TX2_IMX290.bmp (5.93 MB)

JerryChang · November 1, 2018, 8:01am

hello IceBlackz,

may I know which JetPack release you’re working with?
you may apply these fixes from Topic 1038067, which fix CSI Camera Capture on R28.2/TX2.
thanks

IceBlackz · November 1, 2018, 8:05am

I’m working with the kernel files from git syncing tag tegra-l4t-r28.2.1. Are those fixes already applied in this newer kernel?

JerryChang · November 1, 2018, 8:13am

hello IceBlackz,

yes, please apply those patches for your l4t/r28.2.1 release.
thanks

IceBlackz · November 1, 2018, 2:12pm

The patches indeed fixed the timestamping. However, I’m still seeing errors in the trace log:

nvidia@tegra-ubuntu:~$ v4l2-ctl -w --verbose --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=5 -d /dev/video0 --stream-to=test.raw 
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG12'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 0
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 454.513946s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done, error
	Field    : None
	Sequence : 1
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 454.546775s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 2
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 454.579604s (Monotonic, End-of-Frame)

	Index    : 3
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 3
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 454.612434s (Monotonic, End-of-Frame)

	Index    : 0
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 4
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 454.645263s (Monotonic, End-of-Frame)

	Index    : 1
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 5
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 454.678092s (Monotonic, End-of-Frame)

	Index    : 2
	Type     : Video Capture
	Flags    : mapped, done
	Field    : None
	Sequence : 6
	Length   : 4147200
	Bytesused: 4147200
	Timestamp: 454.710921s (Monotonic, End-of-Frame)

VIDIOC_STREAMOFF: ok

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 94/94   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
...
...
...
     kworker/0:2-270   [000] ...1   443.426863: rtos_queue_peek_from_isr_failed: tstamp:14191385270 queue:0x0b4a3c58
     kworker/0:2-270   [000] ...1   443.426871: rtcpu_start: tstamp:14191387018
     kworker/0:2-270   [000] ...1   443.582922: rtos_queue_peek_from_isr_failed: tstamp:14196386215 queue:0x0b4a3c58
     kworker/0:2-270   [000] ...1   443.738898: rtos_queue_peek_from_isr_failed: tstamp:14201386722 queue:0x0b4a3c58
     kworker/0:2-270   [000] ...1   443.790903: rtcpu_vinotify_handle_msg: tstamp:14203550188 tag:ATOMP_FS channel:0x00 frame:8 vi_tstamp:1318647836 data:0x00000000
     kworker/0:2-270   [000] ...1   443.842942: rtcpu_vinotify_handle_msg: tstamp:14203561266 tag:CHANSEL_PXL_SOF channel:0x00 frame:8 vi_tstamp:1318658932 data:0x00000001
     kworker/0:2-270   [000] ...1   443.842950: rtcpu_vinotify_handle_msg: tstamp:14203566416 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:8 vi_tstamp:1318664082 data:0x08000000
     kworker/0:2-270   [000] ...1   443.842953: rtcpu_vinotify_handle_msg: tstamp:14204561436 tag:CHANSEL_PXL_EOF channel:0x00 frame:8 vi_tstamp:1319658798 data:0x04370002
     kworker/0:2-270   [000] ...1   443.842956: rtcpu_vinotify_handle_msg: tstamp:14204561615 tag:ATOMP_FE channel:0x00 frame:8 vi_tstamp:1319658905 data:0x00000000
     kworker/0:2-270   [000] ...1   443.842959: rtcpu_vinotify_handle_msg: tstamp:14204576086 tag:ATOMP_FS channel:0x00 frame:9 vi_tstamp:1319673751 data:0x00000000
     kworker/0:2-270   [000] ...1   443.842962: rtcpu_vinotify_handle_msg: tstamp:14204587176 tag:CHANSEL_PXL_SOF channel:0x00 frame:9 vi_tstamp:1319684846 data:0x00000001
     kworker/0:2-270   [000] ...1   443.842966: rtcpu_vinotify_handle_msg: tstamp:14204591699 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:9 vi_tstamp:1319689365 data:0x08000000
     kworker/0:2-270   [000] ...1   443.895054: rtcpu_vinotify_handle_msg: tstamp:14205587314 tag:CHANSEL_PXL_EOF channel:0x00 frame:9 vi_tstamp:1320684713 data:0x04370002
     kworker/0:2-270   [000] ...1   443.895073: rtcpu_vinotify_handle_msg: tstamp:14205587493 tag:ATOMP_FE channel:0x00 frame:9 vi_tstamp:1320684820 data:0x00000000
     kworker/0:2-270   [000] ...1   443.895082: rtcpu_vinotify_handle_msg: tstamp:14205602000 tag:ATOMP_FS channel:0x00 frame:10 vi_tstamp:1320699666 data:0x00000000
     kworker/0:2-270   [000] ...1   443.895091: rtcpu_vinotify_handle_msg: tstamp:14205613104 tag:CHANSEL_PXL_SOF channel:0x00 frame:10 vi_tstamp:1320710761 data:0x00000001
     kworker/0:2-270   [000] ...1   443.895100: rtcpu_vinotify_handle_msg: tstamp:14205617458 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:10 vi_tstamp:1320715122 data:0x08000000
     kworker/0:2-270   [000] ...1   443.895121: rtos_queue_peek_from_isr_failed: tstamp:14206387478 queue:0x0b4a3c58
     kworker/0:2-270   [000] ...1   443.895130: rtcpu_vinotify_handle_msg: tstamp:14206613511 tag:CHANSEL_PXL_EOF channel:0x00 frame:10 vi_tstamp:1321710627 data:0x04370002
     kworker/0:2-270   [000] ...1   443.895137: rtcpu_vinotify_handle_msg: tstamp:14206614196 tag:ATOMP_FE channel:0x00 frame:10 vi_tstamp:1321710734 data:0x00000000
     kworker/0:2-270   [000] ...1   443.895148: rtcpu_vinotify_handle_msg: tstamp:14206628160 tag:ATOMP_FS channel:0x00 frame:11 vi_tstamp:1321725581 data:0x00000000
     kworker/0:2-270   [000] ...1   443.895158: rtcpu_vinotify_handle_msg: tstamp:14206639252 tag:CHANSEL_PXL_SOF channel:0x00 frame:11 vi_tstamp:1321736675 data:0x00000001
     kworker/0:2-270   [000] ...1   443.895167: rtcpu_vinotify_handle_msg: tstamp:14206646452 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:11 vi_tstamp:1321743873 data:0x08000000
     kworker/0:2-270   [000] ...1   443.946884: rtcpu_vinotify_handle_msg: tstamp:14207639121 tag:CHANSEL_PXL_EOF channel:0x00 frame:11 vi_tstamp:1322736542 data:0x04370002
     kworker/0:2-270   [000] ...1   443.946889: rtcpu_vinotify_handle_msg: tstamp:14207639300 tag:ATOMP_FE channel:0x00 frame:11 vi_tstamp:1322736650 data:0x00000000
     kworker/0:2-270   [000] ...1   443.946890: rtcpu_vinotify_handle_msg: tstamp:14207653828 tag:ATOMP_FS channel:0x00 frame:12 vi_tstamp:1322751495 data:0x00000000
     kworker/0:2-270   [000] ...1   443.946892: rtcpu_vinotify_handle_msg: tstamp:14207664916 tag:CHANSEL_PXL_SOF channel:0x00 frame:12 vi_tstamp:1322762590 data:0x00000001
     kworker/0:2-270   [000] ...1   443.946893: rtcpu_vinotify_handle_msg: tstamp:14207668709 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:12 vi_tstamp:1322766377 data:0x08000000
     kworker/0:2-270   [000] ...1   443.998898: rtcpu_vinotify_handle_msg: tstamp:14208665001 tag:CHANSEL_PXL_EOF channel:0x00 frame:12 vi_tstamp:1323762457 data:0x04370002
     kworker/0:2-270   [000] ...1   443.998903: rtcpu_vinotify_handle_msg: tstamp:14208665104 tag:ATOMP_FE channel:0x00 frame:12 vi_tstamp:1323762564 data:0x00000000
     kworker/0:2-270   [000] ...1   443.998905: rtcpu_vinotify_handle_msg: tstamp:14208679712 tag:ATOMP_FS channel:0x00 frame:13 vi_tstamp:1323777410 data:0x00000000
     kworker/0:2-270   [000] ...1   443.998906: rtcpu_vinotify_handle_msg: tstamp:14208690801 tag:CHANSEL_PXL_SOF channel:0x00 frame:13 vi_tstamp:1323788504 data:0x00000001
     kworker/0:2-270   [000] ...1   443.998908: rtcpu_vinotify_handle_msg: tstamp:14208692921 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:13 vi_tstamp:1323790623 data:0x08000000
     kworker/0:2-270   [000] ...1   443.998909: rtcpu_vinotify_handle_msg: tstamp:14209690926 tag:CHANSEL_PXL_EOF channel:0x00 frame:13 vi_tstamp:1324788371 data:0x04370002
     kworker/0:2-270   [000] ...1   443.998910: rtcpu_vinotify_handle_msg: tstamp:14209691082 tag:ATOMP_FE channel:0x00 frame:13 vi_tstamp:1324788479 data:0x00000000
     kworker/0:2-270   [000] ...1   443.998912: rtcpu_vinotify_handle_msg: tstamp:14209705625 tag:ATOMP_FS channel:0x00 frame:14 vi_tstamp:1324803325 data:0x00000000
     kworker/0:2-270   [000] ...1   443.998913: rtcpu_vinotify_handle_msg: tstamp:14209716715 tag:CHANSEL_PXL_SOF channel:0x00 frame:14 vi_tstamp:1324814420 data:0x00000001
     kworker/0:2-270   [000] ...1   443.998915: rtcpu_vinotify_handle_msg: tstamp:14209718698 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:14 vi_tstamp:1324816400 data:0x08000000
     kworker/0:2-270   [000] ...1   444.050890: rtcpu_vinotify_handle_msg: tstamp:14210716866 tag:CHANSEL_PXL_EOF channel:0x00 frame:14 vi_tstamp:1325814286 data:0x04370002
     kworker/0:2-270   [000] ...1   444.050895: rtcpu_vinotify_handle_msg: tstamp:14210717006 tag:ATOMP_FE channel:0x00 frame:14 vi_tstamp:1325814393 data:0x00000000
     kworker/0:2-270   [000] ...1   444.050897: rtcpu_vinotify_handle_msg: tstamp:14210731564 tag:ATOMP_FS channel:0x00 frame:15 vi_tstamp:1325829239 data:0x00000000
     kworker/0:2-270   [000] ...1   444.050898: rtcpu_vinotify_handle_msg: tstamp:14210742653 tag:CHANSEL_PXL_SOF channel:0x00 frame:15 vi_tstamp:1325840334 data:0x00000001
     kworker/0:2-270   [000] ...1   444.050899: rtcpu_vinotify_handle_msg: tstamp:14210745215 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:15 vi_tstamp:1325842900 data:0x08000000
     kworker/0:2-270   [000] ...1   444.050903: rtos_queue_peek_from_isr_failed: tstamp:14211387705 queue:0x0b4a3c58
     kworker/0:2-270   [000] ...1   444.102889: rtcpu_vinotify_handle_msg: tstamp:14211742784 tag:CHANSEL_PXL_EOF channel:0x00 frame:15 vi_tstamp:1326840200 data:0x04370002
     kworker/0:2-270   [000] ...1   444.102894: rtcpu_vinotify_handle_msg: tstamp:14211742926 tag:ATOMP_FE channel:0x00 frame:15 vi_tstamp:1326840309 data:0x00000000
     kworker/0:2-270   [000] ...1   444.102897: rtcpu_vinotify_handle_msg: tstamp:14211757486 tag:ATOMP_FS channel:0x00 frame:16 vi_tstamp:1326855154 data:0x00000000
     kworker/0:2-270   [000] ...1   444.102899: rtcpu_vinotify_handle_msg: tstamp:14211768571 tag:CHANSEL_PXL_SOF channel:0x00 frame:16 vi_tstamp:1326866249 data:0x00000001
     kworker/0:2-270   [000] ...1   444.102902: rtcpu_vinotify_handle_msg: tstamp:14212768694 tag:CHANSEL_PXL_EOF channel:0x00 frame:16 vi_tstamp:1327866115 data:0x04370002
     kworker/0:2-270   [000] ...1   444.102905: rtcpu_vinotify_handle_msg: tstamp:14212768835 tag:ATOMP_FE channel:0x00 frame:16 vi_tstamp:1327866222 data:0x00000000
     kworker/0:2-270   [000] ...1   444.102907: rtcpu_vinotify_handle_msg: tstamp:14212783396 tag:ATOMP_FS channel:0x00 frame:17 vi_tstamp:1327881068 data:0x00000000
     kworker/0:2-270   [000] ...1   444.102909: rtcpu_vinotify_handle_msg: tstamp:14212794487 tag:CHANSEL_PXL_SOF channel:0x00 frame:17 vi_tstamp:1327892163 data:0x00000001
     kworker/0:2-270   [000] ...1   444.154909: rtcpu_vinotify_handle_msg: tstamp:14213794633 tag:CHANSEL_PXL_EOF channel:0x00 frame:17 vi_tstamp:1328892029 data:0x04370002
     kworker/0:2-270   [000] ...1   444.154919: rtcpu_vinotify_handle_msg: tstamp:14213794812 tag:ATOMP_FE channel:0x00 frame:17 vi_tstamp:1328892137 data:0x00000000
     kworker/0:2-270   [000] ...1   444.154927: rtos_queue_peek_from_isr_failed: tstamp:14213816592 queue:0x0b4a3c58
nvidia@tegra-ubuntu:~$

I’m not sure if my line length and/or pix_clk_hz are correct, I’ll play around with them a bit (currently running multiplier x1, pix_clk_hz=74250000 and line_length = “4512”;)

Edit: these patches do seem to have fixed the issues I had with incomplete frames after running ./jetson_clocks. It’s now returning proper images!

JerryChang · November 2, 2018, 2:21am

hello IceBlackz,

it’s glad to know your raw dump works well.
you may check sensor init register tables to get the line_length settings.
also, top two bytes of CHANSEL_PXL_EOF indicate the line numbers of sensor signaling. you may check if your active_h is setting correctly.

tag:CHANSEL_PXL_EOF channel:0x00 frame:11 vi_tstamp:1322736542 data:0x04370002

in addition, could you please have a summary and update the latest status.
thanks

IceBlackz · November 5, 2018, 7:34am

0x0437 is 1079 decimal, but counting from 0 this totals to 1080, which seems correct. active_h is also set to 1080, so this should be correct. To experiment I tried changing active_h=1079, but this didn’t change anything, it seems this setting has no effect, at least on v4l-ctl command.
What I do note is that the Sony datasheet defines a NULL line between the EBD(Embedded data) line and the vertical effective OB. ~~Should I change embedded_metadata_height to 2 to counter this?~~ I tried editing, couldn’t get it to work anymore. The example files for IMX185 and IMX274 also use a height of 1, so I assume this is correct.

A quick recap/history of things changed from the start of this topic:

Custom carrier design
Kernel version: r28.2.1
Sensor used: Sony IMX290
Configuration: 6x 2-lane MIPI CSI

Through playing with the v4l-ctl command I eventually found the ‘correct’ default resolution to be 1948x1079. This could be independently tested from active_h/w settings, they seem to have no real effect. However, when capturing a frame on this resolution it was distorted heavily
On advice of Jerry I changed the sensor resolution to a crop of 1920x1080, this immediately produced correct images, but only when jetson_clocks hasn’t been run. If it was run, images would only contain 2~4 lines, the rest beeing black.
I applied the fixes to the kernel from this topic/post, this fixed the timestamps in the v4l-ctl --verbose outputs, and also fixed the problem of the incomplete frames after running jetson_clocks.
Current common-dtsi settings:

mode0 {/*mode IMX290_MODE_1920X1080_30FPS*/
					mclk_khz = "74250";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					dynamic_pixel_bit_depth = "12";
					csi_pixel_bit_depth = "12";
					mode_type = "bayer";
					pixel_phase = "rggb";

					active_w = "1920";
					active_h = "1080";
					/* pixel_t = "bayer_rggb12"; */		/* Shouldn't be neccesary, trying now */
					readout_orientation = "0";
					line_length = "2200";
					inherent_gain = "1";
					mclk_multiplier = "6";
					pix_clk_hz = "445500000";

					min_gain_val = "0"; /* dB */
					max_gain_val = "48"; /* dB */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "1.5";
					max_framerate = "30";
					min_exp_time = "30";
					max_exp_time = "660000";
					embedded_metadata_height = "1";
				};

IceBlackz · November 5, 2018, 8:48am

empty post

Topic		Replies	Views
TX2 CSI Capture with Jetpack 4.5.1 Jetson TX2 camera	45	2653	October 18, 2021
Failed to stream video via HDMI to CSI chipset Jetson AGX Xavier	6	904	May 6, 2020
MT9M021 unable to capture image using yavta Jetson TX2	13	1833	April 9, 2018
CSI Camera Custom Driver Errors Jetson TX2	9	2001	October 18, 2021
Xavier NX capture error from IMX390 Jetson Xavier NX sony-imx390	9	1293	October 18, 2021
JetsonTX2 L4T28.2.1 AuvideaJ90+LI-IMX264-MIPI-M12 Jetson TX2	5	686	October 25, 2019
TX2 CSI 4-lane with L4T r28.2 Jetson TX2	17	1840	October 18, 2021
tx2 - Camera mipi timing and PXL_SOF Jetson TX2	30	5661	October 18, 2021
[L4T R32.5.1] VI got no reply from camera processor while capturing frames Jetson Xavier NX camera	6	730	October 18, 2021
Custom carrier board camera playback failed Jetson Xavier NX camera , board-design	26	2066	October 18, 2021

Custom Carrier TX2 CSI Video problems

Related topics