Hi,
We have a solution with an Xavier running 4x Alvium CSI-MIPI cameras. We’ve had some intermittent drops in the encoding output, but this was thought to be related to encoder settings. Now we see that it maybe is connected to the CSI interface.
The problem occurs more frequently the more cameras are running, but the frame drops do not look to scale with the frame per second setup of the cameras. Would appreciate some help with decoding the trace logs, I see alot of output even though the streams are working fine.
Setup:
L4T 32.5.1
4x4lane CSI-MIPI to Alvium 507-c
Trace and clock settings according to this: Jetson/l4t/Camera BringUp - eLinux.org
Running only one camera:
kworker/0:2-1651 [000] .... 1753.176309: rtos_queue_send_from_isr_failed: tstamp:55103636059 queue:0x0bcb41f8 kworker/0:2-1651 [000] .... 1753.176310: rtos_queue_send_from_isr_failed: tstamp:55103636224 queue:0x0bcb8a60 kworker/0:2-1651 [000] .... 1753.176311: rtos_queue_send_from_isr_failed: tstamp:55103636376 queue:0x0bcba5e0 kworker/0:2-1651 [000] .... 1753.176311: rtos_queue_send_from_isr_failed: tstamp:55103636525 queue:0x0bcbb3a0 kworker/0:2-1651 [000] .... 1753.176311: rtos_queue_send_from_isr_failed: tstamp:55103636671 queue:0x0bcbc160 kworker/0:2-1651 [000] .... 1753.176316: rtcpu_vinotify_event: tstamp:55104023701 tag:FS channel:0x00 frame:1 vi_tstamp:55103822789 data:0x00000010 kworker/0:2-1651 [000] .... 1753.176316: rtcpu_vinotify_event: tstamp:55104023850 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:55103822790 data:0x00000000 kworker/0:2-1651 [000] .... 1753.176317: rtcpu_vinotify_event: tstamp:55104024013 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:55103824074 data:0x00000001 kworker/0:2-1651 [000] .... 1753.176317: rtcpu_vinotify_event: tstamp:55104024151 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:53925387072 data:0x08020ca8 kworker/0:2-1651 [000] .... 1753.176317: rtcpu_vinotify_event: tstamp:55104024313 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:53925448576 data:0x10000000 kworker/0:2-1651 [000] .... 1753.176318: rtcpu_vinotify_event: tstamp:55104024450 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:53925452448 data:0x31000ca9 kworker/0:2-1651 [000] .... 1753.176322: rtos_queue_peek_from_isr_failed: tstamp:55104703146 queue:0x0bcbcf78 kworker/0:2-1651 [000] .... 1753.176325: rtcpu_vinotify_event: tstamp:55104738934 tag:CHANSEL_PXL_EOF channel:0x23 frame:1 vi_tstamp:55104726379 data:0x08070002 kworker/0:2-1651 [000] .... 1753.176325: rtcpu_vinotify_event: tstamp:55104739079 tag:ATOMP_FRAME_DONE channel:0x23 frame:1 vi_tstamp:55104726397 data:0x00000000 kworker/0:2-1651 [000] .... 1753.176325: rtcpu_vinotify_event: tstamp:55104739241 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:53954261376 data:0x02020ca8 kworker/0:2-1651 [000] .... 1753.176326: rtcpu_vinotify_event: tstamp:55104739380 tag:FE channel:0x00 frame:1 vi_tstamp:55104738011 data:0x00000020 kworker/0:2-1651 [000] .... 1753.176326: rtcpu_vinotify_event: tstamp:55104739538 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:55104738011 data:0x00000000 kworker/0:2-1651 [000] .... 1753.176326: rtcpu_vinotify_event: tstamp:55104739673 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:53954634272 data:0x00020ca8 kworker/0:2-1651 [000] .... 1753.176327: rtcpu_vinotify_event: tstamp:55105025319 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:53954637632 data:0x07020ca9 ALVIUM 1800 C-5-4641 [003] .... 1753.218288: tegra_channel_capture_frame: sof:1763.384867200 ALVIUM 1800 C-5-4641 [003] .... 1753.218290: tegra_channel_capture_frame: eof:1763.414113184 kworker/0:2-1651 [000] .... 1753.236542: rtos_queue_send_from_isr_failed: tstamp:55105657483 queue:0x0bcb41f8 kworker/0:2-1651 [000] .... 1753.236544: rtos_queue_send_from_isr_failed: tstamp:55105657639 queue:0x0bcb8a60 kworker/0:2-1651 [000] .... 1753.236544: rtos_queue_send_from_isr_failed: tstamp:55105657793 queue:0x0bcba5e0 kworker/0:2-1651 [000] .... 1753.236545: rtos_queue_send_from_isr_failed: tstamp:55105657939 queue:0x0bcbb3a0 kworker/0:2-1651 [000] .... 1753.236545: rtos_queue_send_from_isr_failed: tstamp:55105658086 queue:0x0bcbc160 kworker/0:2-1651 [000] .... 1753.236553: rtcpu_vinotify_event: tstamp:55105964307 tag:FS channel:0x00 frame:1 vi_tstamp:55105775816 data:0x00000010 kworker/0:2-1651 [000] .... 1753.236558: rtcpu_vinotify_event: tstamp:55105964473 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:55105775816 data:0x00000000 kworker/0:2-1651 [000] .... 1753.236558: rtcpu_vinotify_event: tstamp:55105964615 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:55105777100 data:0x00000001 kworker/0:2-1651 [000] .... 1753.236559: rtcpu_vinotify_event: tstamp:55105964773 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:53987883904 data:0x08020ca9 kworker/0:2-1651 [000] .... 1753.236559: rtcpu_vinotify_event: tstamp:55105964908 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:53987945536 data:0x10000000 kworker/0:2-1651 [000] .... 1753.236559: rtcpu_vinotify_event: tstamp:55105965065 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:53987949600 data:0x31000caa kworker/0:2-1651 [000] .... 1753.236560: rtcpu_vinotify_event: tstamp:55106903401 tag:CHANSEL_PXL_EOF channel:0x23 frame:1 vi_tstamp:55106679405 data:0x08070002 kworker/0:2-1651 [000] .... 1753.236561: rtcpu_vinotify_event: tstamp:55106903564 tag:ATOMP_FRAME_DONE channel:0x23 frame:1 vi_tstamp:55106679423 data:0x00000000 kworker/0:2-1651 [000] .... 1753.236561: rtcpu_vinotify_event: tstamp:55106903703 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:54016758176 data:0x02020ca9 kworker/0:2-1651 [000] .... 1753.236561: rtcpu_vinotify_event: tstamp:55106903860 tag:FE channel:0x00 frame:1 vi_tstamp:55106691037 data:0x00000020 kworker/0:2-1651 [000] .... 1753.236562: rtcpu_vinotify_event: tstamp:55106904000 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:55106691037 data:0x00000000 kworker/0:2-1651 [000] .... 1753.236562: rtcpu_vinotify_event: tstamp:55106904157 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:54017131072 data:0x00020ca9 kworker/0:2-1651 [000] .... 1753.236562: rtcpu_vinotify_event: tstamp:55106904292 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:54017133696 data:0x07020caa ALVIUM 1800 C-5-4641 [003] .... 1753.280781: tegra_channel_capture_frame: sof:1763.447364032 ALVIUM 1800 C-5-4641 [003] .... 1753.280782: tegra_channel_capture_frame: eof:1763.476610016 kworker/0:2-1651 [000] .... 1753.298966: rtos_queue_send_from_isr_failed: tstamp:55107563902 queue:0x0bcb41f8 kworker/0:2-1651 [000] .... 1753.298968: rtos_queue_send_from_isr_failed: tstamp:55107564065 queue:0x0bcb8a60 kworker/0:2-1651 [000] .... 1753.298969: rtos_queue_send_from_isr_failed: tstamp:55107564215 queue:0x0bcba5e0 kworker/0:2-1651 [000] .... 1753.298969: rtos_queue_send_from_isr_failed: tstamp:55107564365 queue:0x0bcbb3a0 kworker/0:2-1651 [000] .... 1753.298969: rtos_queue_send_from_isr_failed: tstamp:55107564512 queue:0x0bcbc160 kworker/0:2-1651 [000] .... 1753.298970: rtcpu_vinotify_event: tstamp:55107729392 tag:FS channel:0x00 frame:1 vi_tstamp:55107728841 data:0x00000010 kworker/0:2-1651 [000] .... 1753.298971: rtcpu_vinotify_event: tstamp:55107729538 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:55107728841 data:0x00000000 kworker/0:2-1651 [000] .... 1753.298975: rtcpu_vinotify_event: tstamp:55108030207 tag:CHANSEL_PXL_SOF channel:0x23 frame:1 vi_tstamp:55107730126 data:0x00000001 kworker/0:2-1651 [000] .... 1753.298975: rtcpu_vinotify_event: tstamp:55108030351 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:54050380736 data:0x08020caa kworker/0:2-1651 [000] .... 1753.298975: rtcpu_vinotify_event: tstamp:55108030512 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:54050440928 data:0x10000000 kworker/0:2-1651 [000] .... 1753.298976: rtcpu_vinotify_event: tstamp:55108030647 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:54050444832 data:0x31000cab kworker/0:2-1651 [000] .... 1753.298976: rtcpu_vinotify_event: tstamp:55108656297 tag:CHANSEL_PXL_EOF channel:0x23 frame:1 vi_tstamp:55108632430 data:0x08070002 kworker/0:2-1651 [000] .... 1753.298977: rtcpu_vinotify_event: tstamp:55108656439 tag:ATOMP_FRAME_DONE channel:0x23 frame:1 vi_tstamp:55108632448 data:0x00000000 kworker/0:2-1651 [000] .... 1753.298977: rtcpu_vinotify_event: tstamp:55108656599 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:54079255040 data:0x02020caa kworker/0:2-1651 [000] .... 1753.298977: rtcpu_vinotify_event: tstamp:55108656736 tag:FE channel:0x00 frame:1 vi_tstamp:55108644063 data:0x00000020 kworker/0:2-1651 [000] .... 1753.298978: rtcpu_vinotify_event: tstamp:55108656897 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:55108644063 data:0x00000000 kworker/0:2-1651 [000] .... 1753.298978: rtcpu_vinotify_event: tstamp:55108657031 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:54079627936 data:0x00020caa kworker/0:2-1651 [000] .... 1753.298978: rtcpu_vinotify_event: tstamp:55108657190 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:54079630560 data:0x07020cab ALVIUM 1800 C-5-4641 [003] .... 1753.343281: tegra_channel_capture_frame: sof:1763.509860832 ALVIUM 1800 C-5-4641 [003] .... 1753.343282: tegra_channel_capture_frame: eof:1763.539106848
Example trace from running all 4 cameras, first getting only 2 images, then getting all 4:
kworker/1:3-1998 [001] .... 956.713006: rtcpu_vinotify_event: tstamp:30213150073 tag:CHANSEL_PXL_EOF channel:0x20 frame:1 vi_tstamp:30213137486 data:0x08070002 kworker/1:3-1998 [001] .... 956.713008: rtcpu_vinotify_event: tstamp:30213150241 tag:ATOMP_FRAME_DONE channel:0x20 frame:1 vi_tstamp:30213137505 data:0x00000000 kworker/1:3-1998 [001] .... 956.713008: rtcpu_vinotify_event: tstamp:30213150377 tag:RESERVED_19 channel:0x20 frame:1 vi_tstamp:30517530176 data:0x02020cb0 kworker/1:3-1998 [001] .... 956.713008: rtcpu_vinotify_event: tstamp:30213150534 tag:FE channel:0x00 frame:1 vi_tstamp:30213149118 data:0x00000025 kworker/1:3-1998 [001] .... 956.713009: rtcpu_vinotify_event: tstamp:30213150674 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:30213149118 data:0x00000000 kworker/1:3-1998 [001] .... 956.713014: rtcpu_vinotify_event: tstamp:30213472234 tag:RESERVED_19 channel:0x20 frame:1 vi_tstamp:30517903840 data:0x00020cb0 kworker/1:3-1998 [001] .... 956.713014: rtcpu_vinotify_event: tstamp:30213472400 tag:RESERVED_19 channel:0x20 frame:0 vi_tstamp:30517906752 data:0x07020cb1 kworker/1:3-1998 [001] .... 956.713016: rtos_queue_send_from_isr_failed: tstamp:30213543902 queue:0x0bcb41f8 kworker/1:3-1998 [001] .... 956.713017: rtos_queue_send_from_isr_failed: tstamp:30213544065 queue:0x0bcb8a60 kworker/1:3-1998 [001] .... 956.713017: rtos_queue_send_from_isr_failed: tstamp:30213544218 queue:0x0bcba5e0 kworker/1:3-1998 [001] .... 956.713018: rtos_queue_send_from_isr_failed: tstamp:30213544365 queue:0x0bcbb3a0 kworker/1:3-1998 [001] .... 956.713018: rtos_queue_send_from_isr_failed: tstamp:30213544511 queue:0x0bcbc160 kworker/1:3-1998 [001] .... 956.713018: rtcpu_vinotify_event: tstamp:30213721518 tag:CHANSEL_PXL_EOF channel:0x22 frame:1 vi_tstamp:30213471686 data:0x08070002 kworker/1:3-1998 [001] .... 956.713018: rtcpu_vinotify_event: tstamp:30213721664 tag:ATOMP_FRAME_DONE channel:0x22 frame:1 vi_tstamp:30213471705 data:0x00000000 kworker/1:3-1998 [001] .... 956.713019: rtcpu_vinotify_event: tstamp:30213721827 tag:RESERVED_19 channel:0x22 frame:1 vi_tstamp:30528224480 data:0x02020d3e kworker/1:3-1998 [001] .... 956.713019: rtcpu_vinotify_event: tstamp:30213721967 tag:FE channel:0x00 frame:1 vi_tstamp:30213483318 data:0x00000022 kworker/1:3-1998 [001] .... 956.713019: rtcpu_vinotify_event: tstamp:30213722127 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:30213483319 data:0x00000000 kworker/1:3-1998 [001] .... 956.713020: rtcpu_vinotify_event: tstamp:30213722263 tag:RESERVED_19 channel:0x22 frame:1 vi_tstamp:30528597568 data:0x00020d3e kworker/1:3-1998 [001] .... 956.713020: rtcpu_vinotify_event: tstamp:30213722422 tag:RESERVED_18 channel:0x20 frame:0 vi_tstamp:30530602624 data:0x10000000 kworker/1:3-1998 [001] .... 956.713020: rtcpu_vinotify_event: tstamp:30213722557 tag:RESERVED_18 channel:0x20 frame:0 vi_tstamp:30530607072 data:0x31000cb2 kworker/1:3-1998 [001] .... 956.713020: rtcpu_vinotify_event: tstamp:30213722715 tag:FE channel:0x00 frame:1 vi_tstamp:30213571183 data:0x00000024 kworker/1:3-1998 [001] .... 956.713021: rtcpu_vinotify_event: tstamp:30213722855 tag:CHANSEL_PXL_EOF channel:0x23 frame:1 vi_tstamp:30213708933 data:0x08070002 kworker/1:3-1998 [001] .... 956.713021: rtcpu_vinotify_event: tstamp:30213723013 tag:ATOMP_FRAME_DONE channel:0x23 frame:1 vi_tstamp:30213708951 data:0x00000000 kworker/1:3-1998 [001] .... 956.713021: rtcpu_vinotify_event: tstamp:30213723150 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:30535816352 data:0x02020d38 kworker/1:3-1998 [001] .... 956.713022: rtcpu_vinotify_event: tstamp:30213723308 tag:FE channel:0x00 frame:1 vi_tstamp:30213720566 data:0x00000020 kworker/1:3-1998 [001] .... 956.713043: rtcpu_vinotify_event: tstamp:30213723448 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:30213720566 data:0x00000000 kworker/1:3-1998 [001] .... 956.713045: rtcpu_vinotify_error: tstamp:30213754531 tag:CHANSEL_NOMATCH channel:0x01 frame:1 vi_tstamp:30213753841 data:0x00000489 kworker/1:3-1998 [001] .... 956.713046: rtcpu_vinotify_event: tstamp:30214056793 tag:RESERVED_19 channel:0x23 frame:1 vi_tstamp:30536190208 data:0x00020d38 kworker/1:3-1998 [001] .... 956.713047: rtcpu_vinotify_event: tstamp:30214056936 tag:FS channel:0x00 frame:1 vi_tstamp:30213752557 data:0x00000010 kworker/1:3-1998 [001] .... 956.713047: rtcpu_vinotify_event: tstamp:30214057098 tag:CHANSEL_NOMATCH channel:0x01 frame:1 vi_tstamp:30213753841 data:0x00000489 kworker/1:3-1998 [001] .... 956.713048: rtcpu_vinotify_event: tstamp:30214400565 tag:FS channel:0x00 frame:1 vi_tstamp:30214186922 data:0x00000015 kworker/1:3-1998 [001] .... 956.713048: rtcpu_vinotify_event: tstamp:30214400734 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:30214186922 data:0x00000000 kworker/1:3-1998 [001] .... 956.713048: rtcpu_vinotify_event: tstamp:30214400871 tag:CHANSEL_PXL_SOF channel:0x20 frame:1 vi_tstamp:30214188206 data:0x00000001 kworker/1:3-1998 [001] .... 956.713049: rtcpu_vinotify_event: tstamp:30214401033 tag:RESERVED_19 channel:0x20 frame:1 vi_tstamp:30551152640 data:0x08020cb1 kworker/1:3-1998 [001] .... 956.713049: rtcpu_vinotify_error: tstamp:30214523125 tag:CHANSEL_NOMATCH channel:0x04 frame:1 vi_tstamp:30214522410 data:0x00000489 kworker/1:3-1998 [001] .... 956.713050: rtos_queue_send_from_isr_failed: tstamp:30214660161 queue:0x0bcb41f8 kworker/1:3-1998 [001] .... 956.713051: rtos_queue_send_from_isr_failed: tstamp:30214660335 queue:0x0bcb8a60 kworker/1:3-1998 [001] .... 956.713051: rtos_queue_send_from_isr_failed: tstamp:30214660489 queue:0x0bcba5e0 kworker/1:3-1998 [001] .... 956.713051: rtos_queue_send_from_isr_failed: tstamp:30214660635 queue:0x0bcbb3a0 kworker/1:3-1998 [001] .... 956.713052: rtos_queue_send_from_isr_failed: tstamp:30214660782 queue:0x0bcbc160 kworker/1:3-1998 [001] .... 956.713052: rtcpu_vinotify_event: tstamp:30214667987 tag:FS channel:0x00 frame:1 vi_tstamp:30214521125 data:0x00000012 kworker/1:3-1998 [001] .... 956.713052: rtcpu_vinotify_event: tstamp:30214668156 tag:CHANSEL_NOMATCH channel:0x04 frame:1 vi_tstamp:30214522410 data:0x00000489 kworker/1:3-1998 [001] .... 956.713052: rtcpu_vinotify_event: tstamp:30214668295 tag:FS channel:0x00 frame:1 vi_tstamp:30214608985 data:0x00000014 kworker/1:3-1998 [001] .... 956.713053: rtcpu_vinotify_event: tstamp:30214668453 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:30214608985 data:0x00000000 kworker/1:3-1998 [001] .... 956.713053: rtcpu_vinotify_event: tstamp:30214668602 tag:CHANSEL_PXL_SOF channel:0x21 frame:1 vi_tstamp:30214610269 data:0x00000001 kworker/1:3-1998 [001] .... 956.713053: rtcpu_vinotify_event: tstamp:30214668763 tag:RESERVED_19 channel:0x21 frame:1 vi_tstamp:30564658656 data:0x08020d13 kworker/1:3-1998 [001] .... 956.713054: rtcpu_vinotify_event: tstamp:30214668898 tag:RESERVED_18 channel:0x21 frame:0 vi_tstamp:30566322720 data:0x10000000 kworker/1:3-1998 [001] .... 956.713054: rtcpu_vinotify_event: tstamp:30214669056 tag:RESERVED_18 channel:0x21 frame:0 vi_tstamp:30566327104 data:0x31000d14 kworker/1:3-1998 [001] .... 956.713054: rtcpu_vinotify_event: tstamp:30214669196 tag:FE channel:0x00 frame:1 vi_tstamp:30214667421 data:0x00000020 kworker/1:3-1998 [001] .... 956.713056: rtcpu_vinotify_error: tstamp:30214701390 tag:CHANSEL_NOMATCH channel:0x01 frame:1 vi_tstamp:30214700697 data:0x00000489 kworker/1:3-1998 [001] .... 956.713058: rtos_queue_peek_from_isr_failed: tstamp:30214707812 queue:0x0bcbcf78 kworker/1:3-1998 [001] .... 956.713060: rtos_queue_send_from_isr_failed: tstamp:30214791931 queue:0x0bcb41f8 kworker/1:3-1998 [001] .... 956.713060: rtos_queue_send_from_isr_failed: tstamp:30214792086 queue:0x0bcb8a60 kworker/1:3-1998 [001] .... 956.713061: rtos_queue_send_from_isr_failed: tstamp:30214792239 queue:0x0bcba5e0 kworker/1:3-1998 [001] .... 956.713061: rtos_queue_send_from_isr_failed: tstamp:30214792392 queue:0x0bcbb3a0 kworker/1:3-1998 [001] .... 956.713061: rtos_queue_send_from_isr_failed: tstamp:30214792543 queue:0x0bcbc160 ALVIUM 1800 C-5-4435 [005] .... 956.722348: tegra_channel_capture_frame: sof:966.854022592 ALVIUM 1800 C-5-4435 [005] .... 956.722351: tegra_channel_capture_frame: eof:966.883268544 ALVIUM 1800 C-5-4418 [005] .... 956.735832: tegra_channel_capture_frame: sof:966.867528608 1800 C-5-4418 [005] .... 956.735835: tegra_channel_capture_frame: eof:966.896774496 kworker/1:3-1998 [001] .... 956.773132: rtos_queue_send_from_isr_failed: tstamp:30214862531 queue:0x0bcb41f8 kworker/1:3-1998 [001] .... 956.773134: rtos_queue_send_from_isr_failed: tstamp:30214862681 queue:0x0bcb8a60 kworker/1:3-1998 [001] .... 956.773134: rtos_queue_send_from_isr_failed: tstamp:30214862833 queue:0x0bcba5e0 kworker/1:3-1998 [001] .... 956.773135: rtos_queue_send_from_isr_failed: tstamp:30214862993 queue:0x0bcbb3a0 kworker/1:3-1998 [001] .... 956.773135: rtos_queue_send_from_isr_failed: tstamp:30214863144 queue:0x0bcbc160 kworker/1:3-1998 [001] .... 956.773136: rtcpu_vinotify_event: tstamp:30214985201 tag:FS channel:0x00 frame:1 vi_tstamp:30214699413 data:0x00000010 kworker/1:3-1998 [001] .... 956.773136: rtcpu_vinotify_event: tstamp:30214985346 tag:CHANSEL_NOMATCH channel:0x01 frame:1 vi_tstamp:30214700697 data:0x00000489 kworker/1:3-1998 [001] .... 956.773136: rtcpu_vinotify_event: tstamp:30214985512 tag:RESERVED_18 channel:0x22 frame:0 vi_tstamp:30570538624 data:0x10000000 kworker/1:3-1998 [001] .... 956.773136: rtcpu_vinotify_event: tstamp:30214985656 tag:RESERVED_18 channel:0x22 frame:0 vi_tstamp:30570542976 data:0x31000d3f kworker/1:3-1998 [001] .... 956.773137: rtcpu_vinotify_event: tstamp:30214985827 tag:RESERVED_19 channel:0x22 frame:0 vi_tstamp:30570544064 data:0x07020d3f kworker/1:3-1998 [001] .... 956.773141: rtcpu_vinotify_event: tstamp:30215329087 tag:CHANSEL_PXL_EOF channel:0x20 frame:1 vi_tstamp:30215090510 data:0x08070002 kworker/1:3-1998 [001] .... 956.773141: rtcpu_vinotify_event: tstamp:30215329249 tag:ATOMP_FRAME_DONE channel:0x20 frame:1 vi_tstamp:30215090528 data:0x00000000 kworker/1:3-1998 [001] .... 956.773141: rtcpu_vinotify_event: tstamp:30215329387 tag:RESERVED_19 channel:0x20 frame:1 vi_tstamp:30580026816 data:0x02020cb1 kworker/1:3-1998 [001] .... 956.773141: rtcpu_vinotify_event: tstamp:30215329542 tag:FE channel:0x00 frame:1 vi_tstamp:30215102142 data:0x00000025 kworker/1:3-1998 [001] .... 956.773142: rtcpu_vinotify_event: tstamp:30215329686 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:30215102142 data:0x00000000 kworker/1:3-1998 [001] .... 956.773142: rtcpu_vinotify_event: tstamp:30215329843 tag:RESERVED_19 channel:0x20 frame:1 vi_tstamp:30580399840 data:0x00020cb1 kworker/1:3-1998 [001] .... 956.773142: rtcpu_vinotify_event: tstamp:30215329978 tag:RESERVED_19 channel:0x20 frame:0 vi_tstamp:30580402752 data:0x07020cb2 kworker/1:3-1998 [001] .... 956.773142: rtcpu_vinotify_event: tstamp:30215513169 tag:FE channel:0x00 frame:1 vi_tstamp:30215436349 data:0x00000022 kworker/1:3-1998 [001] .... 956.773142: rtcpu_vinotify_event: tstamp:30215513305 tag:CHANSEL_PXL_EOF channel:0x21 frame:1 vi_tstamp:30215512570 data:0x08070002 kworker/1:3-1998 [001] .... 956.773143: rtcpu_vinotify_event: tstamp:30215513466 tag:ATOMP_FRAME_DONE channel:0x21 frame:1 vi_tstamp:30215512597 data:0x00000000 kworker/1:3-1998 [001] .... 956.773143: rtcpu_vinotify_event: tstamp:30215513602 tag:RESERVED_19 channel:0x21 frame:1 vi_tstamp:30593533152 data:0x02020d13 kworker/1:3-1998 [001] .... 956.773146: rtcpu_vinotify_error: tstamp:30215648249 tag:CHANSEL_NOMATCH channel:0x01 frame:1 vi_tstamp:30215647552 data:0x00000489 kworker/1:3-1998 [001] .... 956.773147: rtcpu_vinotify_event: tstamp:30215844817 tag:FE channel:0x00 frame:1 vi_tstamp:30215524203 data:0x00000024 kworker/1:3-1998 [001] .... 956.773147: rtcpu_vinotify_event: tstamp:30215844961 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:30215524203 data:0x00000000 kworker/1:3-1998 [001] .... 956.773148: rtcpu_vinotify_event: tstamp:30215845121 tag:RESERVED_19 channel:0x21 frame:1 vi_tstamp:30593905824 data:0x00020d13 kworker/1:3-1998 [001] .... 956.773148: rtcpu_vinotify_event: tstamp:30215845258 tag:RESERVED_19 channel:0x21 frame:0 vi_tstamp:30593908672 data:0x07020d14 kworker/1:3-1998 [001] .... 956.773148: rtcpu_vinotify_event: tstamp:30215845412 tag:FE channel:0x00 frame:1 vi_tstamp:30215614276 data:0x00000020 kworker/1:3-1998 [001] .... 956.773148: rtcpu_vinotify_event: tstamp:30215845551 tag:FS channel:0x00 frame:1 vi_tstamp:30215646267 data:0x00000010 kworker/1:3-1998 [001] .... 956.773149: rtcpu_vinotify_event: tstamp:30215845708 tag:CHANSEL_NOMATCH channel:0x01 frame:1 vi_tstamp:30215647552 data:0x00000489 kworker/1:3-1998 [001] .... 956.773149: rtos_queue_send_from_isr_failed: tstamp:30215982064 queue:0x0bcb41f8 kworker/1:3-1998 [001] .... 956.773149: rtos_queue_send_from_isr_failed: tstamp:30215982221 queue:0x0bcb8a60 kworker/1:3-1998 [001] .... 956.773149: rtos_queue_send_from_isr_failed: tstamp:30215982374 queue:0x0bcba5e0 kworker/1:3-1998 [001] .... 956.773150: rtos_queue_send_from_isr_failed: tstamp:30215982522 queue:0x0bcbb3a0 kworker/1:3-1998 [001] .... 956.773150: rtos_queue_send_from_isr_failed: tstamp:30215982669 queue:0x0bcbc160 kworker/1:3-1998 [001] .... 956.773150: rtos_queue_send_from_isr_failed: tstamp:30216082635 queue:0x0bcb41f8 kworker/1:3-1998 [001] .... 956.773150: rtos_queue_send_from_isr_failed: tstamp:30216082789 queue:0x0bcb8a60 kworker/1:3-1998 [001] .... 956.773150: rtos_queue_send_from_isr_failed: tstamp:30216082940 queue:0x0bcba5e0 kworker/1:3-1998 [001] .... 956.773151: rtos_queue_send_from_isr_failed: tstamp:30216083090 queue:0x0bcbb3a0 kworker/1:3-1998 [001] .... 956.773151: rtos_queue_send_from_isr_failed: tstamp:30216083235 queue:0x0bcbc160 kworker/1:3-1998 [001] .... 956.773152: rtcpu_vinotify_event: tstamp:30216143657 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:30608625088 data:0x10000000 kworker/1:3-1998 [001] .... 956.773152: rtcpu_vinotify_event: tstamp:30216143826 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:30608629504 data:0x31000d39 kworker/1:3-1998 [001] .... 956.773152: rtcpu_vinotify_event: tstamp:30216143965 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:30608630592 data:0x07020d39 kworker/1:3-1998 [001] .... 956.773152: rtcpu_vinotify_event: tstamp:30216144126 tag:FS channel:0x00 frame:1 vi_tstamp:30216139946 data:0x00000015 kworker/1:3-1998 [001] .... 956.773153: rtcpu_vinotify_event: tstamp:30216144267 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:30216139947 data:0x00000000 kworker/1:3-1998 [001] .... 956.773153: rtcpu_vinotify_event: tstamp:30216144428 tag:CHANSEL_PXL_SOF channel:0x20 frame:1 vi_tstamp:30216141230 data:0x00000001 kworker/1:3-1998 [001] .... 956.773153: rtcpu_vinotify_event: tstamp:30216144566 tag:RESERVED_19 channel:0x20 frame:1 vi_tstamp:30613649408 data:0x08020cb2 kworker/1:3-1998 [001] .... 956.773156: rtcpu_vinotify_event: tstamp:30216144725 tag:RESERVED_18 channel:0x20 frame:0 vi_tstamp:30613704768 data:0x10000000 kworker/1:3-1998 [001] .... 956.773158: rtcpu_vinotify_event: tstamp:30216144860 tag:RESERVED_18 channel:0x20 frame:0 vi_tstamp:30613709248 data:0x31000cb3 kworker/1:3-1998 [001] .... 956.773158: rtcpu_vinotify_event: tstamp:30216498209 tag:FS channel:0x00 frame:1 vi_tstamp:30216474155 data:0x00000012 kworker/1:3-1998 [001] .... 956.773158: rtcpu_vinotify_event: tstamp:30216498351 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:30216474156 data:0x00000000 kworker/1:3-1998 [001] .... 956.773158: rtcpu_vinotify_event: tstamp:30216498510 tag:CHANSEL_PXL_SOF channel:0x22 frame:1 vi_tstamp:30216475439 data:0x00000001 kworker/1:3-1998 [001] .... 956.773159: rtcpu_vinotify_event: tstamp:30216498645 tag:RESERVED_19 channel:0x22 frame:1 vi_tstamp:30624344064 data:0x08020d3f ALVIUM 1800 C-5-4435 [005] .... 956.784843: tegra_channel_capture_frame: sof:966.916519360 ALVIUM 1800 C-5-4435 [005] .... 956.784846: tegra_channel_capture_frame: eof:966.945765344 ALVIUM 1800 C-5-4401 [005] .... 956.795542: tegra_channel_capture_frame: sof:966.927214048 ALVIUM 1800 C-5-4401 [005] .... 956.795545: tegra_channel_capture_frame: eof:966.956460128 ALVIUM 1800 C-5-4418 [005] .... 956.798325: tegra_channel_capture_frame: sof:966.930025216 ALVIUM 1800 C-5-4418 [005] .... 956.798326: tegra_channel_capture_frame: eof:966.959271136 ALVIUM 1800 C-5-4380 [005] .... 956.799292: tegra_channel_capture_frame: sof:966.931021056 ALVIUM 1800 C-5-4380 [005] .... 956.799293: tegra_channel_capture_frame: eof:966.960255616 ALVIUM 1800 C-5-4435 [005] .... 956.847372: tegra_channel_capture_frame: sof:966.979016128 ALVIUM 1800 C-5-4435 [005] .... 956.847375: tegra_channel_capture_frame: eof:967.8262112