I can see IVC(Inter-VM Communication) is used in xavier camera driver. Function tegra_capture_ivc_worker in file capture-ivc.c will read IVC messages and dispatch it. IVC seems to user ivc-bus.
1.Who is the message sender for vi_capture_ivc_status_callback in file capture.c? My CSI camera data can trigger VI CHANSEL(Channel Selector) match event, but it can not trigger vi_capture_ivc_status_callback. So I fail to get camera data.
2.Is there document for IVC?
thanks
hello 756948396,
there’re functions to handle rtcpu messages, which is the firmware you could find in the rce-fw partition (camera-rtcpu-rce.img).
you might refer to [Xavier Series (SoC) Technical Reference Manual] and check the [Multimedia Complex]-> [Camera Subsystem]-> [Real-Time Camera Engine (RCE)] for details.
suggest you should debug your sensor drivers and sensor device tree,
could you please check from kernel side to see if there’s suspicious failures.
please also refer to Sensor Driver Programming Guide and review your drivers.
in addition, please follow below steps to enable VI tracing logs to gather more details from rtcpu side.
enable VI tracing logs.
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace
Hi JerryChang,
I use v4l2-ctl to get one frame and the tracing log is as following:
root@jetson-0423318035480:/home/nvidia/pro1# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 274/274 #P:4
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
kworker/3:0-30 [003] .... 1233.325550: rtos_queue_peek_from_isr_failed: tstamp:38674975353 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1233.325569: rtcpu_start: tstamp:38674978504
kworker/3:0-30 [003] .... 1233.325574: rtos_queue_send_from_isr_failed: tstamp:38674983307 queue:0x0bcc10a0
kworker/3:0-30 [003] .... 1233.325575: rtos_queue_send_from_isr_failed: tstamp:38674983602 queue:0x0bcc49d8
kworker/3:0-30 [003] .... 1233.325576: rtos_queue_send_from_isr_failed: tstamp:38674983923 queue:0x0bcc6558
kworker/3:0-30 [003] .... 1233.325578: rtos_queue_send_from_isr_failed: tstamp:38674984212 queue:0x0bcc7318
kworker/3:0-30 [003] .... 1233.325580: rtos_queue_send_from_isr_failed: tstamp:38674984496 queue:0x0bcc80d8
kworker/3:0-30 [003] .... 1233.325585: rtcpu_dbg_set_loglevel: tstamp:38674986524 old:0 new:2
kworker/3:0-30 [003] .... 1233.325594: rtos_queue_send_from_isr_failed: tstamp:38674998893 queue:0x0bcc10a0
kworker/3:0-30 [003] .... 1233.325596: rtos_queue_send_from_isr_failed: tstamp:38674999177 queue:0x0bcc49d8
kworker/3:0-30 [003] .... 1233.325597: rtos_queue_send_from_isr_failed: tstamp:38674999465 queue:0x0bcc6558
kworker/3:0-30 [003] .... 1233.325598: rtos_queue_send_from_isr_failed: tstamp:38674999749 queue:0x0bcc7318
kworker/3:0-30 [003] .... 1233.325599: rtos_queue_send_from_isr_failed: tstamp:38675000046 queue:0x0bcc80d8
kworker/3:0-30 [003] .... 1233.493539: rtos_queue_peek_from_isr_failed: tstamp:38679975860 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1233.661451: rtos_queue_peek_from_isr_failed: tstamp:38684975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1233.773445: rtos_queue_peek_from_isr_failed: tstamp:38689975860 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1233.941441: rtos_queue_peek_from_isr_failed: tstamp:38694975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1234.109437: rtos_queue_peek_from_isr_failed: tstamp:38699975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1234.277446: rtos_queue_peek_from_isr_failed: tstamp:38704975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1234.445439: rtos_queue_peek_from_isr_failed: tstamp:38709975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1234.613439: rtos_queue_peek_from_isr_failed: tstamp:38714975866 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1234.781447: rtos_queue_peek_from_isr_failed: tstamp:38719975856 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1234.893441: rtos_queue_peek_from_isr_failed: tstamp:38724975856 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1235.061438: rtos_queue_peek_from_isr_failed: tstamp:38729975858 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1235.229440: rtos_queue_peek_from_isr_failed: tstamp:38734975856 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1235.397482: rtos_queue_peek_from_isr_failed: tstamp:38739975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1235.565437: rtos_queue_peek_from_isr_failed: tstamp:38744975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1235.733442: rtos_queue_peek_from_isr_failed: tstamp:38749975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1235.901440: rtos_queue_peek_from_isr_failed: tstamp:38754975859 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1236.013442: rtos_queue_peek_from_isr_failed: tstamp:38759975858 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1236.181439: rtos_queue_peek_from_isr_failed: tstamp:38764975858 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1236.349447: rtos_queue_peek_from_isr_failed: tstamp:38769975858 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1236.517446: rtos_queue_peek_from_isr_failed: tstamp:38774975858 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1236.685441: rtos_queue_peek_from_isr_failed: tstamp:38779975859 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1236.853442: rtos_queue_peek_from_isr_failed: tstamp:38784975859 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1237.021439: rtos_queue_peek_from_isr_failed: tstamp:38789975858 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1237.133440: rtos_queue_peek_from_isr_failed: tstamp:38794975866 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1237.301438: rtos_queue_peek_from_isr_failed: tstamp:38799975856 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1237.469441: rtos_queue_peek_from_isr_failed: tstamp:38804975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1237.637449: rtos_queue_peek_from_isr_failed: tstamp:38809975856 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1237.805440: rtos_queue_peek_from_isr_failed: tstamp:38814975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1237.973438: rtos_queue_peek_from_isr_failed: tstamp:38819975856 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1238.141439: rtos_queue_peek_from_isr_failed: tstamp:38824975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1238.253455: rtos_queue_peek_from_isr_failed: tstamp:38829975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1238.421453: rtos_queue_peek_from_isr_failed: tstamp:38834975855 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1238.477439: rtos_queue_peek_from_isr_failed: tstamp:38836798583 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1305.237606: rtos_queue_peek_from_isr_failed: tstamp:40922177909 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1305.237623: rtcpu_start: tstamp:40922180448
kworker/3:0-30 [003] .... 1305.237627: rtos_queue_send_from_isr_failed: tstamp:40922919798 queue:0x0bcc10a0
kworker/3:0-30 [003] .... 1305.237628: rtos_queue_send_from_isr_failed: tstamp:40922920085 queue:0x0bcc49d8
kworker/3:0-30 [003] .... 1305.237629: rtos_queue_send_from_isr_failed: tstamp:40922920373 queue:0x0bcc6558
kworker/3:0-30 [003] .... 1305.237630: rtos_queue_send_from_isr_failed: tstamp:40922920657 queue:0x0bcc7318
kworker/3:0-30 [003] .... 1305.237632: rtos_queue_send_from_isr_failed: tstamp:40922920939 queue:0x0bcc80d8
kworker/3:0-30 [003] .... 1305.237636: rtcpu_string: tstamp:40922931625 id:0x04010000 str:"vi5_hwinit: firmware CL2018061801 protocol versi"
kworker/3:0-30 [003] .... 1305.237640: rtcpu_string: tstamp:40922931891 id:0x04010000 str:"on 2.2
"
kworker/3:0-30 [003] .... 1305.294773: rtos_queue_send_from_isr_failed: tstamp:40924449771 queue:0x0bcc10a0
kworker/3:0-30 [003] .... 1305.294779: rtos_queue_send_from_isr_failed: tstamp:40924449945 queue:0x0bcc49d8
kworker/3:0-30 [003] .... 1305.294780: rtos_queue_send_from_isr_failed: tstamp:40924450095 queue:0x0bcc6558
kworker/3:0-30 [003] .... 1305.294781: rtos_queue_send_from_isr_failed: tstamp:40924450241 queue:0x0bcc7318
kworker/3:0-30 [003] .... 1305.294825: rtos_queue_send_from_isr_failed: tstamp:40924450387 queue:0x0bcc80d8
kworker/3:0-30 [003] .... 1305.353456: rtos_queue_send_from_isr_failed: tstamp:40926395290 queue:0x0bcc10a0
kworker/3:0-30 [003] .... 1305.353461: rtos_queue_send_from_isr_failed: tstamp:40926395438 queue:0x0bcc49d8
kworker/3:0-30 [003] .... 1305.353463: rtos_queue_send_from_isr_failed: tstamp:40926395588 queue:0x0bcc6558
kworker/3:0-30 [003] .... 1305.353464: rtos_queue_send_from_isr_failed: tstamp:40926395733 queue:0x0bcc7318
kworker/3:0-30 [003] .... 1305.353465: rtos_queue_send_from_isr_failed: tstamp:40926395877 queue:0x0bcc80d8
kworker/3:0-30 [003] .... 1305.353468: rtcpu_vinotify_event: tstamp:40926452944 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:42629429504 data:0x10000000
kworker/3:0-30 [003] .... 1305.353470: rtcpu_vinotify_event: tstamp:40926453092 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:42629462432 data:0x31000001
kworker/3:0-30 [003] .... 1305.353471: rtcpu_vinotify_event: tstamp:40926453280 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:42629471168 data:0x07020001
kworker/3:0-30 [003] .... 1305.409482: rtos_queue_peek_from_isr_failed: tstamp:40927178301 queue:0x0bcc8ef0
kworker/3:0-30 [003] .... 1305.409489: rtos_queue_send_from_isr_failed: tstamp:40927399711 queue:0x0bcc10a0
kworker/3:0-30 [003] .... 1305.409490: rtos_queue_send_from_isr_failed: tstamp:40927399866 queue:0x0bcc49d8
kworker/3:0-30 [003] .... 1305.409491: rtos_queue_send_from_isr_failed: tstamp:40927400016 queue:0x0bcc6558
kworker/3:0-30 [003] .... 1305.409493: rtos_queue_send_from_isr_failed: tstamp:40927400163 queue:0x0bcc7318
kworker/3:0-30 [003] .... 1305.409494: rtos_queue_send_from_isr_failed: tstamp:40927400308 queue:0x0bcc80d8
kworker/3:0-30 [003] .... 1305.409496: rtcpu_vinotify_event: tstamp:40927538481 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:42661556704 data:0x10000000
kworker/3:0-30 [003] .... 1305.409497: rtcpu_vinotify_event: tstamp:40927538652 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:42661586144 data:0x31000002
kworker/3:0-30 [003] .... 1305.409499: rtos_queue_send_from_isr_failed: tstamp:40928382922 queue:0x0bcc10a0
kworker/3:0-30 [003] .... 1305.409500: rtos_queue_send_from_isr_failed: tstamp:40928383071 queue:0x0bcc49d8
thanks
Error of rtos_queue_send_from_isr_failed still exists although I do noe excute v4l2-ctl command.
Hi JerryChang,
I tried to trace VI logs from rtcpu side, and execute v4l2-ctl cmd to get raw frame but failed, the tracing log showed below:
[ +0.000103] tegra194-vi5 15c10000.vi: vi capture dequeue status failed
[ +0.000220] tegra194-vi5 15c10000.vi: channel error, resetting the channel
The RTCPU log normally should be:
kworker/0:1-1215 [000] .... 71.693735: rtcpu_vinotify_error: tstamp:2373182753 tag:CSIMUX_STREAM channel:0x00 frame:0 vi_tstamp:2373179503 data:0x00000001
kworker/0:1-1215 [000] .... 71.693738: rtcpu_vinotify_event: tstamp:2373191565 tag:CSIMUX_STREAM channel:0x00 frame:0 vi_tstamp:2373179503 data:0x00000001
kworker/0:1-1215 [000] .... 71.693739: rtcpu_vinotify_event: tstamp:2373191713 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:2927459232 data:0x10000000
kworker/0:1-1215 [000] .... 71.693740: rtcpu_vinotify_event: tstamp:2373191883 tag:RESERVED_18 channel:0x23 frame:0 vi_tstamp:2927488288 data:0x31000001
kworker/0:1-1215 [000] .... 71.693769: rtcpu_vinotify_event: tstamp:2373192024 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:2927497056 data:0x07020001
kworker/0:1-1215 [000] .... 71.693770: rtcpu_vinotify_error: tstamp:2373433081 tag:CHANSEL_NOMATCH channel:0x01 frame:0 vi_tstamp:2373427073 data:0x000006e9
kworker/0:1-1215 [000] .... 71.693777: rtcpu_vinotify_event: tstamp:2373734161 tag:FS channel:0x00 frame:0 vi_tstamp:2373426643 data:0x00000010
kworker/0:1-1215 [000] .... 71.693778: rtcpu_vinotify_event: tstamp:2373734309 tag:ATOMP_FS channel:0x00 frame:0 vi_tstamp:2373426646 data:0x00000000
kworker/0:1-1215 [000] .... 71.693779: rtcpu_vinotify_event: tstamp:2373734478 tag:CHANSEL_EMBED_SOF channel:0x23 frame:0 vi_tstamp:2373426845 data:0x00000004
kworker/0:1-1215 [000] .... 71.693780: rtcpu_vinotify_event: tstamp:2373734620 tag:CHANSEL_EMBED_EOF channel:0x23 frame:0 vi_tstamp:2373426852 data:0x00000008
kworker/0:1-1215 [000] .... 71.693781: rtcpu_vinotify_event: tstamp:2373734803 tag:ATOMP_EMB_DATA_DONE channel:0x23 frame:0 vi_tstamp:2373426926 data:0x00000000
kworker/0:1-1215 [000] .... 71.693782: rtcpu_vinotify_event: tstamp:2373734943 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:2935232224 data:0x03020001
kworker/0:1-1215 [000] .... 71.693783: rtcpu_vinotify_event: tstamp:2373735109 tag:CHANSEL_NOMATCH channel:0x01 frame:0 vi_tstamp:2373427073 data:0x000006e9
kworker/0:1-1215 [000] .... 71.693785: rtcpu_vinotify_event: tstamp:2373735249 tag:CHANSEL_PXL_SOF channel:0x23 frame:0 vi_tstamp:2373428899 data:0x00000001
kworker/0:1-1215 [000] .... 71.693786: rtcpu_vinotify_event: tstamp:2373735415 tag:RESERVED_19 channel:0x23 frame:0 vi_tstamp:2935297696 data:0x08020001
kworker/0:1-1215 [000] .... 71.745703: rtcpu_vinotify_event: tstamp:2373926425 tag:CHANSEL_PXL_EOF channel:0x23 frame:0 vi_tstamp:2373922004 data:0x086f0002
kworker/0:1-1215 [000] .... 71.745709: rtcpu_vinotify_event: tstamp:2373926591 tag:FE channel:0x00 frame:0 vi_tstamp:2373922022 data:0x00000020
But the RTCPU log which I cann’t get raw image showed below:
kworker/1:2-1266 [001] .... 71.656121: rtcpu_vinotify_event: tstamp:2370488066 tag:FE channel:0x01 frame:329 vi_tstamp:2370044682 data:0x00000020
kworker/1:2-1266 [001] .... 71.656124: rtcpu_vinotify_error: tstamp:2370603150 tag:CHANSEL_NOMATCH channel:0x41 frame:330 vi_tstamp:2370600445 data:0x00000569
kworker/1:2-1266 [001] .... 71.656126: rtcpu_vinotify_event: tstamp:2370833305 tag:FS channel:0x01 frame:330 vi_tstamp:2370593840 data:0x00000010
kworker/1:2-1266 [001] .... 71.656127: rtcpu_vinotify_event: tstamp:2370833473 tag:CHANSEL_NOMATCH channel:0x41 frame:330 vi_tstamp:2370600445 data:0x00000569
kworker/1:2-1266 [001] .... 71.656128: rtcpu_vinotify_event: tstamp:2370833613 tag:FE channel:0x01 frame:330 vi_tstamp:2370830814 data:0x00000020
kworker/1:2-1266 [001] .... 71.656129: rtcpu_vinotify_error: tstamp:2371379387 tag:CHANSEL_NOMATCH channel:0x41 frame:331 vi_tstamp:2371376683 data:0x00000569
kworker/1:2-1266 [001] .... 71.656130: rtcpu_vinotify_event: tstamp:2371410050 tag:FS channel:0x01 frame:331 vi_tstamp:2371370078 data:0x00000010
kworker/1:2-1266 [001] .... 71.656132: rtcpu_vinotify_event: tstamp:2371410208 tag:CHANSEL_NOMATCH channel:0x41 frame:331 vi_tstamp:2371376683 data:0x00000569
kworker/1:2-1266 [001] .... 71.656133: rtcpu_vinotify_event: tstamp:2371952970 tag:FE channel:0x01 frame:331 vi_tstamp:2371607052 data:0x00000020
It seems that some kinds of tags missed, Why did it happen?
Thanks
hello buaa_hebiao,
according to the tracing logs.
tag:CHANSEL_NOMATCH channel:0x41 frame:330 vi_tstamp:2370600445 data:0x00000569
it shows no match failure, and report the streaming as below.
DTYPE: NvCsiDataType_RAW10
NO_MATCH: True
please double confirm the stream configuration and review the kernel driver.
thanks
Hi JerryChang,
The pixelfmt of my camera is RAW10, I checked “tegra_channel_surface_setup” function in vi5_fops.c, the data_type was shown to be “TEGRA_IMAGE_DT_RAW10”, is that mean RTCPU will match RAW10 image at incoming stream? Is there some convenient method i can apply to confirm the real datatype from camera?
desc->ch_cfg.match.datatype = data_type;
Thanks
Hi JerryChang,
Might I ask where you found the documentation to decode this CHANSEL_NOMATCH data? I cannot seem to find it in the Xavier TRM?
Thanks!
hello arnout.diels,
sorry did not notice this question since this is an old discussion thread,
please refer to Xavier TRM, you should check [Chapter-7.2.2.2.2 CHANSEL]; please refer to [Notifications] session for the descriptions.
for example,
NOMATCH (many-channel) is emitted because of the “no-match” condition described above.