I’m running multiple Xavier NX devices in production with IMX477 CSI cameras. The devices are operating normally, until at some point I start seeing the following error messages.
Jan 18 01:15:31 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: NvCaptureStatusErrorDecode Capture-Error: FALCON_ERROR (0x0000000e)
Jan 18 01:15:31 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: SCF: Error InvalidState: Capture error with status 14 (channel 0) (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 930)
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: NvCaptureStatusErrorDecode Stream 2.0 failed: sof_ts 0 eof_ts 47717838939936 frame 0 error 2 data 0x00a00462
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: NvCaptureStatusErrorDecode Capture-Error: CSIMUX_FRAME (0x00000002)
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: CsimuxFrameError_Regular : 0x00a00462
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: Stream ID [ 2: 0]: 2
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]:
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: VPR state from fuse block [ 3]: 0
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]:
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: Frame end (FE) [ 5]: 1
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: A frame end has been found on a regular mode stream.
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: CSI_FAULT [ 6]: 1
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: An FE packet was found and marked with a CSI Error
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: PXL_ENABLE_FAULT [10]: 1
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: An illegal pixel enable encoding has been detected on a long packete.g. 6 pixels enabled on a CSI packet without line end
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: CSI_CODE [25:20]: 0xa
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: Check the CSI specification for the meaning. See https://wiki.nvidia.com/wmpwiki/index.php/Camera_Debugging/CaptureError_debugging#CSI_CODE
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: Binary VC number [3:2] [27:26]: 0
Jan 18 01:15:32 xavier-nx-devkit-emmc-1420122041297 nvargus-daemon[5037]: To get full binary VC number, user need to concatenate VC[3:2] and VC[1:0] together.
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: name=[nvhost_sync:46], current value=759 waiting value=760
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: ---- mlocks ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: ---- syncpts ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: name=[nvhost_sync:46], current value=759 waiting value=760
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: ---- mlocks ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: ---- syncpts ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 17 (gv11b_511) min 231 max 231 refs 1 (previous client : )
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 18 (gv11b_510) min 3 max 3 refs 1 (previous client : )
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 19 (15340000.vic_CaptureAndDetec_0) min 1744728 max 1744728 refs 1 (previous client : 15340000.vic_CaptureAndDetec_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 21 (gv11b_498_user) min 2094331 max 2093714 refs 1 (previous client : gv11b_507_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 23 (gv11b_495_user) min 2772329 max 2770107 refs 1 (previous client : gv11b_509_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 25 (gv11b_499_user) min 3260813 max 3234882 refs 1 (previous client : gv11b_506_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 29 (gv11b_497_user) min 3012424 max 3011383 refs 1 (previous client : gv11b_508_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 31 (gv11b_496_user) min 1048576 max 1048576 refs 1 (previous client : gv11b_505_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 33 (progress) min 3290 max 0 refs 1 (previous client : progress)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 34 (stats_progress) min 6110 max 0 refs 1 (previous client : stats_progress)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 36 (gv11b_502_user) min 262144 max 262144 refs 1 (previous client : gv11b_507_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 38 (gv11b_503_user) min 262208 max 262179 refs 1 (previous client : gv11b_509_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 40 (gv11b_504_user) min 262144 max 262144 refs 1 (previous client : gv11b_508_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 42 (gv11b_500_user) min 262144 max 262144 refs 1 (previous client : gv11b_506_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 44 (gv11b_501_user) min 262144 max 262144 refs 1 (previous client : gv11b_505_user)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 45 (15340000.vic_CaptureAndDetec_0) min 1744716 max 1744716 refs 1 (previous client : 15340000.vic_CaptureAndDetec_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 46 (progress) min 759 max 0 refs 1 (previous client : progress)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 47 (embdata) min 232 max 0 refs 1 (previous client : embdata)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 49 (15340000.vic_nvargus-daemon_0) min 30 max 30 refs 1 (previous client : gv11b_507)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 50 (gv11b_505) min 77 max 77 refs 1 (previous client : 15340000.vic_nvargus-daemon_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 51 (gv11b_508) min 106 max 106 refs 1 (previous client : gv11b_509)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 52 (gv11b_506) min 105 max 105 refs 1 (previous client : gv11b_506)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 53 (gv11b_509) min 105 max 105 refs 1 (previous client : gv11b_508)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 54 (gv11b_507) min 105 max 105 refs 1 (previous client : gv11b_505)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 55 (15880000.nvdla0_CaptureAndDetec_0) min 143162 max 143162 refs 1 (previous client : 15340000.vic_argus_thread_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 56 (15340000.vic_CaptureAndDetec_0) min 257 max 257 refs 1 (previous client : 15340000.vic_CaptureAndDetec_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 57 (15340000.vic_CaptureAndDetec_0) min 43921 max 43921 refs 1 (previous client : 15340000.vic_argus_thread_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 58 (15880000.nvdla0_CaptureAndDetec_1) min 95255 max 95255 refs 1 (previous client : 15880000.nvdla0_CaptureAndDetec_2)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 59 (15880000.nvdla0_CaptureAndDetec_2) min 927 max 927 refs 1 (previous client : 15340000.vic_argus_thread_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: id 60 (15880000.nvdla0_CaptureAndDetec_3) min 959 max 959 refs 1 (previous client : 15340000.vic_argus_thread_0)
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: ---- channels ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
channel 2 - 15820000.se
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: NvHost basic channel registers:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDFIFO_STAT_0: 00002040
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDFIFO_RDATA_0: a4024028
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDP_OFFSET_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDP_CLASS_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CHANNELSTAT_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: The CDMA sync queue is empty.
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
channel 3 - 15830000.se
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: NvHost basic channel registers:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDFIFO_STAT_0: 00002040
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDFIFO_RDATA_0: 4083c044
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDP_OFFSET_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDP_CLASS_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CHANNELSTAT_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: The CDMA sync queue is empty.
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
channel 4 - 15840000.se
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: NvHost basic channel registers:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDFIFO_STAT_0: 00002040
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDFIFO_RDATA_0: 080605a9
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDP_OFFSET_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CMDP_CLASS_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: CHANNELSTAT_0: 00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: The CDMA sync queue is empty.
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
---- host general irq ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: sync_intc0mask = 0x00000001
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: sync_intmask = 0x50000003
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
---- host syncpt irq mask ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel:
---- host syncpt irq status ----
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(0) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(1) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(2) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(3) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(4) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(5) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(6) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(7) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(8) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(9) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(10) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(11) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(12) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(13) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(14) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(15) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(16) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(17) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(18) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(19) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(20) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: syncpt_thresh_cpu0_int_status(21) = 0x00000000
Jan 18 01:16:10 xavier-nx-devkit-emmc-1420122041297 kernel: misc tegra_camera_ctrl: tegra_camera_update_isobw: Warning, Requested ISO BW 1640625 has been capped to VI's max BW 1500000
I’ve been trying to figure out what might cause the camera failure. The problem persists for some time, and it affect the uptime of the devices as the stream from the camera is broken.
I cannot make sense of the FALCON_ERROR, whatever that means, and the kernel messages.
Note: CaptureAndDetect is part of my application which runs inferencing using TensorRT optimized engines on both the GPU and the DLAs. I’m also using VPI to process the frames from the camera.