We have a carrier board that has 6 2-lane camera inputs. Each of the 6 cameras works well individually, but when I try to run two simultaneously, I get a few frames initially, and then I see a fence timeout in the kernel messages, and Argus complains about timing out while waiting for the end of a frame. There is a wealth of debugging information in the kernel messages, but it’s largely meaningless to me. I’ve attached the kernel message log:
https://1drv.ms/t/s!AjqS0Pe9SwR2h1d2iU3wD7Ki7uVX
Here are the Argus messages (note that my code attempts to restart the camera by releasing all the Argus objects and rebuilding the pipeline):
SCF: Error Timeout: (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc
Error: waitCsiFrameEnd timeout guid 0
VI Stream Id = 0 Virtual Channel = 0
************VI Debug Registers**********
VI_CSIMUX_STAT_FRAME_0 = 0x000000e2
VI_CSIMUX_FRAME_STATUS_0 = 0x00000000
VI_CFG_INTERRUPT_STATUS_0 = 0x3f000000
VI_ISPBUFA_ERROR_0 = 0x00000000
VI_FMLITE_ERROR_0 = 0x00000000
VI_NOTIFY_ERROR_0 = 0x00000000
*****************************************
CSI Stream Id = 0 Brick Id = 0
************CSI Debug Registers**********
CILA_INTR_STATUS_CILA[0x10400] = 0x00000089
CILB_INTR_STATUS_CILB[0x10c00] = 0x00000089
INTR_STATUS[0x100a4] = 0x00000000
INTR_STATUS[0x100a4] = 0x00000000
ERR_INTR_STATUS[0x100ac] = 0x00000000
ERROR_STATUS2VI_VC0[0x10094] = 0x00000000
ERROR_STATUS2VI_VC1[0x10098] = 0x00000000
ERROR_STATUS2VI_VC2[0x1009c] = 0x00000000
ERROR_STATUS2VI_VC3[0x100a0] = 0x00000000
*****************************************
SCF: Error Timeout: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 992)
[2018-07-30 15:24:13.086] [CameraTest] [error] JetsonCamera: Argus reported a error 6
Error: waitCsiFrameEnd Something went wrong with waiting on csi frame end
[2018-07-30 15:24:13.087] [CameraTest] [error] JetsonCamera: Camera panic! Reset called
PowerServiceCore:handleRequests: timePassed = 528
Error: waitCsiFrameStart timeout guid 0
SCF: Error Timeout: VI SOF timed out! (in src/services/capture/CaptureServiceDeviceViCsi.cpp, function waitCsiFrameStart(), line 948)
VI Stream Id = 0 Virtual Channel = 0
************VI Debug Registers**********
VI_CSIMUX_STAT_FRAME_0 = 0x000000e2
VI_CSIMUX_FRAME_STATUS_0 = 0x00000000
VI_CFG_INTERRUPT_STATUS_0 = 0x3f000000
VI_ISPBUFA_ERROR_0 = 0x00000000
VI_FMLITE_ERROR_0 = 0x00000000
VI_NOTIFY_ERROR_0 = 0x00000000
*****************************************
CSI Stream Id = 0 Brick Id = 0
************CSI Debug Registers**********
CILA_INTR_STATUS_CILA[0x10400] = 0x00000089
CILB_INTR_STATUS_CILB[0x10c00] = 0x00000089
INTR_STATUS[0x100a4] = 0x00000000
INTR_STATUS[0x100a4] = 0x00000000
ERR_INTR_STATUS[0x100ac] = 0x00000000
ERROR_STATUS2VI_VC0[0x10094] = 0x00000000
ERROR_STATUS2VI_VC1[0x10098] = 0x00000000
ERROR_STATUS2VI_VC2[0x1009c] = 0x00000000
ERROR_STATUS2VI_VC3[0x100a0] = 0x00000000
*****************************************
Error: waitCsiFrameStart Something went wrong with waiting on frame start
SCF: Error Timeout: (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function waitCompletion(), line 368)
SCF: Error Timeout: (propagating from src/services/capture/CaptureServiceDevice.cpp, function pause(), line 873)
SCF: Error Timeout: (propagating from src/services/capture/CaptureServiceDevice.cpp, function handleCancelSourceRequests(), line 955)
SCF: Error Timeout: (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 983)
SCF: Error Timeout: (propagating from src/common/Utils.cpp, function workerThread(), line 114)
SCF: Error Timeout: Worker thread CaptureScheduler frameStart failed (in src/common/Utils.cpp, function workerThread(), line 131)
SCF: Error InvalidState: Capture Scheduler not running (in src/services/capture/CaptureServiceDevice.cpp, function addNewItemToSchedule(), line 841)
SCF: Error InvalidState: (propagating from src/services/capture/CaptureService.cpp, function addRequest(), line 739)
SCF: Error InvalidState: (propagating from src/components/stages/SensorCaptureStage.cpp, function doHandleRequest(), line 87)
SCF: Error InvalidState: (propagating from src/components/stages/OrderedStage.cpp, function doExecute(), line 158)
SCF: Error InvalidState: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 992)
Any ideas? I tried applying the kernel patches from this post: https://devtalk.nvidia.com/default/topic/1037809/jetson-tx2/jetpack3-2-1-tx2-csi-mipi-can-only-get-the-the-first-frame-of-image-but-tx1-works-fine-/, but nothing changed.