Using Argus with Camera Hardware Sync

Hello,

We have gotten hardware sync working for our IMX185 cameras but I am running into issues when Argus tries to shutdown.

My code is based off of the syncSensor sample with the CaptureSession created from a vector with both camera devices. The master camera is at index 0, and the slave is at index 1.

In dmesg you can see that the master camera (index 0) starts up first followed by the slave:

[ 4001.113262] imx185 2-001a: imx185_power_on: power on
[ 4001.119219] imx185 2-001a: imx185_set_frame_rate: val: 251658240, , frame_length: 1125
[ 4001.129997] imx185 2-001a: imx185_set_exposure: val: 69905
[ 4001.135674] imx185 2-001a: imx185_set_coarse_time: coarse1:1124, shs1:0, FL:1125
[ 4001.172647] imx185 2-001a: imx185_s_stream++ enable 1
...
[ 4001.495134] imx185 1-001a: imx185_power_on: power on
[ 4001.500726] imx185 1-001a: imx185_set_frame_rate: val: 251658240, , frame_length: 1125
[ 4001.509333] imx185 1-001a: imx185_set_exposure: val: 69905
[ 4001.514977] imx185 1-001a: imx185_set_coarse_time: coarse1:1124, shs1:0, FL:1125
[ 4001.570740] imx185 1-001a: imx185_s_stream++ enable 1

Everything works great until I try to stop the streams. As soon as the master is disabled the slave stops sending (since it is no longer being triggered).

[ 4149.377361] imx185 2-001a: imx185_s_stream++ enable 0

This causes a timeout in Argus waiting for the slave’s stream to end:

Error: waitCsiFrameEnd timeout guid 1
VI Stream Id = 2 Virtual Channel = 0
************VI Debug Registers**********
VI_CSIMUX_STAT_FRAME_8	 = 0x000000b6
VI_CSIMUX_FRAME_STATUS_0	 = 0x00000000
VI_CFG_INTERRUPT_STATUS_0	 = 0x3f000000
VI_ISPBUFA_ERROR_0	 = 0x00000000
VI_FMLITE_ERROR_0	 = 0x00000000
VI_NOTIFY_ERROR_0	 = 0x00000000
*****************************************
CSI Stream Id = 2 Brick Id = 1
************CSI Debug Registers**********
CILA_INTR_STATUS_CILA[0x20400]	 = 0x00000089
CILB_INTR_STATUS_CILB[0x20c00]	 = 0x00000088
INTR_STATUS[0x200a4]	 = 0x00000000
INTR_STATUS[0x200a4]	 = 0x00000000
ERR_INTR_STATUS[0x200ac]	 = 0x00000000
ERROR_STATUS2VI_VC0[0x20094]	 = 0x00000000
ERROR_STATUS2VI_VC1[0x20098]	 = 0x00000000
ERROR_STATUS2VI_VC2[0x2009c]	 = 0x00000000
ERROR_STATUS2VI_VC3[0x200a0]	 = 0x00000000
*****************************************
Error: waitCsiFrameEnd Something went wrong with waiting on csi frame end
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function waitCompletion(), line 368)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function pause(), line 873)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function handleCancelSourceRequests(), line 955)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 983)
SCF: Error Timeout:  (propagating from src/common/Utils.cpp, function workerThread(), line 114)
SCF: Error Timeout: Worker thread CaptureScheduler frameStart failed (in src/common/Utils.cpp, function workerThread(), line 131)
Error: waitCsiFrameStart timeout guid 1
SCF: Error Timeout: VI SOF timed out! (in src/services/capture/CaptureServiceDeviceViCsi.cpp, function waitCsiFrameStart(), line 948)
VI Stream Id = 2 Virtual Channel = 0
************VI Debug Registers**********
VI_CSIMUX_STAT_FRAME_8	 = 0x000000b6
VI_CSIMUX_FRAME_STATUS_0	 = 0x00000000
VI_CFG_INTERRUPT_STATUS_0	 = 0x3f000000
VI_ISPBUFA_ERROR_0	 = 0x00000000
VI_FMLITE_ERROR_0	 = 0x00000000
VI_NOTIFY_ERROR_0	 = 0x00000000
*****************************************
CSI Stream Id = 2 Brick Id = 1
************CSI Debug Registers**********
CILA_INTR_STATUS_CILA[0x20400]	 = 0x00000089
CILB_INTR_STATUS_CILB[0x20c00]	 = 0x00000088
INTR_STATUS[0x200a4]	 = 0x00000000
INTR_STATUS[0x200a4]	 = 0x00000000
ERR_INTR_STATUS[0x200ac]	 = 0x00000000
ERROR_STATUS2VI_VC0[0x20094]	 = 0x00000000
ERROR_STATUS2VI_VC1[0x20098]	 = 0x00000000
ERROR_STATUS2VI_VC2[0x2009c]	 = 0x00000000
ERROR_STATUS2VI_VC3[0x200a0]	 = 0x00000000
*****************************************
Error: waitCsiFrameStart Something went wrong with waiting on frame start

With a sync’d CaptureSession is there a way to specify that index 1 should be disabled before index 0? I expect I could just create two separate capture sessions, but that would not be as clean.

This is my shutdown sequence:

// Stop request
m_request.reset();

// Stop capture
ICaptureSession *iCaptureSession = interface_cast<ICaptureSession>(m_captureSession.get());
if(iCaptureSession) {
    iCaptureSession->stopRepeat();
    iCaptureSession->waitForIdle(2e9);
}

// Join capture thread
if(m_thread) {
    if(pthread_join(m_thread, nullptr) != 0)
        throw std::runtime_error("Failed to join capture thread");
}

// Reset slave
m_stream[1].reset();
m_consumer[1].reset();

// Reset master
m_stream[0].reset();
m_consumer[0].reset();

// Reset Argus
m_streamSettings.reset();
m_captureSession.reset();
m_cameraProvider.reset();

I wasn’t thinking, I just had to reverse my indices so that the slave is at index 0. Thus is both starts and stops before the master.

Hi Atrer,
May I know if you faced any issues by using this method? (enabling slave device before master)
Has your setup run without any issues?
Won’t this same setup cause issues when there are more than two cameras? Will the slave device producer hang since no master device is enabled yet?

If I could suggest an alternative, you could create two capture sessions and use repeatBurst method of implementation to acquire frames. Since the cameras are hardware synchronized, I believe this would be a cleaner way of implementation.

There seem to be delays inherent in Argus or the ISP which can cause the captures to deviate even with hardware sync. I do not know if doing burst captures would help. Ultimately I did end up doing two independent capture sessions with metadata enabled so I could extract the hardware timestamp and compare them on a frame by frame basis. Then I discard any pairs with timestamps that are too different.

As to your question, yes the slave producer does hang until the master begins, but Argus allows several frame timeouts before it quits. I doubt it would cause issues even with more than two cameras, but you’d have to test it, we only ever plan to use two cameras.

More info in my thread here:
https://devtalk.nvidia.com/default/topic/1061467/jetson-tx2/disparity-help