JetPack 3.3 - multiple camera with multi-stream capture does not work

Hi,

I am writing an application that eventually will capture multiple streams from each camera for up to 5 cameras concurrently.
I am currently experimenting with the following:

  1. nVidia Jetson TX2 EVM with 3-port Leopard Imaging adapter board
  2. an IMX274 CSI camera and an IMX185 CSI camera connected to the Leopard Imaging adapter board

Consulting the example and the <13_multi_camera> example from the tegra_multimedia_api/samples directory, I was able to get the following to work:

  • running IMX274 (enumerated as camera device 1) alone with 4 streams works fine, able to get 30 fps/stream
  • running IMX185 (enumerated as camera device 0) alone with 2 streams works fine, able to get 30 fps/stream
  • running IMX274 with 1 stream and IMX185 with 1 stream concurrently in 1 single process (i.e. same producer thread) works fine

However, I was not able to get the following to work using the same consumer configurations as above:

  1. running IMX274 and IMX185 producer in separate processes concurrently (i.e. one process captures camera 0 and another process captures camera 1) will not work, regardless of consumer streams for each camera/producer.
  2. IMX274 process reports:
    ===== MSENC blits (mode: 1) into tiled surfaces =====
    SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
    Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc
    Error: waitCsiFrameStart timeout guid 0
    SCF: Error Timeout: VI SOF timed out! (in src/services/capture/CaptureServiceDeviceViCsi.cpp, function waitCsiFrameStart(), line 948)
    VI Stream Id = 0 Virtual Channel = 0
    ************VI Debug Registers**********
    VI_CSIMUX_STAT_FRAME_0	 = 0x00000000
    VI_CSIMUX_FRAME_STATUS_0	 = 0x00010000
    VI_CFG_INTERRUPT_STATUS_0	 = 0x3f000000
    VI_ISPBUFA_ERROR_0	 = 0x00000000
    VI_FMLITE_ERROR_0	 = 0x00000000
    VI_NOTIFY_ERROR_0	 = 0x00000000
    *****************************************
    CSI Stream Id = 0 Brick Id = 0
    ************CSI Debug Registers**********
    CILA_INTR_STATUS_CILA[0x10400]	 = 0x00000089
    CILB_INTR_STATUS_CILB[0x10c00]	 = 0x00000088
    INTR_STATUS[0x100a4]	 = 0x00000000
    INTR_STATUS[0x100a4]	 = 0x00000000
    ERR_INTR_STATUS[0x100ac]	 = 0x00000000
    ERROR_STATUS2VI_VC0[0x10094]	 = 0x00000000
    ERROR_STATUS2VI_VC1[0x10098]	 = 0x00000000
    ERROR_STATUS2VI_VC2[0x1009c]	 = 0x00000000
    ERROR_STATUS2VI_VC3[0x100a0]	 = 0x00000000
    *****************************************
    launchCC abort cc 157 session 0
    SCF: Error Timeout:  (propagating from src/api/Session.cpp, function capture(), line 830)
    (Argus) Error Timeout:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
    ...
    

    IMX185 process reports:

    SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
    Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc
    launchCC abort cc 104 session 4
    SCF: Error Timeout:  (propagating from src/api/Session.cpp, function capture(), line 830)
    (Argus) Error Timeout:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
    launchCC abort cc 105 session 4
    SCF: Error Timeout:  (propagating from src/api/Session.cpp, function capture(), line 830)
    (Argus) Error Timeout:  (propagating from src/api/ScfCaptureThread.cpp, function run(), line 109)
    launchCC abort cc 106 session 4(Argus) Error InvalidState: MetadataResult callback for unknown capture. (in src/api/CaptureSessionImpl.cpp, function metadataResult(), line 705)
    ...
    
  3. Using 2 capture sessions (one for camera 0 and one for camera 1) in one process (similar to <13_multi_camera> producer setup) and capturing more than 1 streams from each camera will fail with similar errors.
  4. ICaptureSession interface can no longer be acquired when multiple camera capture sessions are in progress using repeat capture.

According to what I have read so far, it seems capture sessions for a given camera must be in the same process but it sort of implies independent processes that contain exclusive camera resource can be run concurrently? Although I was not able to get this to work in all the different scenarios I have tried. Is this a valid assumption or a permissible operation mode?

Is there a valid path to capture multiple streams from multiple cameras concurrently? If so, how should I go about reaching this goal?

Best Regards,
Joseph

HI Jying,
I believe you have already sound knowledge on how the libargus framework is. Have you checked all the samples provided in the tegra_mutlimedia_api?
Kindly check the ‘multiSensor’ and ‘multiStream’ samples provided in tegra_multimedia_api/argus/samples directory.

To give a short insight: Check the multiSensor code to create two capture sessions for device 0 and device 1. Once the capture sessions are created, try creating two streams on each of the capture sessions using ‘multistream’ as reference.

Hi, waisskharni.sm,

Thank you for your response.
I went ahead and rebuild my program based on argus samples (multiSensor and multiStream) as you have suggested. I was able to capture multiple streams from both cameras concurrently by doing this.
Previously when I ran into trouble is when I followed the samples in the multimedia samples directory (tegra_multimedia_api/samples) instead of the argus samples directory (tegra_multimedia_api/argus/samples).
It turns out my program is quite similar following both samples but the main difference is the compile flow and the packages. Not really sure why it works under argus but not the multimedia samples.

I am really thankful for your help.

Regards,
Joseph