Argus - Syncing multiple capture sessions

hello mmorsy,

as I mentioned in comment #2,
you may need to contact Leopard team to have hardware solution for camera synchronization.
thanks

I contacted them and I’m waiting for a response. Do you have any idea why a capture session can’t handle more than 3 cameras? If I fix that, my issue might be resolved.

hello mmorsy,

may I know the sensor resolution and also frame rate of your use-case.
had you tried to lower the bandwidth in a single capture session, is it still cannot handle more than 3-cameras?
thanks

I’ve tried it with multiple resolutions (3840x2160, 1920x1080). I also tried frame rates between 1 to 30 fps. I have the same issue every time I try to capture with more than 3 cameras.

hello mmorsy,

we had verify 6-camera use-case with our reference camera board, the running resolution is 2592x1944@30-fps. it works as expect with Argus camera, multi-session.
could you please share the kernel message when the failure reproduced.
thanks

I was always able to capture 6 cameras using multi-session. The issue is that they’re not synchronized. That’s why I am trying to capture them using a single session (then they might be synchronized). Is that possible? Right now I can only capture 3 cameras.

Also, I’ve been getting these error messages when my program terminates.

(Argus) Objects still active during exit: [CameraProvider (0x1098e20): refs: 1, cref: 0]
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=7 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=6 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=6 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2

Do you know what this means or how to fix it?

Thank you

hello mmorsy,

  1. Right now I can only capture 3 cameras.
    you had also confirm these 3-cameras were synchronized, right?

  2. NvEglHandle messages is trying to deinit global internal state.
    could you please have another try to launch six cameras in single session without EGL, please have a try to save the output result into files.
    thanks

  1. I am using IArgusCaptureMetadata with each frame to get their timestamps and I’m finding the time difference between each camera, which turns out to be 0.000. I am assuming these timestamps are specific to each camera’s frame and not just repetitions of the first camera’s frame. So yes, they appear synchronized.

  2. I tried to launch 6 cameras and I get these error messages:

SCF: Error BadValue: device not found (in src/common/DeviceRegistry.h, function assign(), line 142)
SCF: Error BadValue: (propagating from src/api/CameraDriver.cpp, function createSession(), line 503)
(Argus) Error BadValue: (propagating from src/api/CaptureSessionImpl.cpp, function initialize(), line 116)
(Argus) Error BadValue: (propagating from src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 234)

The same happens when I try 5. But when I try 4, the first camera launches but not the other 3.

When I use a single session, I don’t get the NvEglHandle messages. I only get it with multiple sessions. So how would I fix it?

Thank you

hello mmorsy,

let’s have clarification as below,
[case-1] multiple sensors per multi sessions

launch each camera sensor with different capture request, they’re not synchronized.

[case-2] multiple sensors per single sessions

using a single capture request to launch several camera frames at once.

argus_camera and MMAPI examples were based-on case-1, you’re also able to capture images from multiple cameras without issues.
you got failures with case-2, and it’s now only able to launch 3-cameras.

FYI,
here’s source limitation of scf session, you’re not able to launch all 6-cameras in a single session,
thanks

Hi mmorsy,

Our 6 x IMX274 camera kit doesn’t support synchronization function since IMX274 doesn’t support slave mode.
We have another camera kit LI-TX1-CB-6CAM + IMX185 camera. This camera kit supports synchronization function. We set the IMX185 to salve mode in driver and use the FPGA (on LI-TX1-CB-6CAM) to send trigger singles to 6 cameras.

IMX185-MIPI: https://leopardimaging.com/product/li-imx185-mipi-m12/
LI-TX1-CB-6CAM: https://leopardimaging.com/product/li-tx1-cb-6cam/

I am aware of the IMX185, but I need better resolution. Will there be an update to the IMX274 driver so it supports synchronization?

Thank you

hello mmorsy,

could you please have a try to replace couple of binaries, libscf.so and libargus.so
please back-up and replace them with the attachment, thanks
Topic1039183_Oct02.7z (1.91 MB)

Where can I find libscf.so and libargus.so? In what directory?

hello mmorsy,

all binaries were located under

/usr/lib/aarch64-linux-gnu/tegra/

please back-up before overwrite them. you may restart camera services or simply perform a warm-reboot to make the change takes effect.
thanks

Hi JerryChang
I now have the same problem with you, all my imx274 work in master mode(csi/mipi mode),could you tell me how to sync the sensors?
Should I change all the sensors to salve mode? and pull all their XVS/XHS to the one signal from fpga?
Thanks very much for your reply.

hello fm2012,

couple of questions need your feedback.

  1. may I know how may camera senors you would like to synchronize for your use-case.
  2. could you please share what’s your device, as Leopard team mentioned in comment #13, their board don’t support synchronization function.

Hi JerryChang
1)I use 4 camera sensor(IMX274).
2)Actually I use my own platform,I plan to use FPGA to control or capture 4 IMX274 sensor,and stitch them.

I want 4 sensor run csi/mipi mode,but it is master mode.The slave mode is lvds.

hello fm2012,

for hardware, you may contact your sensor vendor about how to configure sensor mode for synchronization use-case.
for the software approach, you may refer to Argus sample for dual camera synchronization application. thanks

<top>/tegra_multimedia_api/argus/samples/syncSensor

Thanks.
Actually,Sony FAE does not care about small companies like ours.