GMSL camera sync issue

[spshin]

Hello all.
We are currently encountering a synchronization issue while using GMSL cameras that support external trigger mode.
We have ported and are using cameras with external trigger mode.
However, when we run the preview using the command below,
the synchronization between the channels appears to be misaligned.

We are currently encountering a synchronization issue while using GMSL cameras that support external trigger mode.

🔧 System Setup
We have ported and are using cameras with external trigger mode. However, when we run the preview using the command below, the synchronization between the channels appears to be misaligned.

When using another application processor (AP) to encode and decode four channels from the same deserializer,
all four channels are perfectly synchronized.

The images below show the results when encoding and decoding:

4 channels from Deserializer A (highlighted in red)

2 channels from Deserializer B (highlighted in blue)

All channels were synchronized correctly when tested on a different AP,

which suggests that the synchronization problem may not be related to the deserializer or the cameras themselves.

I used below command to preview cameras
GST_DEBUG=4 gst-launch-1.0 \
nvcompositor name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=800 sink_0::height=600 \
sink_1::xpos=800 sink_1::ypos=0 sink_1::width=800 sink_1::height=600 \
sink_2::xpos=0 sink_2::ypos=600 sink_2::width=800 sink_2::height=600 \
sink_3::xpos=800 sink_3::ypos=600 sink_3::width=800 sink_3::height=600 \
! nvvidconv ! nveglglessink sync=true \
v4l2src device=/dev/video0 ! ‘video/x-raw, format=UYVY, width=1920, height=1200, io-mode=4’ \
! nvvidconv ! ‘video/x-raw(memory:NVMM), width=800, height=600’ ! queue ! comp.sink_0 \
v4l2src device=/dev/video1 ! ‘video/x-raw, format=UYVY, width=1920, height=1200, io-mode=4’ \
! nvvidconv ! ‘video/x-raw(memory:NVMM), width=800, height=600’ ! queue ! comp.sink_1 \
v4l2src device=/dev/video2 ! ‘video/x-raw, format=UYVY, width=1920, height=1200, io-mode=4’ \
! nvvidconv ! ‘video/x-raw(memory:NVMM), width=800, height=600’ ! queue ! comp.sink_2 \
v4l2src device=/dev/video3 ! ‘video/x-raw, format=UYVY, width=1920, height=1200, io-mode=4’ \
! nvvidconv ! ‘video/x-raw(memory:NVMM), width=800, height=600’ ! queue ! comp.sink_3

We appreciate your prompt response in advance.

[JerryChang]

hello spshin,

that’s due to you’re using four v4l2src for sending capture requests separately.
from the software level, it doesn’t send capture request at the same time.

there’s Argus sample, syncSensor, it create two streams, and set common settings to both streams, it can achieve software level synchronization. however, as you can see per Camera Architecture Stack, Argus and v4l2src they’ve went through different pipelines.

[spshin]

Hello JerryChang
Thank you for prompt response.!
Could you let me know where " there’s Argus sample, syncSensor " is ..
We appreciate your prompt response in advance.

[JerryChang]

hello spshin,

you may download MMAPI package, $ sudo apt install nvidia-l4t-jetson-multimedia-api
here’s path of the sample application, /usr/src/jetson_multimedia_api/argus/samples/syncSensor

as mentioned,
it looks you used cameras with UYVY formats, however, Argus only works with Bayer sensors (i.e. Raw contents).

[spshin]

Hello JerryChang
Thank you for very fast prompt response .!!!
Is there any way to support UYVY format in Argus?
I always appreciate your prompt response .

[JerryChang]

hello spshin,

@spshin

Is there any way to support UYVY format in Argus?

unfortunately not.

since you’re running with v4l2src, which doesn’t support multiple sources per single session.
we’ve never tested it before, you may need to have implementation.
for instance, please refer to /usr/src/jetson_multimedia_api/samples/12_v4l2_camera_cuda, which use standard v4l2 IOCTL to fetch frames, you should have implementation to extend it as four threads, and making 1st camera as primary threads to send capture request to all of your camera simultaneously.