CAM0 Port Latency (IMX219 CAM0 lags behind CAM1)

[ocheng]

I am using an IMX219 stereo camera connected to the CAM0 and CAM1 ports. I am using GStreamer and OpenCV to display the video feeds side-by-side. Whichever camera is connected to CAM0 lags noticeably behind CAM1. Any suggestions for how to sync the video feeds?

I have tried switching which cameras are plugged into which ports and switching the displays of the video feeds, but no luck. I am running Jetpack 6.0-b52 from the SDKManager, using a WaveShare IMX219 Binocular Camera Module and GStreamer 1.20.3.

Here is the code:

gst0 = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=15/1 ! nvvidconv ! appsink -v"
gst1 = "nvarguscamerasrc sensor-id=1 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=15/1 ! nvvidconv ! appsink -v"

cam0 = cv.VideoCapture(gst0, cv.CAP_GSTREAMER)
cam1 = cv.VideoCapture(gst1, cv.CAP_GSTREAMER)

while True:
    grab0 = cam0.grab()
    grab1 = cam1.grab()

    ret0, frame0 = cam0.retrieve()
    ret1, frame1 = cam1.retrieve()

    imOut = np.hstack((frame0, frame1))
    cv.imshow(f"FPS{fps}, cam0 vs. cam1", imOut)

    key = cv.waitKey(1)
    if key == ord('q'):
        break

cv.destroyAllWindows()
cam0.release()
cam1.release()

Thank you in advance!

[JerryChang]

hello ocheng,

please test with synchronized applications. for example, syncStereo

[ocheng]

Thanks @JerryChang

I installed and ran syncStereo (and related packages, symlinks) and got the following output:

Executing Argus Sample: argus_syncstereo
Argus Version: 0.98.3 (multi-process)
getSensorPlacement for sensor i 0 is SENSOR_PLACEMENT_FRONT_OR_TOP_O
getSensorPlacement for sensor i 1 is SENSOR_PLACEMENT_REAR_OR_BOTTOM
PRODUCER: Done -- exiting.

However, when I run my same script again, I still see the camera feeds are not synced. Is there anything else I can do to sync the feeds?

[JerryChang]

hello ocheng,

you should refer to sample app to implement the synchronization mechanism.

[ocheng]

Using CSI-Camera/dual_camera.py at master · JetsonHacksNano/CSI-Camera · GitHub I was able to sync my cameras without needing such a low-level implementation. Turns out I just needed to run the cameras in different threads.