Multiple RectifyNodes not functional with Multiple ArgusMonoNodes

[Calvin-InDro]

Jetpack 6.0/6.1
x2 IMX219
x2 isaac_ros_argus_camera ArgusMonoNodes
x2 isaac_ros_image_pipeline RectifyNodes

While able to access any CSI connected IMX219s and stream multiple simultaneously via gst on Jetpack 6.0/6.1, opening multiple isaac_ros_argus_camera ArgusMonoNodes simultaneously with different cameras results in strange behavior from isaac_ros_image_pipeline RectifyNodes. When attempting to rectify the images for example, one stream (random if steam no.1or stream no.2) will be rectified perfectly, while the other results in an empty image. Sometimes, however, the second stream appears to utilize nonsensical intrinsic camera parameters (while other times, rectifying completely normally). These are low-distortion lenses, and the camera calibrations are accurate, and DO result in perfectly rectified images when applied correctly (plum_bob models).

Both streams are running in their own ComposableNodeContainers, and the ComposableNodes are utilizing separate namespaces (cam_one or cam_two).

Pictured here are the rare times the second stream results in a non-empty image with nonsensical rectifications (vs. completely empty):

Any insights greatly appreciated.

[Raffaello]

Hi @Calvin-InDro

Looking at your image, it seems like the camera is not calibrated.

Check online to start the correct calibration, an example here: ArduCam 8MP IMX219 Red Tint - #5 by fazevedo

Best,
Raffaello

[Calvin-InDro]

Hi Raffaello, thanks for your attention on this.

The camera is calibrated, and the calibration files are correct. Pictured below is a correctly rectified image using the exact same camera calibration file. There is no IR block (purple hue). Again, these are low-distortion lenses so the warp is minimal, you can faintly see the curved black borders.

When the images are rectified correctly, the output images are perfect. The intermittent behavior I am referencing produces the warped images (from the same correct calibration file) when trying to rectify multiple camera streams.

The additional problem is that it is random which camera works properly.