Multiple RectifyNodes not functional with Multiple ArgusMonoNodes

Robotics & Edge Computing Robotics - Isaac Isaac ROS
, , ,
1

Jetpack 6.0/6.1
x2 IMX219
x2 isaac_ros_argus_camera ArgusMonoNodes
x2 isaac_ros_image_pipeline RectifyNodes

While able to access any CSI connected IMX219s and stream multiple simultaneously via gst on Jetpack 6.0/6.1, opening multiple isaac_ros_argus_camera ArgusMonoNodes simultaneously with different cameras results in strange behavior from isaac_ros_image_pipeline RectifyNodes. When attempting to rectify the images for example, one stream (random if steam no.1or stream no.2) will be rectified perfectly, while the other results in an empty image. Sometimes, however, the second stream appears to utilize nonsensical intrinsic camera parameters (while other times, rectifying completely normally). These are low-distortion lenses, and the camera calibrations are accurate, and DO result in perfectly rectified images when applied correctly (plum_bob models).

Both streams are running in their own ComposableNodeContainers, and the ComposableNodes are utilizing separate namespaces (cam_one or cam_two).

Pictured here are the rare times the second stream results in a non-empty image with nonsensical rectifications (vs. completely empty):

rectify1
rectify11640×1232 352 KB

rectify2
rectify21640×1232 50.1 KB

Any insights greatly appreciated.

2

Hi @Calvin-InDro

Looking at your image, it seems like the camera is not calibrated.

Check online to start the correct calibration, an example here: ArduCam 8MP IMX219 Red Tint - #5 by fazevedo

Best,
Raffaello

3

Hi Raffaello, thanks for your attention on this.

The camera is calibrated, and the calibration files are correct. Pictured below is a correctly rectified image using the exact same camera calibration file. There is no IR block (purple hue). Again, these are low-distortion lenses so the warp is minimal, you can faintly see the curved black borders.

cam_down_rect
cam_down_rect1640×1232 265 KB

When the images are rectified correctly, the output images are perfect. The intermittent behavior I am referencing produces the warped images (from the same correct calibration file) when trying to rectify multiple camera streams.

The additional problem is that it is random which camera works properly.

4

@Raffaello you are a popular person on the forums, but I wanted to offer this clarification on the situation. Definitely not a calibration issue, maybe memory boundary problems?

5

Hi @Calvin-InDro

Than you for this follow up.

Let me know,
Raffaello

6

Hi Raffaello,

Thanks for your response.
I am also experiencing this issue with a single pipeline, and it appears to work randomly (but sporadically). The system is not under any other load. The image_raw topics are publishing fine from argus_camera.

With no changes at all to code, configuraitons or file paths, I either I get:
a) a perfectly rectified stream at high fps
b) an empty image
c) a completely warped image (like I described earlier was occuring with x2 simultaneous pipelines)

To your points:

  1. The camera_info_url is set up and loading the correct config. When echoing the camera_info topic, it reflects this data correctly.
  2. rqt_graph attached. Looks OK to me maybe I am missing something.

Please find attached (filenames are altered for attachment):

1 Like
7

I am also experiencing this issue intermittently when trying to rectify images from 4x IMX296 cameras.