ROS2 | Camera info's distortion_model is invalid


The published camera_info using the ROS2 Camera Helper node, has pinhole under its distortion_model:

This is an unsupported distortion model, which causes exceptions such as: Cannot call rectifyPoint when distortion is unknown.

Under CameraInfos documentation it references the supported distortion models, which are:

Hi - Sorry for the delay in the response. Let us know if you still having this issue with the latest Isaac Sim 2022.2.1 release.

@rthaker still having this issue in 2022.2.1:

[depthimage_to_laserscan_node-2] [ERROR] [1685898103.609667495] [depthimage_to_laserscan_front]: Could not convert depth image to laserscan: Cannot call rectifyPoint when distortion is unknown.

Camera info’s topic echo:

    sec: 32
    nanosec: 366668354
  frame_id: trunk1_camera_link
height: 720
width: 1280
distortion_model: pinhole

Hi I am facing the same issue :( @omers were you able to solve it?

Hi @omers and @jasonjabbour . I understand that this error could be a result of a call to the rectifyPoint, that allows to get rectified pixel coordinates from unrectified ones.

The issue is, image is already undistorted one, so this call shouldn’t happen in the first place (rectifyPoint should return the same point in that case). If possible, please try to comment out that call for the case, when the distortion coefficient parameter array have zero length (pinhole camera case), in the ROS node.

On the ROS interface, yes, it is likely that logic of the Isaac Sim to supply the ‘pinhole’ in the distortion type for undistorted images is wrong. And the parameter should be omitted altogether, instead of specifying ‘pinhole’.

The workaround for this is to check, if the distortion coefficient array length is zero (many ROS nodes use that check to determine that the image comes from the pinhole camera). Or to check explicitly that ‘pinhole’ is specified.

We have a bit more flexible support for distortions upcoming, that allows to specify distortion model communicated to ROS separately from the underlying RTX rendering model.