Incorrect depth image generated by camera in IsaacSim


In my project, I was trying to send camera data (rgb and depth) into ROS, and then convert it to point cloud via this node
While using it with gazebo, it works out perfectly:
Original pointcloud, published by simulated camera:

Estimated pointcloud from depth image:

Both pointclouds in the same time:

As can be seen it’s pretty the same, what means, that this node works, as expected. But when I’m trying to use same node with data from Isaac Sim, point cloud look distorted and shifted:

BTW, here is depth image from GazeboSim:

And from Isaac Sim:

Looks like CamerInfo, or Depth Image itself is wrong in Isaac Sim. Is it planned by the Isaac Sim team, to implement point cloud publishing from a simulated camera? Or maybe you have a solution, how to fix this problem?

Looks like a 90 degree rotation clockwise is missing for the bottle environment. Can you see if that helps? Are you using uint16 depth or float in Gazebo/input into depth_image_proc? I’ll try to repro. Thanks.

The distortion was a bug in sim where the vertical aperture used in the camera intrinsics was not computed correctly. Its fixed for the next release.

To work around this temporarily you can compute the correct vertical aperture value and enter it in the USD Camera Prim:

vertical_aperture = image_height / image_width * horizontal_aperture

Where the horizontal_aperture value is what is currently set in the camera prim in the stage. If no camera prim exists, create a new one and make it active in the viewport window.